2023-10-08T08:51:34.269 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-08T08:51:34.270 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} --verbose --owner scheduled_pdonnell@teuthology --name pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/orig.config.yaml 2023-10-08T08:51:34.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T08:51:34.434 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770 branch: wip-batrick-testing-20231007.154158 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7417770' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20231007.154158 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231007.154158 sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8022 sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de sleep_before_teardown: 0 subset: 128/256 suite: fs suite_branch: wip-batrick-testing-20231007.154158 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_20:01:29 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-08T08:51:34.434 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa; will attempt to use it 2023-10-08T08:51:34.435 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks 2023-10-08T08:51:34.435 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-08T08:51:34.437 INFO:teuthology.task.internal:Checking packages... 2023-10-08T08:51:34.459 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de' 2023-10-08T08:51:34.459 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-08T08:51:34.459 INFO:teuthology.packaging:ref: None 2023-10-08T08:51:34.459 INFO:teuthology.packaging:tag: None 2023-10-08T08:51:34.460 INFO:teuthology.packaging:branch: wip-batrick-testing-20231007.154158 2023-10-08T08:51:34.460 INFO:teuthology.packaging:sha1: 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T08:51:34.460 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20231007.154158 2023-10-08T08:51:34.685 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6639.g7b3143ba 2023-10-08T08:51:34.685 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-08T08:51:34.692 INFO:teuthology.task.internal:no buildpackages task found 2023-10-08T08:51:34.692 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-08T08:51:34.703 INFO:teuthology.task.internal:Saving configuration 2023-10-08T08:51:34.725 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-08T08:51:34.737 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T08:51:34.762 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 08:44:51.684378', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T08:51:34.782 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 08:44:51.683179', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T08:51:34.804 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 08:44:51.681789', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T08:51:34.804 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-08T08:51:34.813 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-08T08:51:34.813 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-08T08:51:34.813 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-08T08:51:34.813 INFO:teuthology.run_tasks:Running task console_log... 2023-10-08T08:51:34.922 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ca6890f70>, signals=[15]) 2023-10-08T08:51:34.922 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-08T08:51:34.931 INFO:teuthology.task.internal:Opening connections... 2023-10-08T08:51:34.931 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T08:51:34.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T08:51:35.000 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2023-10-08T08:51:35.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T08:51:35.069 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T08:51:35.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T08:51:35.136 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-08T08:51:35.146 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2023-10-08T08:51:35.167 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2023-10-08T08:51:35.167 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2023-10-08T08:51:35.227 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T08:51:35.227 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T08:51:35.227 INFO:teuthology.orchestra.run.smithi049.stdout:ID="rhel" 2023-10-08T08:51:35.227 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="fedora" 2023-10-08T08:51:35.227 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="8.6" 2023-10-08T08:51:35.227 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el8" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T08:51:35.228 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T08:51:35.229 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T08:51:35.229 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2023-10-08T08:51:35.252 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-10-08T08:51:35.272 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-10-08T08:51:35.272 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-10-08T08:51:35.331 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T08:51:35.331 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T08:51:35.331 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2023-10-08T08:51:35.332 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2023-10-08T08:51:35.332 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.6" 2023-10-08T08:51:35.332 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2023-10-08T08:51:35.332 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T08:51:35.332 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-10-08T08:51:35.332 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T08:51:35.333 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T08:51:35.333 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T08:51:35.333 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T08:51:35.333 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T08:51:35.333 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T08:51:35.333 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T08:51:35.333 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T08:51:35.334 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T08:51:35.334 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-10-08T08:51:35.360 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2023-10-08T08:51:35.380 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2023-10-08T08:51:35.381 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2023-10-08T08:51:35.440 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T08:51:35.440 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="8.6 (Ootpa)" 2023-10-08T08:51:35.440 INFO:teuthology.orchestra.run.smithi184.stdout:ID="rhel" 2023-10-08T08:51:35.440 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="fedora" 2023-10-08T08:51:35.440 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="8.6" 2023-10-08T08:51:35.440 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el8" 2023-10-08T08:51:35.440 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T08:51:35.441 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-08T08:51:35.442 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2023-10-08T08:51:35.468 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-08T08:51:35.480 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-08T08:51:35.493 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-08T08:51:35.493 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2023-10-08T08:51:35.495 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2023-10-08T08:51:35.498 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2023-10-08T08:51:35.516 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-08T08:51:35.527 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-08T08:51:35.527 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2023-10-08T08:51:35.555 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2023-10-08T08:51:35.557 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2023-10-08T08:51:35.585 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-08T08:51:35.673 INFO:teuthology.run_tasks:Running task kernel... 2023-10-08T08:51:35.690 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-08T08:51:35.690 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-10-08T08:51:35.690 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-10-08T08:51:35.691 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T08:51:35.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T08:51:35.711 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2023-10-08T08:51:35.771 INFO:teuthology.orchestra.run.smithi049.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-08T08:51:35.771 INFO:teuthology.task.kernel:Running kernel on smithi049: 4.18.0-372.9.1.el8.x86_64 2023-10-08T08:51:35.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2023-10-08T08:51:37.016 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-10-08T08:51:37.017 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2023-10-08T08:51:38.470 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-10-08T08:51:39.171 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-10-08T08:51:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 407 kB/s | 24 kB 00:00 2023-10-08T08:51:43.582 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-10-08T08:51:43.637 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2023-10-08T08:51:43.638 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2023-10-08T08:51:43.638 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2023-10-08T08:51:43.760 DEBUG:teuthology.orchestra.run.smithi049:> echo no | sudo yum reinstall kernel || true 2023-10-08T08:51:44.108 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-10-08T08:51:44.108 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2023-10-08T08:51:44.389 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:02 ago on Sun 08 Oct 2023 08:51:42 AM UTC. 2023-10-08T08:51:44.433 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-08T08:51:44.466 INFO:teuthology.orchestra.run.smithi049.stderr:Error: No packages marked for reinstall. 2023-10-08T08:51:44.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum reinstall -y kernel || true 2023-10-08T08:51:44.864 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-10-08T08:51:44.864 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2023-10-08T08:51:45.141 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:03 ago on Sun 08 Oct 2023 08:51:42 AM UTC. 2023-10-08T08:51:45.185 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-10-08T08:51:45.221 INFO:teuthology.orchestra.run.smithi049.stderr:Error: No packages marked for reinstall. 2023-10-08T08:51:45.272 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2023-10-08T08:51:45.355 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-10-08T08:51:45.356 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-10-08T08:51:45.356 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-10-08T08:51:45.356 DEBUG:teuthology.parallel:result is None 2023-10-08T08:51:45.356 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-08T08:51:45.365 INFO:teuthology.task.internal:Creating test directory... 2023-10-08T08:51:45.365 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T08:51:45.368 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T08:51:45.370 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T08:51:45.390 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-08T08:51:45.421 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-08T08:51:45.438 INFO:teuthology.task.internal:Creating archive directory... 2023-10-08T08:51:45.438 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T08:51:45.440 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T08:51:45.442 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T08:51:45.479 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-08T08:51:45.487 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-08T08:51:45.487 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T08:51:45.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T08:51:45.529 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T08:51:45.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T08:51:45.548 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T08:51:45.550 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T08:51:45.572 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T08:51:45.588 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T08:51:45.606 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T08:51:45.622 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T08:51:45.638 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T08:51:45.644 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T08:51:45.654 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T08:51:45.656 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T08:51:45.665 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T08:51:45.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T08:51:45.667 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T08:51:45.669 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T08:51:45.698 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T08:51:45.711 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T08:51:45.712 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T08:51:45.735 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T08:51:45.738 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T08:51:45.757 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T08:51:45.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T08:51:45.882 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T08:51:45.965 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T08:51:46.009 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T08:51:46.009 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T08:51:46.078 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T08:51:46.128 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T08:51:46.178 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T08:51:46.265 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T08:51:46.310 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T08:51:46.310 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T08:51:46.380 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T08:51:46.429 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T08:51:46.488 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T08:51:46.535 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T08:51:46.578 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T08:51:46.578 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T08:51:46.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2023-10-08T08:51:46.650 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2023-10-08T08:51:46.653 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2023-10-08T08:51:46.706 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T08:51:46.719 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T08:51:46.745 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T08:51:47.260 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T08:51:47.274 INFO:teuthology.task.internal:Starting timer... 2023-10-08T08:51:47.274 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T08:51:47.290 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T08:51:47.305 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-08T08:51:47.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2023-10-08T08:51:47.398 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2023-10-08T08:51:47.400 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2023-10-08T08:51:47.457 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2023-10-08T08:51:47.458 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2023-10-08T08:51:47.521 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: 2023-10-08T08:51:47.523 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T08:51:47.523 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2023-10-08T08:51:47.544 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2023-10-08T08:51:47.544 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2023-10-08T08:51:47.568 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2023-10-08T08:51:47.569 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2023-10-08T08:51:47.592 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2023-10-08T08:51:47.592 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2023-10-08T08:51:47.593 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T08:51:47.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T08:51:47.621 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T08:51:47.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T08:51:47.649 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T08:51:47.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T08:51:47.677 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T08:51:47.677 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2023-10-08T08:51:47.704 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2023-10-08T08:51:47.731 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2023-10-08T08:51:47.758 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T08:51:47.770 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-08T08:51:47.770 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T08:51:47.787 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T08:51:47.788 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi132.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T09:01:26.769 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2023-10-08T09:01:26.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-10-08T09:01:26.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:01:26.841 DEBUG:teuthology.orchestra.run.smithi049:> true 2023-10-08T09:01:26.913 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2023-10-08T09:01:26.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-08T09:01:26.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:01:26.982 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-10-08T09:01:27.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-10-08T09:01:27.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T09:01:27.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T09:01:27.117 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-10-08T09:01:27.184 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T09:01:27.185 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T09:01:27.196 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T09:01:27.196 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T09:01:27.196 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:01:27.199 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T09:01:27.199 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:01:27.201 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T09:01:27.202 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:01:27.232 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T09:01:27.233 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T09:01:27.247 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T09:01:27.249 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T09:01:27.257 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T09:01:27.272 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T09:01:27.326 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2023-10-08T09:01:27.333 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2023-10-08T09:01:27.337 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2023-10-08T09:01:27.344 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2023-10-08T09:01:27.346 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2023-10-08T09:01:27.352 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T09:01:27.356 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2023-10-08T09:01:27.361 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T09:01:27.365 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T09:01:27.370 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T09:01:27.376 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T09:01:27.383 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T09:01:27.434 INFO:teuthology.orchestra.run.smithi132.stderr:bash: ntpq: command not found 2023-10-08T09:01:27.436 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:01:27.437 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2023-10-08T09:01:27.437 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.437 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.437 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.437 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.461 INFO:teuthology.orchestra.run.smithi184.stderr:bash: ntpq: command not found 2023-10-08T09:01:27.465 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:01:27.465 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-10-08T09:01:27.465 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.465 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.465 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.465 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.471 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2023-10-08T09:01:27.474 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:01:27.474 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2023-10-08T09:01:27.475 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.475 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.475 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.475 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:01:27.476 INFO:teuthology.run_tasks:Running task pexec... 2023-10-08T09:01:27.485 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-08T09:01:27.485 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:01:27.485 DEBUG:teuthology.orchestra.run.smithi049:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T09:01:27.486 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi132.front.sepia.ceph.com 2023-10-08T09:01:27.486 DEBUG:teuthology.orchestra.run.smithi132:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T09:01:27.486 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T09:01:27.486 DEBUG:teuthology.orchestra.run.smithi184:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-08T09:01:27.856 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-10-08T09:01:27.878 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-08T09:01:27.907 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-10-08T09:01:29.808 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:52 ago on Sun 08 Oct 2023 08:59:37 AM UTC. 2023-10-08T09:01:29.838 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:55 ago on Sun 08 Oct 2023 08:59:34 AM UTC. 2023-10-08T09:01:30.042 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:43 ago on Sun 08 Oct 2023 08:59:46 AM UTC. 2023-10-08T09:01:31.599 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout:Resetting modules: 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2023-10-08T09:01:31.600 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:01:31.729 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout: Package Architecture Version Repository Size 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout:Resetting modules: 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout: container-tools 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout:================================================================================ 2023-10-08T09:01:31.730 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:01:32.013 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2023-10-08T09:01:32.014 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2023-10-08T09:01:32.014 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2023-10-08T09:01:32.014 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2023-10-08T09:01:32.015 INFO:teuthology.orchestra.run.smithi049.stdout:Resetting modules: 2023-10-08T09:01:32.015 INFO:teuthology.orchestra.run.smithi049.stdout: container-tools 2023-10-08T09:01:32.015 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:01:32.015 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2023-10-08T09:01:32.015 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2023-10-08T09:01:32.015 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:01:32.562 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:55 ago on Sun 08 Oct 2023 08:59:37 AM UTC. 2023-10-08T09:01:32.707 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:58 ago on Sun 08 Oct 2023 08:59:34 AM UTC. 2023-10-08T09:01:33.013 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:01:46 ago on Sun 08 Oct 2023 08:59:46 AM UTC. 2023-10-08T09:01:42.732 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-10-08T09:01:43.022 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-10-08T09:01:43.202 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-10-08T09:01:43.500 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-08T09:01:44.796 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:02:07 ago on Sun 08 Oct 2023 08:59:37 AM UTC. 2023-10-08T09:01:45.030 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2023-10-08T09:01:45.244 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:10 ago on Sun 08 Oct 2023 08:59:34 AM UTC. 2023-10-08T09:01:45.501 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-10-08T09:01:46.654 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================================ 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================================ 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout:Installing group/module packages: 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T09:01:46.750 INFO:teuthology.orchestra.run.smithi132.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout:Installing module profiles: 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools/common 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling module streams: 2023-10-08T09:01:46.751 INFO:teuthology.orchestra.run.smithi132.stdout: container-tools rhel8 2023-10-08T09:01:46.752 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:01:46.752 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2023-10-08T09:01:46.752 INFO:teuthology.orchestra.run.smithi132.stdout:============================================================================================================ 2023-10-08T09:01:46.752 INFO:teuthology.orchestra.run.smithi132.stdout:Install 10 Packages 2023-10-08T09:01:46.752 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:01:46.752 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 18 M 2023-10-08T09:01:46.753 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 65 M 2023-10-08T09:01:46.753 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2023-10-08T09:01:46.953 INFO:teuthology.orchestra.run.smithi132.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 124 kB/s | 25 kB 00:00 2023-10-08T09:01:46.960 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-10-08T09:01:46.970 INFO:teuthology.orchestra.run.smithi132.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 431 kB/s | 94 kB 00:00 2023-10-08T09:01:46.995 INFO:teuthology.orchestra.run.smithi132.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 168 kB/s | 41 kB 00:00 2023-10-08T09:01:47.021 INFO:teuthology.orchestra.run.smithi132.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 730 kB/s | 49 kB 00:00 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================================ 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================================ 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout:Installing group/module packages: 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T09:01:47.053 INFO:teuthology.orchestra.run.smithi184.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout:Installing module profiles: 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: container-tools/common 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout:Enabling module streams: 2023-10-08T09:01:47.054 INFO:teuthology.orchestra.run.smithi184.stdout: container-tools rhel8 2023-10-08T09:01:47.055 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:01:47.055 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2023-10-08T09:01:47.055 INFO:teuthology.orchestra.run.smithi184.stdout:============================================================================================================ 2023-10-08T09:01:47.055 INFO:teuthology.orchestra.run.smithi184.stdout:Install 10 Packages 2023-10-08T09:01:47.055 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:01:47.055 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 18 M 2023-10-08T09:01:47.055 INFO:teuthology.orchestra.run.smithi184.stdout:Installed size: 65 M 2023-10-08T09:01:47.056 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2023-10-08T09:01:47.079 INFO:teuthology.orchestra.run.smithi132.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.7 MB/s | 524 kB 00:00 2023-10-08T09:01:47.202 INFO:teuthology.orchestra.run.smithi184.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 278 kB/s | 41 kB 00:00 2023-10-08T09:01:47.221 INFO:teuthology.orchestra.run.smithi132.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 747 kB/s | 149 kB 00:00 2023-10-08T09:01:47.227 INFO:teuthology.orchestra.run.smithi184.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 545 kB/s | 94 kB 00:00 2023-10-08T09:01:47.253 INFO:teuthology.orchestra.run.smithi184.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 126 kB/s | 25 kB 00:00 2023-10-08T09:01:47.313 INFO:teuthology.orchestra.run.smithi132.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 25 MB/s | 8.0 MB 00:00 2023-10-08T09:01:47.319 INFO:teuthology.orchestra.run.smithi184.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 421 kB/s | 49 kB 00:00 2023-10-08T09:01:47.338 INFO:teuthology.orchestra.run.smithi132.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 822 kB/s | 213 kB 00:00 2023-10-08T09:01:47.387 INFO:teuthology.orchestra.run.smithi184.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.2 MB/s | 524 kB 00:00 2023-10-08T09:01:47.387 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:02:00 ago on Sun 08 Oct 2023 08:59:46 AM UTC. 2023-10-08T09:01:47.514 INFO:teuthology.orchestra.run.smithi132.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 24 MB/s | 7.0 MB 00:00 2023-10-08T09:01:47.554 INFO:teuthology.orchestra.run.smithi184.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 27 MB/s | 8.0 MB 00:00 2023-10-08T09:01:47.570 INFO:teuthology.orchestra.run.smithi184.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 596 kB/s | 149 kB 00:00 2023-10-08T09:01:47.572 INFO:teuthology.orchestra.run.smithi132.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 8.5 MB/s | 2.2 MB 00:00 2023-10-08T09:01:47.573 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:01:47.573 INFO:teuthology.orchestra.run.smithi132.stdout:Total 22 MB/s | 18 MB 00:00 2023-10-08T09:01:47.638 INFO:teuthology.orchestra.run.smithi184.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 848 kB/s | 213 kB 00:00 2023-10-08T09:01:47.713 INFO:teuthology.orchestra.run.smithi184.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 15 MB/s | 2.2 MB 00:00 2023-10-08T09:01:47.772 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2023-10-08T09:01:47.790 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2023-10-08T09:01:47.790 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2023-10-08T09:01:47.838 INFO:teuthology.orchestra.run.smithi184.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 25 MB/s | 7.0 MB 00:00 2023-10-08T09:01:47.839 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:01:47.839 INFO:teuthology.orchestra.run.smithi184.stdout:Total 23 MB/s | 18 MB 00:00 2023-10-08T09:01:47.971 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2023-10-08T09:01:47.975 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2023-10-08T09:01:48.038 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2023-10-08T09:01:48.057 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2023-10-08T09:01:48.058 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2023-10-08T09:01:48.240 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2023-10-08T09:01:48.244 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2023-10-08T09:01:48.475 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2023-10-08T09:01:48.641 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T09:01:48.740 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2023-10-08T09:01:48.853 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:01:48.927 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T09:01:49.046 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T09:01:49.086 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:01:49.254 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T09:01:49.262 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T09:01:49.341 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2023-10-08T09:01:49.443 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================================ 2023-10-08T09:01:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2023-10-08T09:01:49.443 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================================ 2023-10-08T09:01:49.444 INFO:teuthology.orchestra.run.smithi049.stdout:Installing group/module packages: 2023-10-08T09:01:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-08T09:01:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-08T09:01:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-08T09:01:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-08T09:01:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-08T09:01:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-08T09:01:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-08T09:01:49.445 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2023-10-08T09:01:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-08T09:01:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-08T09:01:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-08T09:01:49.446 INFO:teuthology.orchestra.run.smithi049.stdout:Installing module profiles: 2023-10-08T09:01:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: container-tools/common 2023-10-08T09:01:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:01:49.446 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling module streams: 2023-10-08T09:01:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: container-tools rhel8 2023-10-08T09:01:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:01:49.447 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2023-10-08T09:01:49.447 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================================ 2023-10-08T09:01:49.447 INFO:teuthology.orchestra.run.smithi049.stdout:Install 10 Packages 2023-10-08T09:01:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:01:49.448 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 18 M 2023-10-08T09:01:49.448 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 65 M 2023-10-08T09:01:49.449 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2023-10-08T09:01:49.473 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T09:01:49.571 INFO:teuthology.orchestra.run.smithi049.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 326 kB/s | 41 kB 00:00 2023-10-08T09:01:49.588 INFO:teuthology.orchestra.run.smithi049.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 660 kB/s | 94 kB 00:00 2023-10-08T09:01:49.614 INFO:teuthology.orchestra.run.smithi049.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 148 kB/s | 25 kB 00:00 2023-10-08T09:01:49.631 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T09:01:49.689 INFO:teuthology.orchestra.run.smithi049.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 421 kB/s | 49 kB 00:00 2023-10-08T09:01:49.722 INFO:teuthology.orchestra.run.smithi049.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.8 MB/s | 524 kB 00:00 2023-10-08T09:01:49.773 INFO:teuthology.orchestra.run.smithi049.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.7 MB/s | 149 kB 00:00 2023-10-08T09:01:49.831 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T09:01:49.856 INFO:teuthology.orchestra.run.smithi049.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.6 MB/s | 213 kB 00:00 2023-10-08T09:01:50.007 INFO:teuthology.orchestra.run.smithi049.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 20 MB/s | 8.0 MB 00:00 2023-10-08T09:01:50.140 INFO:teuthology.orchestra.run.smithi049.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 19 MB/s | 7.0 MB 00:00 2023-10-08T09:01:50.215 INFO:teuthology.orchestra.run.smithi049.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 6.1 MB/s | 2.2 MB 00:00 2023-10-08T09:01:50.216 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:01:50.216 INFO:teuthology.orchestra.run.smithi049.stdout:Total 24 MB/s | 18 MB 00:00 2023-10-08T09:01:50.409 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T09:01:50.424 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2023-10-08T09:01:50.445 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2023-10-08T09:01:50.445 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2023-10-08T09:01:50.614 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T09:01:50.640 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2023-10-08T09:01:50.644 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2023-10-08T09:01:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2023-10-08T09:01:51.319 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T09:01:51.386 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-08T09:01:51.508 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T09:01:51.525 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T09:01:51.539 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:01:51.709 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T09:01:51.710 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T09:01:51.762 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-08T09:01:51.886 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:01:51.926 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T09:01:51.949 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-08T09:01:52.095 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:01:52.148 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T09:01:52.149 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T09:01:52.330 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-08T09:01:52.360 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:01:52.360 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T09:01:52.361 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:01:52.361 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T09:01:52.361 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T09:01:52.361 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T09:01:52.361 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T09:01:52.362 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T09:01:52.362 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T09:01:52.362 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T09:01:53.127 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-08T09:01:53.325 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T09:01:53.325 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:02:15 ago on Sun 08 Oct 2023 08:59:37 AM UTC. 2023-10-08T09:01:53.538 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T09:01:53.538 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:19 ago on Sun 08 Oct 2023 08:59:34 AM UTC. 2023-10-08T09:01:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-08T09:01:54.269 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-08T09:01:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-08T09:01:54.613 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:01:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-08T09:01:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-08T09:01:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-08T09:01:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-08T09:01:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-08T09:01:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-08T09:01:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-08T09:01:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-08T09:01:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-08T09:01:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-08T09:01:56.070 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-08T09:01:56.070 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:02:09 ago on Sun 08 Oct 2023 08:59:46 AM UTC. 2023-10-08T09:02:09.251 INFO:teuthology.orchestra.run.smithi184.stdout:Installed products updated. 2023-10-08T09:02:09.445 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:02:09.445 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2023-10-08T09:02:09.445 INFO:teuthology.orchestra.run.smithi184.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:09.445 INFO:teuthology.orchestra.run.smithi184.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:02:09.445 INFO:teuthology.orchestra.run.smithi184.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:09.446 INFO:teuthology.orchestra.run.smithi184.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T09:02:09.446 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T09:02:09.446 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T09:02:09.446 INFO:teuthology.orchestra.run.smithi184.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:09.446 INFO:teuthology.orchestra.run.smithi184.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:09.447 INFO:teuthology.orchestra.run.smithi184.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:02:09.447 INFO:teuthology.orchestra.run.smithi184.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T09:02:09.447 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:02:09.447 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-10-08T09:02:09.592 DEBUG:teuthology.parallel:result is None 2023-10-08T09:02:09.873 INFO:teuthology.orchestra.run.smithi132.stdout:Installed products updated. 2023-10-08T09:02:10.065 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:02:10.065 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2023-10-08T09:02:10.065 INFO:teuthology.orchestra.run.smithi132.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:10.065 INFO:teuthology.orchestra.run.smithi132.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:02:10.065 INFO:teuthology.orchestra.run.smithi132.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:10.065 INFO:teuthology.orchestra.run.smithi132.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T09:02:10.065 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T09:02:10.066 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T09:02:10.066 INFO:teuthology.orchestra.run.smithi132.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:10.066 INFO:teuthology.orchestra.run.smithi132.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:10.066 INFO:teuthology.orchestra.run.smithi132.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:02:10.066 INFO:teuthology.orchestra.run.smithi132.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T09:02:10.066 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:02:10.066 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-10-08T09:02:10.212 DEBUG:teuthology.parallel:result is None 2023-10-08T09:02:10.979 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2023-10-08T09:02:11.191 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:02:11.192 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2023-10-08T09:02:11.192 INFO:teuthology.orchestra.run.smithi049.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:11.192 INFO:teuthology.orchestra.run.smithi049.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:02:11.192 INFO:teuthology.orchestra.run.smithi049.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:11.192 INFO:teuthology.orchestra.run.smithi049.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-08T09:02:11.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-08T09:02:11.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-08T09:02:11.193 INFO:teuthology.orchestra.run.smithi049.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:11.193 INFO:teuthology.orchestra.run.smithi049.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-08T09:02:11.193 INFO:teuthology.orchestra.run.smithi049.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-08T09:02:11.194 INFO:teuthology.orchestra.run.smithi049.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-08T09:02:11.194 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:02:11.194 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2023-10-08T09:02:11.332 DEBUG:teuthology.parallel:result is None 2023-10-08T09:02:11.332 INFO:teuthology.run_tasks:Running task install... 2023-10-08T09:02:11.340 DEBUG:teuthology.task.install:project ceph 2023-10-08T09:02:11.341 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'}} 2023-10-08T09:02:11.341 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de'} 2023-10-08T09:02:11.341 INFO:teuthology.task.install:Using flavor: default 2023-10-08T09:02:11.347 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T09:02:11.347 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T09:02:11.347 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T09:02:11.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:02:11.350 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T09:02:11.350 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:02:11.351 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'tag': None, 'wait_for_package': False} 2023-10-08T09:02:11.351 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:02:11.496 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T09:02:11.496 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T09:02:11.502 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T09:02:11.502 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T09:02:11.566 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/ 2023-10-08T09:02:11.566 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6639.g7b3143ba 2023-10-08T09:02:11.615 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T09:02:11.615 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:02:11.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T09:02:11.616 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T09:02:11.616 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:02:11.616 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T09:02:11.644 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T09:02:11.645 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T09:02:11.650 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T09:02:11.652 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T09:02:11.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T09:02:11.724 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T09:02:11.733 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T09:02:11.734 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:02:11.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T09:02:11.767 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T09:02:11.767 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T09:02:11.790 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T09:02:11.809 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T09:02:11.840 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T09:02:11.857 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2023-10-08T09:02:11.858 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2023-10-08T09:02:11.882 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2023-10-08T09:02:11.884 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2023-10-08T09:02:11.923 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T09:02:11.991 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2023-10-08T09:02:11.992 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2023-10-08T09:02:12.250 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-10-08T09:02:12.281 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-08T09:02:12.365 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-10-08T09:02:13.021 INFO:teuthology.orchestra.run.smithi184.stdout:44 files removed 2023-10-08T09:02:13.062 INFO:teuthology.orchestra.run.smithi049.stdout:44 files removed 2023-10-08T09:02:13.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T09:02:13.114 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T09:02:13.121 INFO:teuthology.orchestra.run.smithi132.stdout:44 files removed 2023-10-08T09:02:13.170 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T09:02:13.402 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-08T09:02:13.467 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2023-10-08T09:02:13.503 INFO:teuthology.orchestra.run.smithi132.stdout:Updating Subscription Management repositories. 2023-10-08T09:02:14.647 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 255 kB/s | 81 kB 00:00 2023-10-08T09:02:14.768 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 209 kB/s | 81 kB 00:00 2023-10-08T09:02:14.997 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-08T09:02:15.066 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 268 kB/s | 81 kB 00:00 2023-10-08T09:02:15.118 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-08T09:02:15.240 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 6.7 kB/s | 1.5 kB 00:00 2023-10-08T09:02:15.374 INFO:teuthology.orchestra.run.smithi184.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-10-08T09:02:15.394 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2023-10-08T09:02:15.425 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-10-08T09:02:15.528 INFO:teuthology.orchestra.run.smithi132.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-10-08T09:02:15.676 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-10-08T09:02:15.801 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-10-08T09:02:16.461 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-10-08T09:02:16.856 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-08T09:02:17.677 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 16 MB 00:02 2023-10-08T09:02:20.236 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 397 kB/s | 24 kB 00:00 2023-10-08T09:02:20.413 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 339 kB/s | 24 kB 00:00 2023-10-08T09:02:21.463 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 MB/s | 47 MB 00:01 2023-10-08T09:02:21.481 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 53 MB/s | 53 MB 00:00 2023-10-08T09:02:21.652 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 46 kB/s | 24 kB 00:00 2023-10-08T09:02:22.844 INFO:teuthology.orchestra.run.smithi184.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-10-08T09:02:29.892 INFO:teuthology.orchestra.run.smithi132.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-10-08T09:02:30.098 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-10-08T09:02:31.084 INFO:teuthology.orchestra.run.smithi184.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 47 MB 00:00 2023-10-08T09:02:41.065 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:02:41.065 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:02:41.079 INFO:teuthology.orchestra.run.smithi132.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T09:02:41.080 INFO:teuthology.orchestra.run.smithi132.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T09:02:41.081 INFO:teuthology.orchestra.run.smithi132.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T09:02:41.082 INFO:teuthology.orchestra.run.smithi132.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T09:02:41.084 INFO:teuthology.orchestra.run.smithi132.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T09:02:41.087 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T09:02:41.089 INFO:teuthology.orchestra.run.smithi132.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T09:02:41.220 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2023-10-08T09:02:41.236 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-10-08T09:02:41.236 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2023-10-08T09:02:41.236 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-10-08T09:02:41.236 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2023-10-08T09:02:41.236 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T09:02:41.236 INFO:teuthology.orchestra.run.smithi132.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T09:02:41.236 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T09:02:41.237 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T09:02:41.238 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T09:02:41.239 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T09:02:41.240 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T09:02:41.241 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T09:02:41.242 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T09:02:41.243 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T09:02:41.244 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T09:02:41.245 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T09:02:41.246 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T09:02:41.247 INFO:teuthology.orchestra.run.smithi132.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout:Enabling module streams: 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql 10 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout:=========================================================================================================================== 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout:Install 128 Packages 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 3 Packages 2023-10-08T09:02:41.248 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:02:41.249 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 209 M 2023-10-08T09:02:41.249 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2023-10-08T09:02:41.500 INFO:teuthology.orchestra.run.smithi132.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 34 kB/s | 6.4 kB 00:00 2023-10-08T09:02:41.855 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:02:41.856 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:02:41.871 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T09:02:41.872 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T09:02:41.873 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T09:02:41.874 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T09:02:41.877 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T09:02:41.880 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T09:02:41.881 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T09:02:41.967 INFO:teuthology.orchestra.run.smithi132.stdout:(2/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 8.3 MB/s | 5.4 MB 00:00 2023-10-08T09:02:42.025 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2023-10-08T09:02:42.033 INFO:teuthology.orchestra.run.smithi132.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 2.5 MB/s | 167 kB 00:00 2023-10-08T09:02:42.038 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================================================== 2023-10-08T09:02:42.038 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2023-10-08T09:02:42.038 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================================================== 2023-10-08T09:02:42.038 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2023-10-08T09:02:42.038 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T09:02:42.038 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T09:02:42.039 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T09:02:42.040 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T09:02:42.041 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T09:02:42.042 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T09:02:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T09:02:42.044 INFO:teuthology.orchestra.run.smithi049.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T09:02:42.045 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T09:02:42.046 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T09:02:42.047 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T09:02:42.048 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T09:02:42.049 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling module streams: 2023-10-08T09:02:42.050 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql 10 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================================================== 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout:Install 128 Packages 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 3 Packages 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 209 M 2023-10-08T09:02:42.051 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2023-10-08T09:02:42.125 INFO:teuthology.orchestra.run.smithi132.stdout:(4/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.4 MB/s | 918 kB 00:00 2023-10-08T09:02:42.234 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:02:42.235 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T09:02:42.249 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-08T09:02:42.251 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-08T09:02:42.252 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-08T09:02:42.253 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T09:02:42.255 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T09:02:42.258 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-08T09:02:42.259 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-08T09:02:42.317 INFO:teuthology.orchestra.run.smithi132.stdout:(5/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 8.2 MB/s | 2.3 MB 00:00 2023-10-08T09:02:42.388 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-10-08T09:02:42.400 INFO:teuthology.orchestra.run.smithi184.stdout:=========================================================================================================================== 2023-10-08T09:02:42.400 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout:=========================================================================================================================== 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.4 k 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 5.4 M 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 918 k 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 167 k 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 1.6 M 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 132 k 2023-10-08T09:02:42.401 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 1.8 M 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 7.4 M 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 47 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 14 M 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 49 M 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 261 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 190 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 261 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 18 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T09:02:42.402 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 31 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 840 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 139 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 197 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 371 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 365 k 2023-10-08T09:02:42.403 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 107 k 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 94 k 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.6 M 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 179 k 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 3.7 M 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 23 M 2023-10-08T09:02:42.404 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 23 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 2.3 M 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 242 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 4.6 M 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 18 M 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6639.g7b3143ba.el8 ceph-noarch 15 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 24 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-08T09:02:42.405 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 184 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 518 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 6.9 M 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T09:02:42.406 INFO:teuthology.orchestra.run.smithi184.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T09:02:42.407 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 45 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:18.0.0-6639.g7b3143ba.el8 ceph 125 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T09:02:42.408 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T09:02:42.409 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T09:02:42.410 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T09:02:42.411 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout:Enabling module streams: 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql 10 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout:=========================================================================================================================== 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout:Install 128 Packages 2023-10-08T09:02:42.412 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 3 Packages 2023-10-08T09:02:42.413 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:02:42.413 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 209 M 2023-10-08T09:02:42.413 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2023-10-08T09:02:42.781 INFO:teuthology.orchestra.run.smithi049.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-10-08T09:02:42.820 INFO:teuthology.orchestra.run.smithi184.stdout:(1/131): ceph-18.0.0-6639.g7b3143ba.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2023-10-08T09:02:42.859 INFO:teuthology.orchestra.run.smithi132.stdout:(6/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 8.5 MB/s | 4.6 MB 00:00 2023-10-08T09:02:43.085 INFO:teuthology.orchestra.run.smithi132.stdout:(7/131): ceph-common-18.0.0-6639.g7b3143ba.el8. 13 MB/s | 23 MB 00:01 2023-10-08T09:02:43.143 INFO:teuthology.orchestra.run.smithi132.stdout:(8/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 1.6 MB/s | 1.6 MB 00:01 2023-10-08T09:02:43.178 INFO:teuthology.orchestra.run.smithi184.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 2.5 MB/s | 918 kB 00:00 2023-10-08T09:02:43.235 INFO:teuthology.orchestra.run.smithi132.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 265 kB/s | 24 kB 00:00 2023-10-08T09:02:43.703 INFO:teuthology.orchestra.run.smithi184.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 319 kB/s | 167 kB 00:00 2023-10-08T09:02:43.823 INFO:teuthology.orchestra.run.smithi049.stdout:(2/131): ceph-fuse-18.0.0-6639.g7b3143ba.el8.x8 882 kB/s | 918 kB 00:01 2023-10-08T09:02:44.115 INFO:teuthology.orchestra.run.smithi049.stdout:(3/131): ceph-immutable-object-cache-18.0.0-663 574 kB/s | 167 kB 00:00 2023-10-08T09:02:44.836 INFO:teuthology.orchestra.run.smithi132.stdout:(10/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x8 9.1 MB/s | 18 MB 00:01 2023-10-08T09:02:44.879 INFO:teuthology.orchestra.run.smithi184.stdout:(4/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 2.3 MB/s | 5.4 MB 00:02 2023-10-08T09:02:45.011 INFO:teuthology.orchestra.run.smithi184.stdout:(5/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-10-08T09:02:45.012 INFO:teuthology.orchestra.run.smithi132.stdout:(11/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 4.4 MB/s | 261 kB 00:00 2023-10-08T09:02:45.012 INFO:teuthology.orchestra.run.smithi132.stdout:(12/131): libcephfs-devel-18.0.0-6639.g7b3143ba 466 kB/s | 31 kB 00:00 2023-10-08T09:02:45.137 INFO:teuthology.orchestra.run.smithi132.stdout:(13/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 4.7 MB/s | 840 kB 00:00 2023-10-08T09:02:45.220 INFO:teuthology.orchestra.run.smithi132.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 2.2 MB/s | 184 kB 00:00 2023-10-08T09:02:45.263 INFO:teuthology.orchestra.run.smithi184.stdout:(6/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 4.2 MB/s | 1.6 MB 00:00 2023-10-08T09:02:45.287 INFO:teuthology.orchestra.run.smithi132.stdout:(15/131): librados-devel-18.0.0-6639.g7b3143ba. 2.0 MB/s | 139 kB 00:00 2023-10-08T09:02:45.504 INFO:teuthology.orchestra.run.smithi132.stdout:(16/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 5.7 MB/s | 14 MB 00:02 2023-10-08T09:02:45.538 INFO:teuthology.orchestra.run.smithi132.stdout:(17/131): libradosstriper1-18.0.0-6639.g7b3143b 2.0 MB/s | 518 kB 00:00 2023-10-08T09:02:45.622 INFO:teuthology.orchestra.run.smithi132.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 537 kB/s | 45 kB 00:00 2023-10-08T09:02:45.714 INFO:teuthology.orchestra.run.smithi132.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.3 MB/s | 125 kB 00:00 2023-10-08T09:02:45.806 INFO:teuthology.orchestra.run.smithi132.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 2.1 MB/s | 197 kB 00:00 2023-10-08T09:02:45.890 INFO:teuthology.orchestra.run.smithi049.stdout:(4/131): ceph-base-18.0.0-6639.g7b3143ba.el8.x8 1.6 MB/s | 5.4 MB 00:03 2023-10-08T09:02:45.914 INFO:teuthology.orchestra.run.smithi132.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 3.3 MB/s | 371 kB 00:00 2023-10-08T09:02:46.023 INFO:teuthology.orchestra.run.smithi049.stdout:(5/131): ceph-mds-18.0.0-6639.g7b3143ba.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-10-08T09:02:46.031 INFO:teuthology.orchestra.run.smithi132.stdout:(22/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 3.1 MB/s | 365 kB 00:00 2023-10-08T09:02:46.148 INFO:teuthology.orchestra.run.smithi132.stdout:(23/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 915 kB/s | 107 kB 00:00 2023-10-08T09:02:46.240 INFO:teuthology.orchestra.run.smithi132.stdout:(24/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.0 MB/s | 94 kB 00:00 2023-10-08T09:02:46.738 INFO:teuthology.orchestra.run.smithi184.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 2.6 MB/s | 4.6 MB 00:01 2023-10-08T09:02:46.973 INFO:teuthology.orchestra.run.smithi132.stdout:(25/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 4.9 MB/s | 3.6 MB 00:00 2023-10-08T09:02:47.057 INFO:teuthology.orchestra.run.smithi132.stdout:(26/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 4.5 MB/s | 6.9 MB 00:01 2023-10-08T09:02:47.083 INFO:teuthology.orchestra.run.smithi132.stdout:(27/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 1.6 MB/s | 179 kB 00:00 2023-10-08T09:02:47.107 INFO:teuthology.orchestra.run.smithi049.stdout:(6/131): ceph-mgr-18.0.0-6639.g7b3143ba.el8.x86 1.3 MB/s | 1.6 MB 00:01 2023-10-08T09:02:47.116 INFO:teuthology.orchestra.run.smithi132.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6639.g 394 kB/s | 23 kB 00:00 2023-10-08T09:02:47.150 INFO:teuthology.orchestra.run.smithi132.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 1.9 MB/s | 132 kB 00:00 2023-10-08T09:02:47.397 INFO:teuthology.orchestra.run.smithi184.stdout:(8/131): ceph-common-18.0.0-6639.g7b3143ba.el8. 4.6 MB/s | 23 MB 00:04 2023-10-08T09:02:47.483 INFO:teuthology.orchestra.run.smithi132.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 4.8 MB/s | 1.8 MB 00:00 2023-10-08T09:02:47.497 INFO:teuthology.orchestra.run.smithi184.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 244 kB/s | 24 kB 00:00 2023-10-08T09:02:47.542 INFO:teuthology.orchestra.run.smithi132.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 4.1 MB/s | 242 kB 00:00 2023-10-08T09:02:47.592 INFO:teuthology.orchestra.run.smithi132.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 951 kB/s | 47 kB 00:00 2023-10-08T09:02:47.634 INFO:teuthology.orchestra.run.smithi132.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 350 kB/s | 15 kB 00:00 2023-10-08T09:02:47.672 INFO:teuthology.orchestra.run.smithi184.stdout:(10/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x8 7.4 MB/s | 18 MB 00:02 2023-10-08T09:02:47.701 INFO:teuthology.orchestra.run.smithi132.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 3.8 MB/s | 261 kB 00:00 2023-10-08T09:02:47.781 INFO:teuthology.orchestra.run.smithi184.stdout:(11/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 2.4 MB/s | 261 kB 00:00 2023-10-08T09:02:47.785 INFO:teuthology.orchestra.run.smithi132.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 2.2 MB/s | 190 kB 00:00 2023-10-08T09:02:47.844 INFO:teuthology.orchestra.run.smithi132.stdout:(36/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 311 kB/s | 18 kB 00:00 2023-10-08T09:02:47.873 INFO:teuthology.orchestra.run.smithi184.stdout:(12/131): libcephfs-devel-18.0.0-6639.g7b3143ba 339 kB/s | 31 kB 00:00 2023-10-08T09:02:47.977 INFO:teuthology.orchestra.run.smithi132.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.2 MB/s | 306 kB 00:00 2023-10-08T09:02:48.039 INFO:teuthology.orchestra.run.smithi184.stdout:(13/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 4.9 MB/s | 840 kB 00:00 2023-10-08T09:02:48.086 INFO:teuthology.orchestra.run.smithi132.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 7.9 MB/s | 7.4 MB 00:00 2023-10-08T09:02:48.106 INFO:teuthology.orchestra.run.smithi184.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 2.7 MB/s | 184 kB 00:00 2023-10-08T09:02:48.111 INFO:teuthology.orchestra.run.smithi132.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 441 kB/s | 59 kB 00:00 2023-10-08T09:02:48.165 INFO:teuthology.orchestra.run.smithi184.stdout:(15/131): librados-devel-18.0.0-6639.g7b3143ba. 2.3 MB/s | 139 kB 00:00 2023-10-08T09:02:48.178 INFO:teuthology.orchestra.run.smithi132.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 7.0 MB/s | 476 kB 00:00 2023-10-08T09:02:48.204 INFO:teuthology.orchestra.run.smithi132.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 641 kB/s | 75 kB 00:00 2023-10-08T09:02:48.229 INFO:teuthology.orchestra.run.smithi132.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 867 kB/s | 44 kB 00:00 2023-10-08T09:02:48.257 INFO:teuthology.orchestra.run.smithi184.stdout:(16/131): libradosstriper1-18.0.0-6639.g7b3143b 5.5 MB/s | 518 kB 00:00 2023-10-08T09:02:48.271 INFO:teuthology.orchestra.run.smithi132.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 795 kB/s | 33 kB 00:00 2023-10-08T09:02:48.296 INFO:teuthology.orchestra.run.smithi132.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-10-08T09:02:48.346 INFO:teuthology.orchestra.run.smithi132.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-10-08T09:02:48.372 INFO:teuthology.orchestra.run.smithi132.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-10-08T09:02:48.405 INFO:teuthology.orchestra.run.smithi132.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2023-10-08T09:02:48.431 INFO:teuthology.orchestra.run.smithi132.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-10-08T09:02:48.456 INFO:teuthology.orchestra.run.smithi132.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 220 kB/s | 11 kB 00:00 2023-10-08T09:02:48.481 INFO:teuthology.orchestra.run.smithi132.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-10-08T09:02:48.524 INFO:teuthology.orchestra.run.smithi132.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-10-08T09:02:48.557 INFO:teuthology.orchestra.run.smithi132.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-10-08T09:02:48.582 INFO:teuthology.orchestra.run.smithi132.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-10-08T09:02:48.608 INFO:teuthology.orchestra.run.smithi132.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-08T09:02:48.633 INFO:teuthology.orchestra.run.smithi132.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-10-08T09:02:48.659 INFO:teuthology.orchestra.run.smithi132.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2023-10-08T09:02:48.684 INFO:teuthology.orchestra.run.smithi132.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 670 kB/s | 34 kB 00:00 2023-10-08T09:02:48.709 INFO:teuthology.orchestra.run.smithi132.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-08T09:02:48.734 INFO:teuthology.orchestra.run.smithi132.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-10-08T09:02:48.751 INFO:teuthology.orchestra.run.smithi132.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 447 kB/s | 19 kB 00:00 2023-10-08T09:02:48.777 INFO:teuthology.orchestra.run.smithi132.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 572 kB/s | 24 kB 00:00 2023-10-08T09:02:48.794 INFO:teuthology.orchestra.run.smithi132.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 696 kB/s | 29 kB 00:00 2023-10-08T09:02:48.811 INFO:teuthology.orchestra.run.smithi132.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 825 kB/s | 27 kB 00:00 2023-10-08T09:02:48.844 INFO:teuthology.orchestra.run.smithi132.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-10-08T09:02:48.870 INFO:teuthology.orchestra.run.smithi132.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-10-08T09:02:48.886 INFO:teuthology.orchestra.run.smithi132.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-10-08T09:02:48.912 INFO:teuthology.orchestra.run.smithi132.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-10-08T09:02:48.915 INFO:teuthology.orchestra.run.smithi049.stdout:(7/131): ceph-mon-18.0.0-6639.g7b3143ba.el8.x86 1.6 MB/s | 4.6 MB 00:02 2023-10-08T09:02:48.929 INFO:teuthology.orchestra.run.smithi132.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 546 kB/s | 23 kB 00:00 2023-10-08T09:02:48.946 INFO:teuthology.orchestra.run.smithi132.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-10-08T09:02:48.966 INFO:teuthology.orchestra.run.smithi184.stdout:(17/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 9.8 MB/s | 6.9 MB 00:00 2023-10-08T09:02:49.013 INFO:teuthology.orchestra.run.smithi132.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2023-10-08T09:02:49.034 INFO:teuthology.orchestra.run.smithi184.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 674 kB/s | 45 kB 00:00 2023-10-08T09:02:49.113 INFO:teuthology.orchestra.run.smithi132.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.0 MB/s | 200 kB 00:00 2023-10-08T09:02:49.124 INFO:teuthology.orchestra.run.smithi184.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.3 MB/s | 125 kB 00:00 2023-10-08T09:02:49.138 INFO:teuthology.orchestra.run.smithi132.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 244 kB/s | 47 kB 00:00 2023-10-08T09:02:49.200 INFO:teuthology.orchestra.run.smithi184.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 2.6 MB/s | 197 kB 00:00 2023-10-08T09:02:49.214 INFO:teuthology.orchestra.run.smithi132.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 425 kB/s | 43 kB 00:00 2023-10-08T09:02:49.283 INFO:teuthology.orchestra.run.smithi184.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 4.4 MB/s | 371 kB 00:00 2023-10-08T09:02:49.314 INFO:teuthology.orchestra.run.smithi132.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-10-08T09:02:49.367 INFO:teuthology.orchestra.run.smithi184.stdout:(22/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 4.3 MB/s | 365 kB 00:00 2023-10-08T09:02:49.398 INFO:teuthology.orchestra.run.smithi132.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 261 kB/s | 48 kB 00:00 2023-10-08T09:02:49.442 INFO:teuthology.orchestra.run.smithi184.stdout:(23/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.4 MB/s | 107 kB 00:00 2023-10-08T09:02:49.457 INFO:teuthology.orchestra.run.smithi132.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 731 kB/s | 43 kB 00:00 2023-10-08T09:02:49.482 INFO:teuthology.orchestra.run.smithi132.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 519 kB/s | 87 kB 00:00 2023-10-08T09:02:49.518 INFO:teuthology.orchestra.run.smithi184.stdout:(24/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.2 MB/s | 94 kB 00:00 2023-10-08T09:02:49.607 INFO:teuthology.orchestra.run.smithi132.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 727 kB/s | 109 kB 00:00 2023-10-08T09:02:49.633 INFO:teuthology.orchestra.run.smithi132.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-10-08T09:02:49.717 INFO:teuthology.orchestra.run.smithi132.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 225 kB/s | 19 kB 00:00 2023-10-08T09:02:49.758 INFO:teuthology.orchestra.run.smithi132.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.6 MB/s | 560 kB 00:00 2023-10-08T09:02:49.834 INFO:teuthology.orchestra.run.smithi132.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.3 MB/s | 155 kB 00:00 2023-10-08T09:02:49.909 INFO:teuthology.orchestra.run.smithi184.stdout:(25/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 9.2 MB/s | 3.6 MB 00:00 2023-10-08T09:02:50.068 INFO:teuthology.orchestra.run.smithi184.stdout:(26/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 4.2 MB/s | 14 MB 00:03 2023-10-08T09:02:50.093 INFO:teuthology.orchestra.run.smithi184.stdout:(27/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 973 kB/s | 179 kB 00:00 2023-10-08T09:02:50.094 INFO:teuthology.orchestra.run.smithi132.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 1.6 MB/s | 559 kB 00:00 2023-10-08T09:02:50.117 INFO:teuthology.orchestra.run.smithi132.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 63 kB/s | 18 kB 00:00 2023-10-08T09:02:50.118 INFO:teuthology.orchestra.run.smithi184.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6639.g 462 kB/s | 23 kB 00:00 2023-10-08T09:02:50.161 INFO:teuthology.orchestra.run.smithi184.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 1.9 MB/s | 132 kB 00:00 2023-10-08T09:02:50.262 INFO:teuthology.orchestra.run.smithi132.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2023-10-08T09:02:50.302 INFO:teuthology.orchestra.run.smithi132.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2023-10-08T09:02:50.444 INFO:teuthology.orchestra.run.smithi132.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-10-08T09:02:50.469 INFO:teuthology.orchestra.run.smithi132.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 433 kB/s | 72 kB 00:00 2023-10-08T09:02:50.594 INFO:teuthology.orchestra.run.smithi132.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 551 kB/s | 83 kB 00:00 2023-10-08T09:02:50.595 INFO:teuthology.orchestra.run.smithi184.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 3.7 MB/s | 1.8 MB 00:00 2023-10-08T09:02:50.619 INFO:teuthology.orchestra.run.smithi132.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 399 kB/s | 60 kB 00:00 2023-10-08T09:02:50.645 INFO:teuthology.orchestra.run.smithi132.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-08T09:02:50.720 INFO:teuthology.orchestra.run.smithi184.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 1.9 MB/s | 242 kB 00:00 2023-10-08T09:02:50.787 INFO:teuthology.orchestra.run.smithi184.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 709 kB/s | 47 kB 00:00 2023-10-08T09:02:50.803 INFO:teuthology.orchestra.run.smithi132.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 314 kB/s | 58 kB 00:00 2023-10-08T09:02:50.829 INFO:teuthology.orchestra.run.smithi132.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 210 kB/s | 39 kB 00:00 2023-10-08T09:02:50.837 INFO:teuthology.orchestra.run.smithi184.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 291 kB/s | 15 kB 00:00 2023-10-08T09:02:50.929 INFO:teuthology.orchestra.run.smithi132.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 5.0 MB/s | 644 kB 00:00 2023-10-08T09:02:50.954 INFO:teuthology.orchestra.run.smithi184.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 9.3 MB/s | 7.4 MB 00:00 2023-10-08T09:02:50.955 INFO:teuthology.orchestra.run.smithi132.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.3 MB/s | 171 kB 00:00 2023-10-08T09:02:50.980 INFO:teuthology.orchestra.run.smithi184.stdout:(35/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 1.8 MB/s | 261 kB 00:00 2023-10-08T09:02:51.021 INFO:teuthology.orchestra.run.smithi184.stdout:(36/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 2.8 MB/s | 190 kB 00:00 2023-10-08T09:02:51.046 INFO:teuthology.orchestra.run.smithi132.stdout:(96/131): NetworkManager-initscripts-updown-1.3 1.1 MB/s | 138 kB 00:00 2023-10-08T09:02:51.047 INFO:teuthology.orchestra.run.smithi184.stdout:(37/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 271 kB/s | 18 kB 00:00 2023-10-08T09:02:51.164 INFO:teuthology.orchestra.run.smithi184.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 504 kB/s | 59 kB 00:00 2023-10-08T09:02:51.171 INFO:teuthology.orchestra.run.smithi132.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 881 kB/s | 110 kB 00:00 2023-10-08T09:02:51.214 INFO:teuthology.orchestra.run.smithi184.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-10-08T09:02:51.239 INFO:teuthology.orchestra.run.smithi184.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 998 kB/s | 75 kB 00:00 2023-10-08T09:02:51.272 INFO:teuthology.orchestra.run.smithi132.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 542 kB/s | 54 kB 00:00 2023-10-08T09:02:51.289 INFO:teuthology.orchestra.run.smithi184.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 873 kB/s | 44 kB 00:00 2023-10-08T09:02:51.315 INFO:teuthology.orchestra.run.smithi184.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.6 MB/s | 476 kB 00:00 2023-10-08T09:02:51.349 INFO:teuthology.orchestra.run.smithi184.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-10-08T09:02:51.365 INFO:teuthology.orchestra.run.smithi184.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2023-10-08T09:02:51.391 INFO:teuthology.orchestra.run.smithi184.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 389 kB/s | 16 kB 00:00 2023-10-08T09:02:51.416 INFO:teuthology.orchestra.run.smithi184.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-10-08T09:02:51.447 INFO:teuthology.orchestra.run.smithi132.stdout:(99/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-10-08T09:02:51.458 INFO:teuthology.orchestra.run.smithi184.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.6 MB/s | 384 kB 00:00 2023-10-08T09:02:51.473 INFO:teuthology.orchestra.run.smithi132.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-10-08T09:02:51.483 INFO:teuthology.orchestra.run.smithi184.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-10-08T09:02:51.509 INFO:teuthology.orchestra.run.smithi184.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-10-08T09:02:51.525 INFO:teuthology.orchestra.run.smithi184.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 421 kB/s | 18 kB 00:00 2023-10-08T09:02:51.568 INFO:teuthology.orchestra.run.smithi184.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-10-08T09:02:51.601 INFO:teuthology.orchestra.run.smithi184.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 13 MB/s | 1.2 MB 00:00 2023-10-08T09:02:51.606 INFO:teuthology.orchestra.run.smithi132.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 291 kB/s | 39 kB 00:00 2023-10-08T09:02:51.618 INFO:teuthology.orchestra.run.smithi184.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-10-08T09:02:51.631 INFO:teuthology.orchestra.run.smithi132.stdout:(102/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.7 MB/s | 323 kB 00:00 2023-10-08T09:02:51.643 INFO:teuthology.orchestra.run.smithi184.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-10-08T09:02:51.669 INFO:teuthology.orchestra.run.smithi184.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-10-08T09:02:51.686 INFO:teuthology.orchestra.run.smithi184.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 371 kB/s | 16 kB 00:00 2023-10-08T09:02:51.711 INFO:teuthology.orchestra.run.smithi184.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 803 kB/s | 34 kB 00:00 2023-10-08T09:02:51.736 INFO:teuthology.orchestra.run.smithi184.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-08T09:02:51.749 INFO:teuthology.orchestra.run.smithi132.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2023-10-08T09:02:51.762 INFO:teuthology.orchestra.run.smithi184.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-10-08T09:02:51.787 INFO:teuthology.orchestra.run.smithi184.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 370 kB/s | 19 kB 00:00 2023-10-08T09:02:51.812 INFO:teuthology.orchestra.run.smithi184.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2023-10-08T09:02:51.824 INFO:teuthology.orchestra.run.smithi049.stdout:(8/131): ceph-common-18.0.0-6639.g7b3143ba.el8. 2.4 MB/s | 23 MB 00:09 2023-10-08T09:02:51.824 INFO:teuthology.orchestra.run.smithi132.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 2.7 MB/s | 538 kB 00:00 2023-10-08T09:02:51.837 INFO:teuthology.orchestra.run.smithi184.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-10-08T09:02:51.849 INFO:teuthology.orchestra.run.smithi132.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2023-10-08T09:02:51.863 INFO:teuthology.orchestra.run.smithi184.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 547 kB/s | 27 kB 00:00 2023-10-08T09:02:51.882 INFO:teuthology.orchestra.run.smithi049.stdout:(9/131): ceph-selinux-18.0.0-6639.g7b3143ba.el8 420 kB/s | 24 kB 00:00 2023-10-08T09:02:51.888 INFO:teuthology.orchestra.run.smithi184.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-10-08T09:02:51.913 INFO:teuthology.orchestra.run.smithi184.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-10-08T09:02:51.939 INFO:teuthology.orchestra.run.smithi184.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-10-08T09:02:51.958 INFO:teuthology.orchestra.run.smithi132.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-10-08T09:02:51.964 INFO:teuthology.orchestra.run.smithi184.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-10-08T09:02:51.989 INFO:teuthology.orchestra.run.smithi184.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-10-08T09:02:51.991 INFO:teuthology.orchestra.run.smithi132.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 4.9 MB/s | 713 kB 00:00 2023-10-08T09:02:52.015 INFO:teuthology.orchestra.run.smithi184.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-08T09:02:52.074 INFO:teuthology.orchestra.run.smithi184.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2023-10-08T09:02:52.185 INFO:teuthology.orchestra.run.smithi132.stdout:(108/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-10-08T09:02:52.214 INFO:teuthology.orchestra.run.smithi184.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2023-10-08T09:02:52.217 INFO:teuthology.orchestra.run.smithi132.stdout:(109/131): python3-jsonpatch-1.21-2.el8.noarch. 118 kB/s | 27 kB 00:00 2023-10-08T09:02:52.249 INFO:teuthology.orchestra.run.smithi184.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 200 kB/s | 47 kB 00:00 2023-10-08T09:02:52.268 INFO:teuthology.orchestra.run.smithi132.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 3.8 MB/s | 320 kB 00:00 2023-10-08T09:02:52.326 INFO:teuthology.orchestra.run.smithi132.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 404 kB/s | 44 kB 00:00 2023-10-08T09:02:52.341 INFO:teuthology.orchestra.run.smithi184.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-10-08T09:02:52.408 INFO:teuthology.orchestra.run.smithi184.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 219 kB/s | 43 kB 00:00 2023-10-08T09:02:52.418 INFO:teuthology.orchestra.run.smithi132.stdout:(112/131): python3-numpy-1.14.3-10.el8.x86_64.r 25 MB/s | 3.7 MB 00:00 2023-10-08T09:02:52.467 INFO:teuthology.orchestra.run.smithi184.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 384 kB/s | 48 kB 00:00 2023-10-08T09:02:52.468 INFO:teuthology.orchestra.run.smithi132.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 3.2 MB/s | 457 kB 00:00 2023-10-08T09:02:52.510 INFO:teuthology.orchestra.run.smithi132.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-10-08T09:02:52.586 INFO:teuthology.orchestra.run.smithi132.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 547 kB/s | 64 kB 00:00 2023-10-08T09:02:52.644 INFO:teuthology.orchestra.run.smithi132.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 944 kB/s | 126 kB 00:00 2023-10-08T09:02:52.678 INFO:teuthology.orchestra.run.smithi132.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 217 kB/s | 20 kB 00:00 2023-10-08T09:02:52.708 INFO:teuthology.orchestra.run.smithi184.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 290 kB/s | 87 kB 00:00 2023-10-08T09:02:52.775 INFO:teuthology.orchestra.run.smithi184.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 138 kB/s | 43 kB 00:00 2023-10-08T09:02:52.795 INFO:teuthology.orchestra.run.smithi132.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 688 kB/s | 103 kB 00:00 2023-10-08T09:02:52.801 INFO:teuthology.orchestra.run.smithi184.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2023-10-08T09:02:52.879 INFO:teuthology.orchestra.run.smithi132.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 3.5 MB/s | 709 kB 00:00 2023-10-08T09:02:53.020 INFO:teuthology.orchestra.run.smithi132.stdout:(120/131): openblas-threads-0.3.15-3.el8.x86_64 22 MB/s | 4.9 MB 00:00 2023-10-08T09:02:53.046 INFO:teuthology.orchestra.run.smithi132.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-10-08T09:02:53.076 INFO:teuthology.orchestra.run.smithi184.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.0 MB/s | 560 kB 00:00 2023-10-08T09:02:53.101 INFO:teuthology.orchestra.run.smithi184.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 703 kB/s | 229 kB 00:00 2023-10-08T09:02:53.109 INFO:teuthology.orchestra.run.smithi049.stdout:(10/131): ceph-radosgw-18.0.0-6639.g7b3143ba.el 3.3 MB/s | 14 MB 00:04 2023-10-08T09:02:53.171 INFO:teuthology.orchestra.run.smithi132.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2023-10-08T09:02:53.202 INFO:teuthology.orchestra.run.smithi184.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 149 kB/s | 19 kB 00:00 2023-10-08T09:02:53.216 INFO:teuthology.orchestra.run.smithi049.stdout:(11/131): cephfs-mirror-18.0.0-6639.g7b3143ba.e 2.4 MB/s | 261 kB 00:00 2023-10-08T09:02:53.227 INFO:teuthology.orchestra.run.smithi184.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.2 MB/s | 155 kB 00:00 2023-10-08T09:02:53.246 INFO:teuthology.orchestra.run.smithi132.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-10-08T09:02:53.272 INFO:teuthology.orchestra.run.smithi132.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-10-08T09:02:53.380 INFO:teuthology.orchestra.run.smithi132.stdout:(125/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-10-08T09:02:53.425 INFO:teuthology.orchestra.run.smithi049.stdout:(12/131): ceph-osd-18.0.0-6639.g7b3143ba.el8.x8 2.8 MB/s | 18 MB 00:06 2023-10-08T09:02:53.450 INFO:teuthology.orchestra.run.smithi049.stdout:(13/131): libcephfs-devel-18.0.0-6639.g7b3143ba 133 kB/s | 31 kB 00:00 2023-10-08T09:02:53.498 INFO:teuthology.orchestra.run.smithi132.stdout:(126/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-10-08T09:02:53.509 INFO:teuthology.orchestra.run.smithi049.stdout:(14/131): libcephsqlite-18.0.0-6639.g7b3143ba.e 3.1 MB/s | 184 kB 00:00 2023-10-08T09:02:53.565 INFO:teuthology.orchestra.run.smithi132.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 8.2 MB/s | 1.5 MB 00:00 2023-10-08T09:02:53.576 INFO:teuthology.orchestra.run.smithi049.stdout:(15/131): librados-devel-18.0.0-6639.g7b3143ba. 2.0 MB/s | 139 kB 00:00 2023-10-08T09:02:53.686 INFO:teuthology.orchestra.run.smithi184.stdout:(83/131): ceph-test-18.0.0-6639.g7b3143ba.el8.x 7.9 MB/s | 49 MB 00:06 2023-10-08T09:02:53.693 INFO:teuthology.orchestra.run.smithi049.stdout:(16/131): libcephfs2-18.0.0-6639.g7b3143ba.el8. 3.1 MB/s | 840 kB 00:00 2023-10-08T09:02:53.703 INFO:teuthology.orchestra.run.smithi184.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 38 kB/s | 18 kB 00:00 2023-10-08T09:02:53.736 INFO:teuthology.orchestra.run.smithi184.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 1.0 MB/s | 559 kB 00:00 2023-10-08T09:02:53.768 INFO:teuthology.orchestra.run.smithi049.stdout:(17/131): libradosstriper1-18.0.0-6639.g7b3143b 2.6 MB/s | 518 kB 00:00 2023-10-08T09:02:53.846 INFO:teuthology.orchestra.run.smithi184.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2023-10-08T09:02:53.860 INFO:teuthology.orchestra.run.smithi049.stdout:(18/131): python3-ceph-argparse-18.0.0-6639.g7b 489 kB/s | 45 kB 00:00 2023-10-08T09:02:53.935 INFO:teuthology.orchestra.run.smithi049.stdout:(19/131): python3-ceph-common-18.0.0-6639.g7b31 1.6 MB/s | 125 kB 00:00 2023-10-08T09:02:53.940 INFO:teuthology.orchestra.run.smithi132.stdout:(128/131): librados2-18.0.0-6639.g7b3143ba.el8. 8.5 MB/s | 3.7 MB 00:00 2023-10-08T09:02:54.004 INFO:teuthology.orchestra.run.smithi184.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 1.9 MB/s | 605 kB 00:00 2023-10-08T09:02:54.046 INFO:teuthology.orchestra.run.smithi184.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 7.7 MB/s | 2.3 MB 00:00 2023-10-08T09:02:54.060 INFO:teuthology.orchestra.run.smithi049.stdout:(20/131): python3-cephfs-18.0.0-6639.g7b3143ba. 1.5 MB/s | 197 kB 00:00 2023-10-08T09:02:54.071 INFO:teuthology.orchestra.run.smithi184.stdout:(89/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 321 kB/s | 72 kB 00:00 2023-10-08T09:02:54.107 INFO:teuthology.orchestra.run.smithi132.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.7 MB/s | 798 kB 00:00 2023-10-08T09:02:54.113 INFO:teuthology.orchestra.run.smithi184.stdout:(90/131): fuse-2.9.7-15.el8.x86_64.rpm 761 kB/s | 83 kB 00:00 2023-10-08T09:02:54.155 INFO:teuthology.orchestra.run.smithi184.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 550 kB/s | 60 kB 00:00 2023-10-08T09:02:54.165 INFO:teuthology.orchestra.run.smithi132.stdout:(130/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 6.2 MB/s | 3.7 MB 00:00 2023-10-08T09:02:54.169 INFO:teuthology.orchestra.run.smithi049.stdout:(21/131): python3-rados-18.0.0-6639.g7b3143ba.e 3.3 MB/s | 371 kB 00:00 2023-10-08T09:02:54.180 INFO:teuthology.orchestra.run.smithi184.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 902 kB/s | 98 kB 00:00 2023-10-08T09:02:54.222 INFO:teuthology.orchestra.run.smithi184.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 530 kB/s | 58 kB 00:00 2023-10-08T09:02:54.273 INFO:teuthology.orchestra.run.smithi184.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 329 kB/s | 39 kB 00:00 2023-10-08T09:02:54.278 INFO:teuthology.orchestra.run.smithi049.stdout:(22/131): python3-rbd-18.0.0-6639.g7b3143ba.el8 3.3 MB/s | 365 kB 00:00 2023-10-08T09:02:54.315 INFO:teuthology.orchestra.run.smithi184.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.7 MB/s | 644 kB 00:00 2023-10-08T09:02:54.353 INFO:teuthology.orchestra.run.smithi049.stdout:(23/131): python3-rgw-18.0.0-6639.g7b3143ba.el8 1.4 MB/s | 107 kB 00:00 2023-10-08T09:02:54.365 INFO:teuthology.orchestra.run.smithi184.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.2 MB/s | 171 kB 00:00 2023-10-08T09:02:54.390 INFO:teuthology.orchestra.run.smithi184.stdout:(97/131): NetworkManager-initscripts-updown-1.3 1.1 MB/s | 138 kB 00:00 2023-10-08T09:02:54.420 INFO:teuthology.orchestra.run.smithi049.stdout:(24/131): rbd-fuse-18.0.0-6639.g7b3143ba.el8.x8 1.4 MB/s | 94 kB 00:00 2023-10-08T09:02:54.490 INFO:teuthology.orchestra.run.smithi184.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 879 kB/s | 110 kB 00:00 2023-10-08T09:02:54.516 INFO:teuthology.orchestra.run.smithi184.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 432 kB/s | 54 kB 00:00 2023-10-08T09:02:54.716 INFO:teuthology.orchestra.run.smithi184.stdout:(100/131): python3-scipy-1.0.0-21.module+el8.5. 34 MB/s | 14 MB 00:00 2023-10-08T09:02:54.741 INFO:teuthology.orchestra.run.smithi184.stdout:(101/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2023-10-08T09:02:54.766 INFO:teuthology.orchestra.run.smithi184.stdout:(102/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2023-10-08T09:02:54.859 INFO:teuthology.orchestra.run.smithi184.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 5.7 MB/s | 688 kB 00:00 2023-10-08T09:02:54.909 INFO:teuthology.orchestra.run.smithi184.stdout:(104/131): python3-markupsafe-0.23-19.el8.x86_6 202 kB/s | 39 kB 00:00 2023-10-08T09:02:54.942 INFO:teuthology.orchestra.run.smithi184.stdout:(105/131): python3-jinja2-2.10.1-3.el8.noarch.r 3.0 MB/s | 538 kB 00:00 2023-10-08T09:02:54.992 INFO:teuthology.orchestra.run.smithi184.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-10-08T09:02:55.018 INFO:teuthology.orchestra.run.smithi184.stdout:(107/131): python3-mako-1.0.6-13.el8.noarch.rpm 987 kB/s | 157 kB 00:00 2023-10-08T09:02:55.068 INFO:teuthology.orchestra.run.smithi184.stdout:(108/131): automake-1.16.1-7.el8.noarch.rpm 5.6 MB/s | 713 kB 00:00 2023-10-08T09:02:55.154 INFO:teuthology.orchestra.run.smithi049.stdout:(25/131): rbd-mirror-18.0.0-6639.g7b3143ba.el8. 4.9 MB/s | 3.6 MB 00:00 2023-10-08T09:02:55.210 INFO:teuthology.orchestra.run.smithi184.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-10-08T09:02:55.235 INFO:teuthology.orchestra.run.smithi184.stdout:(110/131): python3-jsonpatch-1.21-2.el8.noarch. 122 kB/s | 27 kB 00:00 2023-10-08T09:02:55.254 INFO:teuthology.orchestra.run.smithi049.stdout:(26/131): librgw2-18.0.0-6639.g7b3143ba.el8.x86 4.4 MB/s | 6.9 MB 00:01 2023-10-08T09:02:55.260 INFO:teuthology.orchestra.run.smithi184.stdout:(111/131): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-10-08T09:02:55.279 INFO:teuthology.orchestra.run.smithi049.stdout:(27/131): rbd-nbd-18.0.0-6639.g7b3143ba.el8.x86 1.4 MB/s | 179 kB 00:00 2023-10-08T09:02:55.313 INFO:teuthology.orchestra.run.smithi049.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6639.g 395 kB/s | 23 kB 00:00 2023-10-08T09:02:55.327 INFO:teuthology.orchestra.run.smithi184.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 375 kB/s | 44 kB 00:00 2023-10-08T09:02:55.338 INFO:teuthology.orchestra.run.smithi049.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6639.g7b3143b 2.2 MB/s | 132 kB 00:00 2023-10-08T09:02:55.428 INFO:teuthology.orchestra.run.smithi184.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2023-10-08T09:02:55.453 INFO:teuthology.orchestra.run.smithi184.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.8 MB/s | 225 kB 00:00 2023-10-08T09:02:55.487 INFO:teuthology.orchestra.run.smithi184.stdout:(115/131): python3-werkzeug-0.12.2-4.el8.noarch 2.0 MB/s | 457 kB 00:00 2023-10-08T09:02:55.533 INFO:teuthology.orchestra.run.smithi184.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 608 kB/s | 64 kB 00:00 2023-10-08T09:02:55.579 INFO:teuthology.orchestra.run.smithi184.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2023-10-08T09:02:55.596 INFO:teuthology.orchestra.run.smithi184.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 183 kB/s | 20 kB 00:00 2023-10-08T09:02:55.621 INFO:teuthology.orchestra.run.smithi184.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.2 MB/s | 103 kB 00:00 2023-10-08T09:02:55.672 INFO:teuthology.orchestra.run.smithi049.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6639.g7b314 4.9 MB/s | 1.8 MB 00:00 2023-10-08T09:02:55.705 INFO:teuthology.orchestra.run.smithi184.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-10-08T09:02:55.781 INFO:teuthology.orchestra.run.smithi049.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6639.g7b 2.2 MB/s | 242 kB 00:00 2023-10-08T09:02:55.813 INFO:teuthology.orchestra.run.smithi184.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 22 MB/s | 4.9 MB 00:00 2023-10-08T09:02:55.830 INFO:teuthology.orchestra.run.smithi184.stdout:(122/131): libpq-13.5-1.el8.x86_64.rpm 946 kB/s | 198 kB 00:00 2023-10-08T09:02:55.864 INFO:teuthology.orchestra.run.smithi049.stdout:(32/131): ceph-mgr-rook-18.0.0-6639.g7b3143ba.e 570 kB/s | 47 kB 00:00 2023-10-08T09:02:55.931 INFO:teuthology.orchestra.run.smithi184.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-10-08T09:02:55.940 INFO:teuthology.orchestra.run.smithi049.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6639.g7 193 kB/s | 15 kB 00:00 2023-10-08T09:02:55.956 INFO:teuthology.orchestra.run.smithi184.stdout:(124/131): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-10-08T09:02:55.981 INFO:teuthology.orchestra.run.smithi184.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2023-10-08T09:02:56.048 INFO:teuthology.orchestra.run.smithi049.stdout:(34/131): ceph-volume-18.0.0-6639.g7b3143ba.el8 2.4 MB/s | 261 kB 00:00 2023-10-08T09:02:56.090 INFO:teuthology.orchestra.run.smithi184.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-10-08T09:02:56.115 INFO:teuthology.orchestra.run.smithi049.stdout:(35/131): cephadm-18.0.0-6639.g7b3143ba.el8.noa 2.8 MB/s | 190 kB 00:00 2023-10-08T09:02:56.190 INFO:teuthology.orchestra.run.smithi184.stdout:(127/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-10-08T09:02:56.205 INFO:teuthology.orchestra.run.smithi049.stdout:(36/131): cephfs-top-18.0.0-6639.g7b3143ba.el8. 204 kB/s | 18 kB 00:00 2023-10-08T09:02:56.240 INFO:teuthology.orchestra.run.smithi184.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 5.8 MB/s | 1.5 MB 00:00 2023-10-08T09:02:56.366 INFO:teuthology.orchestra.run.smithi184.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.2 MB/s | 798 kB 00:00 2023-10-08T09:02:56.599 INFO:teuthology.orchestra.run.smithi049.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 777 kB/s | 306 kB 00:00 2023-10-08T09:02:56.691 INFO:teuthology.orchestra.run.smithi049.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 5.5 MB/s | 7.4 MB 00:01 2023-10-08T09:02:56.716 INFO:teuthology.orchestra.run.smithi049.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 503 kB/s | 59 kB 00:00 2023-10-08T09:02:56.813 INFO:teuthology.orchestra.run.smithi184.stdout:(130/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 6.0 MB/s | 3.7 MB 00:00 2023-10-08T09:02:56.850 INFO:teuthology.orchestra.run.smithi049.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-10-08T09:02:56.924 INFO:teuthology.orchestra.run.smithi184.stdout:(131/131): librados2-18.0.0-6639.g7b3143ba.el8. 4.5 MB/s | 3.7 MB 00:00 2023-10-08T09:02:56.927 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:02:56.927 INFO:teuthology.orchestra.run.smithi184.stdout:Total 14 MB/s | 209 MB 00:14 2023-10-08T09:02:56.959 INFO:teuthology.orchestra.run.smithi049.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-10-08T09:02:56.984 INFO:teuthology.orchestra.run.smithi049.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 257 kB/s | 75 kB 00:00 2023-10-08T09:02:57.043 INFO:teuthology.orchestra.run.smithi049.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-10-08T09:02:57.068 INFO:teuthology.orchestra.run.smithi049.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 394 kB/s | 33 kB 00:00 2023-10-08T09:02:57.118 INFO:teuthology.orchestra.run.smithi049.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-10-08T09:02:57.210 INFO:teuthology.orchestra.run.smithi049.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2023-10-08T09:02:57.235 INFO:teuthology.orchestra.run.smithi049.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2023-10-08T09:02:57.311 INFO:teuthology.orchestra.run.smithi049.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 812 kB/s | 81 kB 00:00 2023-10-08T09:02:57.328 INFO:teuthology.orchestra.run.smithi049.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 120 kB/s | 11 kB 00:00 2023-10-08T09:02:57.365 INFO:teuthology.orchestra.run.smithi132.stdout:(131/131): ceph-test-18.0.0-6639.g7b3143ba.el8. 3.5 MB/s | 49 MB 00:14 2023-10-08T09:02:57.369 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:02:57.369 INFO:teuthology.orchestra.run.smithi132.stdout:Total 13 MB/s | 209 MB 00:16 2023-10-08T09:02:57.378 INFO:teuthology.orchestra.run.smithi049.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 262 kB/s | 18 kB 00:00 2023-10-08T09:02:57.445 INFO:teuthology.orchestra.run.smithi049.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 732 kB/s | 49 kB 00:00 2023-10-08T09:02:57.520 INFO:teuthology.orchestra.run.smithi049.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 784 kB/s | 59 kB 00:00 2023-10-08T09:02:57.562 INFO:teuthology.orchestra.run.smithi049.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-10-08T09:02:57.596 INFO:teuthology.orchestra.run.smithi049.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 802 kB/s | 60 kB 00:00 2023-10-08T09:02:57.671 INFO:teuthology.orchestra.run.smithi049.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-10-08T09:02:57.688 INFO:teuthology.orchestra.run.smithi049.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 169 kB/s | 16 kB 00:00 2023-10-08T09:02:57.738 INFO:teuthology.orchestra.run.smithi049.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 504 kB/s | 34 kB 00:00 2023-10-08T09:02:57.739 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2023-10-08T09:02:57.764 INFO:teuthology.orchestra.run.smithi049.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-10-08T09:02:57.814 INFO:teuthology.orchestra.run.smithi049.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 852 kB/s | 64 kB 00:00 2023-10-08T09:02:57.848 INFO:teuthology.orchestra.run.smithi049.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2023-10-08T09:02:57.858 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2023-10-08T09:02:57.858 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2023-10-08T09:02:57.882 INFO:teuthology.orchestra.run.smithi049.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-10-08T09:02:57.924 INFO:teuthology.orchestra.run.smithi049.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-10-08T09:02:57.957 INFO:teuthology.orchestra.run.smithi049.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-10-08T09:02:58.024 INFO:teuthology.orchestra.run.smithi049.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-10-08T09:02:58.050 INFO:teuthology.orchestra.run.smithi049.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-10-08T09:02:58.100 INFO:teuthology.orchestra.run.smithi049.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2023-10-08T09:02:58.117 INFO:teuthology.orchestra.run.smithi049.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-10-08T09:02:58.175 INFO:teuthology.orchestra.run.smithi049.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-10-08T09:02:58.201 INFO:teuthology.orchestra.run.smithi049.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-08T09:02:58.237 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2023-10-08T09:02:58.293 INFO:teuthology.orchestra.run.smithi049.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 9.9 MB/s | 1.7 MB 00:00 2023-10-08T09:02:58.356 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2023-10-08T09:02:58.357 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2023-10-08T09:02:58.443 INFO:teuthology.orchestra.run.smithi049.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 735 kB/s | 110 kB 00:00 2023-10-08T09:02:58.560 INFO:teuthology.orchestra.run.smithi049.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 465 kB/s | 54 kB 00:00 2023-10-08T09:02:58.726 INFO:teuthology.orchestra.run.smithi049.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-10-08T09:02:58.761 INFO:teuthology.orchestra.run.smithi049.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-10-08T09:02:58.869 INFO:teuthology.orchestra.run.smithi049.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2023-10-08T09:02:58.894 INFO:teuthology.orchestra.run.smithi049.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 291 kB/s | 39 kB 00:00 2023-10-08T09:02:59.001 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2023-10-08T09:02:59.005 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2023-10-08T09:02:59.020 INFO:teuthology.orchestra.run.smithi049.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.2 MB/s | 538 kB 00:00 2023-10-08T09:02:59.053 INFO:teuthology.orchestra.run.smithi049.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-10-08T09:02:59.204 INFO:teuthology.orchestra.run.smithi049.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-10-08T09:02:59.229 INFO:teuthology.orchestra.run.smithi049.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 751 kB/s | 157 kB 00:00 2023-10-08T09:02:59.321 INFO:teuthology.orchestra.run.smithi049.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 6.0 MB/s | 713 kB 00:00 2023-10-08T09:02:59.470 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2023-10-08T09:02:59.474 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2023-10-08T09:02:59.479 INFO:teuthology.orchestra.run.smithi049.stdout:(82/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-10-08T09:02:59.505 INFO:teuthology.orchestra.run.smithi049.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 144 kB/s | 27 kB 00:00 2023-10-08T09:02:59.589 INFO:teuthology.orchestra.run.smithi049.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 527 kB/s | 44 kB 00:00 2023-10-08T09:02:59.622 INFO:teuthology.orchestra.run.smithi049.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2023-10-08T09:02:59.756 INFO:teuthology.orchestra.run.smithi049.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.3 MB/s | 457 kB 00:00 2023-10-08T09:02:59.931 INFO:teuthology.orchestra.run.smithi049.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-10-08T09:03:00.188 INFO:teuthology.orchestra.run.smithi049.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 521 kB/s | 225 kB 00:00 2023-10-08T09:03:00.224 INFO:teuthology.orchestra.run.smithi049.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 219 kB/s | 64 kB 00:00 2023-10-08T09:03:00.707 INFO:teuthology.orchestra.run.smithi049.stdout:(90/131): ceph-test-18.0.0-6639.g7b3143ba.el8.x 5.5 MB/s | 49 MB 00:08 2023-10-08T09:03:00.733 INFO:teuthology.orchestra.run.smithi049.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 39 kB/s | 20 kB 00:00 2023-10-08T09:03:00.766 INFO:teuthology.orchestra.run.smithi049.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 219 kB/s | 126 kB 00:00 2023-10-08T09:03:00.867 INFO:teuthology.orchestra.run.smithi049.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 653 kB/s | 103 kB 00:00 2023-10-08T09:03:00.950 INFO:teuthology.orchestra.run.smithi049.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2023-10-08T09:03:00.984 INFO:teuthology.orchestra.run.smithi049.stdout:(95/131): libtool-2.4.6-25.el8.x86_64.rpm 2.9 MB/s | 709 kB 00:00 2023-10-08T09:03:01.101 INFO:teuthology.orchestra.run.smithi049.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-10-08T09:03:01.151 INFO:teuthology.orchestra.run.smithi049.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-10-08T09:03:01.235 INFO:teuthology.orchestra.run.smithi049.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-10-08T09:03:01.276 INFO:teuthology.orchestra.run.smithi049.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.0 MB/s | 354 kB 00:00 2023-10-08T09:03:01.368 INFO:teuthology.orchestra.run.smithi049.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-10-08T09:03:01.452 INFO:teuthology.orchestra.run.smithi049.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 8.6 MB/s | 1.5 MB 00:00 2023-10-08T09:03:01.485 INFO:teuthology.orchestra.run.smithi049.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 401 kB/s | 47 kB 00:00 2023-10-08T09:03:01.552 INFO:teuthology.orchestra.run.smithi049.stdout:(103/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-10-08T09:03:01.603 INFO:teuthology.orchestra.run.smithi049.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2023-10-08T09:03:01.645 INFO:teuthology.orchestra.run.smithi049.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 268 kB/s | 43 kB 00:00 2023-10-08T09:03:01.728 INFO:teuthology.orchestra.run.smithi049.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 384 kB/s | 48 kB 00:00 2023-10-08T09:03:01.787 INFO:teuthology.orchestra.run.smithi049.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 941 kB/s | 220 kB 00:00 2023-10-08T09:03:01.812 INFO:teuthology.orchestra.run.smithi049.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 520 kB/s | 87 kB 00:00 2023-10-08T09:03:01.904 INFO:teuthology.orchestra.run.smithi049.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 243 kB/s | 43 kB 00:00 2023-10-08T09:03:01.929 INFO:teuthology.orchestra.run.smithi049.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 768 kB/s | 109 kB 00:00 2023-10-08T09:03:01.955 INFO:teuthology.orchestra.run.smithi049.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-10-08T09:03:02.047 INFO:teuthology.orchestra.run.smithi049.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.9 MB/s | 560 kB 00:00 2023-10-08T09:03:02.063 INFO:teuthology.orchestra.run.smithi049.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 140 kB/s | 19 kB 00:00 2023-10-08T09:03:02.089 INFO:teuthology.orchestra.run.smithi049.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-10-08T09:03:02.173 INFO:teuthology.orchestra.run.smithi049.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 164 kB/s | 18 kB 00:00 2023-10-08T09:03:02.223 INFO:teuthology.orchestra.run.smithi049.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.1 MB/s | 559 kB 00:00 2023-10-08T09:03:02.257 INFO:teuthology.orchestra.run.smithi049.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2023-10-08T09:03:02.290 INFO:teuthology.orchestra.run.smithi049.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.0 MB/s | 238 kB 00:00 2023-10-08T09:03:02.374 INFO:teuthology.orchestra.run.smithi049.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 15 MB/s | 2.3 MB 00:00 2023-10-08T09:03:02.399 INFO:teuthology.orchestra.run.smithi049.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 508 kB/s | 72 kB 00:00 2023-10-08T09:03:02.441 INFO:teuthology.orchestra.run.smithi049.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 550 kB/s | 83 kB 00:00 2023-10-08T09:03:02.516 INFO:teuthology.orchestra.run.smithi049.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 421 kB/s | 60 kB 00:00 2023-10-08T09:03:02.533 INFO:teuthology.orchestra.run.smithi049.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 733 kB/s | 98 kB 00:00 2023-10-08T09:03:02.567 INFO:teuthology.orchestra.run.smithi049.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 459 kB/s | 58 kB 00:00 2023-10-08T09:03:02.626 INFO:teuthology.orchestra.run.smithi049.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 354 kB/s | 39 kB 00:00 2023-10-08T09:03:02.676 INFO:teuthology.orchestra.run.smithi049.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.4 MB/s | 644 kB 00:00 2023-10-08T09:03:02.710 INFO:teuthology.orchestra.run.smithi049.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.2 MB/s | 171 kB 00:00 2023-10-08T09:03:02.760 INFO:teuthology.orchestra.run.smithi049.stdout:(128/131): NetworkManager-initscripts-updown-1. 1.0 MB/s | 138 kB 00:00 2023-10-08T09:03:02.910 INFO:teuthology.orchestra.run.smithi049.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.2 MB/s | 798 kB 00:00 2023-10-08T09:03:03.327 INFO:teuthology.orchestra.run.smithi049.stdout:(130/131): librados2-18.0.0-6639.g7b3143ba.el8. 5.8 MB/s | 3.7 MB 00:00 2023-10-08T09:03:03.564 INFO:teuthology.orchestra.run.smithi049.stdout:(131/131): librbd1-18.0.0-6639.g7b3143ba.el8.x8 4.3 MB/s | 3.7 MB 00:00 2023-10-08T09:03:03.566 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:03:03.567 INFO:teuthology.orchestra.run.smithi049.stdout:Total 9.7 MB/s | 209 MB 00:21 2023-10-08T09:03:03.872 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2023-10-08T09:03:04.092 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T09:03:04.283 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T09:03:04.444 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2023-10-08T09:03:04.452 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T09:03:04.504 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2023-10-08T09:03:04.577 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2023-10-08T09:03:04.577 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2023-10-08T09:03:04.598 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T09:03:04.717 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T09:03:04.819 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T09:03:04.926 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T09:03:05.111 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T09:03:05.217 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-08T09:03:05.228 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T09:03:05.410 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T09:03:05.436 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T09:03:05.770 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2023-10-08T09:03:05.774 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2023-10-08T09:03:05.821 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T09:03:05.842 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-08T09:03:06.002 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:03:06.052 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T09:03:06.129 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:03:06.343 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:03:06.448 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:03:06.461 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 5/134 2023-10-08T09:03:06.615 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:03:06.677 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:03:06.794 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:03:06.799 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-08T09:03:07.002 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:03:07.038 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T09:03:07.086 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:03:07.249 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:03:07.254 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T09:03:07.421 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-08T09:03:07.648 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T09:03:07.664 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T09:03:07.880 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T09:03:07.909 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T09:03:08.059 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:03:08.172 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:03:08.269 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T09:03:08.351 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:03:08.494 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:03:08.534 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T09:03:08.694 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:03:08.696 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-08T09:03:08.789 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:03:08.879 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-08T09:03:08.944 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:03:09.066 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-08T09:03:09.078 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:03:09.272 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-08T09:03:09.276 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T09:03:09.432 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-08T09:03:09.608 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-08T09:03:09.625 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T09:03:09.752 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T09:03:09.768 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T09:03:09.894 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-08T09:03:10.052 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T09:03:10.386 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T09:03:10.542 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T09:03:10.666 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-08T09:03:10.825 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2023-10-08T09:03:10.827 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T09:03:11.001 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-08T09:03:11.144 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T09:03:11.159 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-08T09:03:11.317 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T09:03:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-08T09:03:11.425 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-08T09:03:11.553 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:03:11.610 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T09:03:11.663 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-08T09:03:11.936 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:03:11.998 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T09:03:12.274 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-08T09:03:12.298 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-08T09:03:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-08T09:03:12.456 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-08T09:03:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:03:12.575 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-08T09:03:12.677 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T09:03:12.782 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-08T09:03:12.898 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T09:03:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:03:13.028 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T09:03:13.036 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T09:03:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:03:13.216 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T09:03:13.263 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T09:03:13.303 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:03:13.405 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T09:03:13.492 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T09:03:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:03:13.601 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:03:13.624 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T09:03:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:03:13.851 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T09:03:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-08T09:03:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-08T09:03:14.820 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-08T09:03:15.044 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-08T09:03:15.206 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:03:15.258 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T09:03:15.312 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 14/134 2023-10-08T09:03:15.452 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:03:15.522 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 15/134 2023-10-08T09:03:15.607 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-08T09:03:15.668 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:03:15.817 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-08T09:03:16.003 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:03:16.272 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-08T09:03:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-08T09:03:16.604 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-08T09:03:16.657 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-08T09:03:16.835 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:03:16.886 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-08T09:03:16.908 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-08T09:03:17.105 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-08T09:03:17.211 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-08T09:03:17.362 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T09:03:17.438 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-08T09:03:17.472 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-08T09:03:17.577 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T09:03:17.638 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T09:03:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:03:17.764 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T09:03:17.836 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T09:03:17.956 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T09:03:18.015 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T09:03:18.114 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-08T09:03:18.207 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T09:03:18.259 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-08T09:03:18.358 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-08T09:03:18.406 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-08T09:03:18.433 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T09:03:18.511 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T09:03:18.512 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-08T09:03:18.684 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T09:03:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-08T09:03:18.746 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-08T09:03:18.754 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-08T09:03:18.951 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-08T09:03:18.961 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-08T09:03:18.980 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-08T09:03:19.133 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T09:03:19.164 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-08T09:03:19.220 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-08T09:03:19.238 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T09:03:19.368 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-08T09:03:19.392 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T09:03:19.462 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-08T09:03:19.475 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-08T09:03:19.590 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-08T09:03:19.628 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-08T09:03:19.707 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-08T09:03:19.798 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-08T09:03:19.818 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-08T09:03:19.897 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-08T09:03:20.008 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 26/134 2023-10-08T09:03:20.076 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-08T09:03:20.131 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 27/134 2023-10-08T09:03:20.146 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-08T09:03:20.236 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-08T09:03:20.435 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-08T09:03:20.436 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-08T09:03:20.604 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-08T09:03:20.628 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-08T09:03:20.817 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-08T09:03:20.839 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-08T09:03:21.039 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-08T09:03:21.075 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-08T09:03:21.098 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 28/134 2023-10-08T09:03:21.434 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-08T09:03:21.437 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-08T09:03:21.459 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-08T09:03:21.709 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-08T09:03:21.751 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-08T09:03:21.777 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-08T09:03:21.990 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-08T09:03:22.003 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-08T09:03:22.019 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-08T09:03:22.201 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:03:22.237 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-08T09:03:22.454 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-08T09:03:22.491 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-08T09:03:22.706 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-08T09:03:22.773 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-08T09:03:22.965 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-08T09:03:23.075 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-08T09:03:23.167 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-08T09:03:23.283 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-08T09:03:23.387 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-08T09:03:23.502 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-08T09:03:23.553 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-08T09:03:23.714 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-08T09:03:23.727 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-08T09:03:23.913 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-08T09:03:24.070 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-08T09:03:24.078 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-08T09:03:24.090 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-08T09:03:24.319 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-08T09:03:24.372 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-08T09:03:24.578 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-08T09:03:24.580 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-08T09:03:24.638 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-08T09:03:24.799 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-08T09:03:24.825 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-08T09:03:24.921 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-08T09:03:24.956 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T09:03:25.005 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-08T09:03:25.151 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-08T09:03:25.151 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T09:03:25.207 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 36/134 2023-10-08T09:03:25.316 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T09:03:25.372 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 37/134 2023-10-08T09:03:25.390 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T09:03:25.494 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T09:03:25.567 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-08T09:03:25.613 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T09:03:25.741 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-08T09:03:25.753 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T09:03:25.777 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T09:03:25.909 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T09:03:25.961 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-08T09:03:25.981 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T09:03:26.106 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-08T09:03:26.106 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T09:03:26.129 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T09:03:26.286 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T09:03:26.301 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T09:03:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-08T09:03:26.502 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T09:03:26.528 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-08T09:03:26.534 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T09:03:26.710 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T09:03:26.721 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T09:03:26.854 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T09:03:26.896 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-08T09:03:26.928 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T09:03:27.094 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T09:03:27.113 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T09:03:27.197 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-08T09:03:27.241 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T09:03:27.361 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T09:03:27.428 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-08T09:03:27.514 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T09:03:27.545 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T09:03:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-08T09:03:27.795 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T09:03:27.802 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T09:03:27.890 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-08T09:03:27.989 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T09:03:28.014 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T09:03:28.154 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T09:03:28.234 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-08T09:03:28.239 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T09:03:28.381 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T09:03:28.465 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T09:03:28.475 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-08T09:03:28.647 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-08T09:03:28.674 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T09:03:28.883 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-08T09:03:28.909 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T09:03:29.047 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-08T09:03:29.246 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-08T09:03:29.462 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T09:03:29.559 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-08T09:03:29.759 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T09:03:29.847 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-08T09:03:29.997 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T09:03:30.011 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T09:03:30.109 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-08T09:03:30.278 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T09:03:30.323 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T09:03:30.348 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-08T09:03:30.499 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-08T09:03:30.515 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T09:03:30.581 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T09:03:30.677 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-08T09:03:30.835 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T09:03:30.849 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:03:30.866 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T09:03:30.922 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-08T09:03:31.148 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T09:03:31.190 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-08T09:03:31.359 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T09:03:31.413 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-08T09:03:31.541 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:03:31.625 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-08T09:03:31.662 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T09:03:31.822 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:03:31.865 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-08T09:03:31.903 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:03:32.018 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-08T09:03:32.064 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T09:03:32.188 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T09:03:32.191 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-08T09:03:32.363 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-08T09:03:32.365 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:03:32.443 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:03:32.635 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-08T09:03:32.649 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T09:03:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-08T09:03:33.029 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-08T09:03:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-08T09:03:33.530 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-08T09:03:33.748 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-08T09:03:33.947 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-08T09:03:34.111 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-08T09:03:34.239 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-08T09:03:34.507 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-08T09:03:34.754 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-08T09:03:34.914 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-08T09:03:35.194 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-08T09:03:35.399 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-08T09:03:35.592 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-08T09:03:35.802 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:35.845 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-08T09:03:36.328 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:36.464 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:36.573 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-08T09:03:37.047 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:37.159 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:37.318 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:03:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-08T09:03:37.425 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:03:37.452 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:03:37.536 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:03:37.554 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:03:37.692 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:03:37.702 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-08T09:03:38.013 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-08T09:03:38.282 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-08T09:03:38.512 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-08T09:03:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-08T09:03:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:03:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 92/134 2023-10-08T09:03:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 93/134 2023-10-08T09:03:43.936 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:44.139 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:44.755 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 94/134 2023-10-08T09:03:45.060 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:03:45.147 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 95/134 2023-10-08T09:03:45.289 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:03:56.654 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /run 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2023-10-08T09:03:56.655 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /dev 2023-10-08T09:03:56.656 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:03:56.723 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:03:56.723 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2023-10-08T09:03:56.723 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2023-10-08T09:03:56.723 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /dev 2023-10-08T09:03:56.724 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /run 2023-10-08T09:03:56.724 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2023-10-08T09:03:56.724 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2023-10-08T09:03:56.724 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2023-10-08T09:03:56.724 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2023-10-08T09:03:56.724 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /dev 2023-10-08T09:03:56.724 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:03:57.011 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:03:57.068 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:03:58.596 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:03:58.656 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:03:58.757 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:03:58.848 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:03:58.975 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:03:59.068 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:03:59.144 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:03:59.234 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:03:59.609 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:03:59.710 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:03:59.744 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:03:59.763 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:03:59.860 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:03:59.876 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:04:01.452 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:04:01.675 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:04:01.757 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:04:01.808 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T09:04:01.975 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:04:02.027 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T09:04:02.098 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T09:04:02.219 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:04:02.300 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T09:04:02.439 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:04:02.492 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:04:02.721 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:04:02.747 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:04:03.005 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:04:03.706 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:04:03.885 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:04:03.965 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:04:03.977 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:04:04.142 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:04:04.232 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:04:04.315 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T09:04:04.553 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:04:04.589 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:04:04.623 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T09:04:04.743 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:04:04.755 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:04:04.822 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:04:04.857 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:04:04.914 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 96/134 2023-10-08T09:04:04.914 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2023-10-08T09:04:04.914 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2023-10-08T09:04:04.914 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2023-10-08T09:04:04.914 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /run 2023-10-08T09:04:04.915 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2023-10-08T09:04:04.915 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2023-10-08T09:04:04.915 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2023-10-08T09:04:04.915 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2023-10-08T09:04:04.915 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2023-10-08T09:04:04.915 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:04:05.025 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:04:05.036 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:04:05.277 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:04:06.088 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T09:04:06.312 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:04:06.346 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:04:06.384 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T09:04:06.493 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:04:06.535 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:04:06.576 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:04:06.609 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:04:06.693 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:04:06.741 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:04:06.790 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:04:06.925 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:04:06.970 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 97/134 2023-10-08T09:04:07.057 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:04:07.156 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:04:07.251 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:04:07.298 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:04:07.390 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 98/134 2023-10-08T09:04:07.433 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:04:07.593 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:04:08.085 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 99/134 2023-10-08T09:04:08.252 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:04:08.269 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 100/134 2023-10-08T09:04:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:04:10.225 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 101/134 2023-10-08T09:04:10.350 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 102/134 2023-10-08T09:04:10.578 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 103/134 2023-10-08T09:04:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:04:10.962 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 104/134 2023-10-08T09:04:11.286 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:04:11.877 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:04:12.071 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:04:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 105/134 2023-10-08T09:04:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:04:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 106/134 2023-10-08T09:04:12.773 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T09:04:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 107/134 2023-10-08T09:04:12.960 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T09:04:13.104 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-08T09:04:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:04:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 108/134 2023-10-08T09:04:13.268 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-08T09:04:13.286 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 117/134 2023-10-08T09:04:13.384 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:04:13.396 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 109/134 2023-10-08T09:04:13.535 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 117/134 2023-10-08T09:04:13.553 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-08T09:04:13.708 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T09:04:13.776 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-08T09:04:13.898 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T09:04:13.923 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T09:04:14.113 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-08T09:04:14.171 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-08T09:04:14.353 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-08T09:04:14.355 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T09:04:14.510 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:04:14.510 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T09:04:14.649 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:04:14.807 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 110/134 2023-10-08T09:04:14.998 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:04:15.033 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 111/134 2023-10-08T09:04:15.217 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:04:15.261 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6639.g7b314 112/134 2023-10-08T09:04:15.409 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:04:15.791 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 113/134 2023-10-08T09:04:15.967 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:04:16.653 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:16.834 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:16.917 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:16.934 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:17.073 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:17.090 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:17.100 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T09:04:17.282 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T09:04:17.416 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T09:04:17.598 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T09:04:17.729 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T09:04:17.924 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T09:04:18.080 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T09:04:18.266 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T09:04:18.283 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T09:04:18.344 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T09:04:18.464 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T09:04:18.542 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T09:04:18.554 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T09:04:18.557 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-08T09:04:18.708 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T09:04:18.716 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T09:04:18.764 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T09:04:18.777 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-08T09:04:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-08T09:04:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:18.931 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T09:04:18.939 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-08T09:04:19.061 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:19.061 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:04:19.074 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-08T09:04:19.075 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:19.280 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:04:19.300 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:19.301 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:04:19.481 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:19.529 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:04:19.730 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:19.991 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:19.991 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T09:04:19.992 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T09:04:19.992 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T09:04:19.992 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T09:04:19.992 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T09:04:19.992 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T09:04:19.993 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T09:04:19.994 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T09:04:19.995 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T09:04:19.995 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T09:04:19.995 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T09:04:19.995 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T09:04:19.995 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T09:04:20.003 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T09:04:20.004 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T09:04:20.005 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T09:04:20.006 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-08T09:04:20.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-08T09:04:20.008 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-08T09:04:20.009 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-08T09:04:20.010 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-08T09:04:20.011 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:20.012 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:04:20.280 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T09:04:20.281 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T09:04:20.282 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T09:04:20.291 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T09:04:20.292 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T09:04:20.293 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-08T09:04:20.294 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-08T09:04:20.295 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-08T09:04:20.296 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-08T09:04:20.297 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-08T09:04:20.298 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:20.299 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T09:04:20.924 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 114/134 2023-10-08T09:04:21.371 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 115/134 2023-10-08T09:04:21.549 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:04:21.756 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-08T09:04:22.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:22.436 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:53 ago on Sun 08 Oct 2023 09:02:29 AM UTC. 2023-10-08T09:04:22.679 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-08T09:04:22.756 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:22.756 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:01:52 ago on Sun 08 Oct 2023 09:02:30 AM UTC. 2023-10-08T09:04:23.033 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-08T09:04:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 119/134 2023-10-08T09:04:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:04:23.579 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 121/134 2023-10-08T09:04:23.718 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:04:26.259 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:26.532 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:26.550 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-08T09:04:26.711 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 124/134 2023-10-08T09:04:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 125/134 2023-10-08T09:04:27.324 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 126/134 2023-10-08T09:04:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-08T09:04:27.781 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:04:27.791 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:04:28.018 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-08T09:04:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:04:28.265 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-08T09:04:28.482 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:04:28.494 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-08T09:04:28.495 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:28.701 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:28.701 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:04:28.913 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-08T09:04:29.131 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:29.675 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:29.675 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 1/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 3/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 4/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6639.g7b314 5/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 6/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 7/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 8/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 9/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 10/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 11/134 2023-10-08T09:04:29.676 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 12/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 13/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_ 14/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 15/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 16/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_6 17/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86 18/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 19/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el 20/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8. 21/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_6 22/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 23/134 2023-10-08T09:04:29.677 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 24/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 25/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 26/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 27/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 28/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba. 29/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noa 30/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.n 31/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3 32/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el 33/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 34/134 2023-10-08T09:04:29.678 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.e 35/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 36/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 37/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 38/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-08T09:04:29.679 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-08T09:04:29.680 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-08T09:04:29.681 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-08T09:04:29.682 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-08T09:04:29.683 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-08T09:04:29.684 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-08T09:04:29.685 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 129/134 2023-10-08T09:04:29.686 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-08T09:04:29.687 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 131/134 2023-10-08T09:04:29.687 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-08T09:04:29.687 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-08T09:04:30.539 INFO:teuthology.orchestra.run.smithi132.stdout:Installed products updated. 2023-10-08T09:04:30.753 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:04:30.753 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2023-10-08T09:04:30.753 INFO:teuthology.orchestra.run.smithi132.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T09:04:30.753 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.753 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.753 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.754 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.755 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: jq-1.6-3.el8.x86_64 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T09:04:30.756 INFO:teuthology.orchestra.run.smithi132.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.757 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T09:04:30.758 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T09:04:30.759 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T09:04:30.760 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T09:04:30.761 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T09:04:30.762 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T09:04:30.763 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T09:04:30.764 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T09:04:30.765 INFO:teuthology.orchestra.run.smithi132.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T09:04:30.765 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-10-08T09:04:30.765 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2023-10-08T09:04:30.936 DEBUG:teuthology.parallel:result is None 2023-10-08T09:04:31.527 INFO:teuthology.orchestra.run.smithi184.stdout:Installed products updated. 2023-10-08T09:04:31.740 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:04:31.740 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.741 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.742 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.6-3.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T09:04:31.744 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T09:04:31.745 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T09:04:31.746 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T09:04:31.747 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:31.747 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:31.747 INFO:teuthology.orchestra.run.smithi184.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:31.747 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T09:04:31.747 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T09:04:31.747 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T09:04:31.747 INFO:teuthology.orchestra.run.smithi184.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T09:04:31.748 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T09:04:31.749 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T09:04:31.750 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T09:04:31.751 INFO:teuthology.orchestra.run.smithi184.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.752 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T09:04:31.753 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:04:31.754 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-10-08T09:04:32.346 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-08T09:04:32.346 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:02:03 ago on Sun 08 Oct 2023 09:02:29 AM UTC. 2023-10-08T09:04:33.638 DEBUG:teuthology.parallel:result is None 2023-10-08T09:04:40.965 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2023-10-08T09:04:41.201 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:04:41.201 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T09:04:41.202 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.203 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.204 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:18.0.0-6639.g7b3143ba.el8.noarch 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-3.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-08T09:04:41.205 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libpq-13.5-1.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.206 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T09:04:41.207 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T09:04:41.208 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-08T09:04:41.209 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T09:04:41.210 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T09:04:41.211 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T09:04:41.212 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T09:04:41.213 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.214 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.214 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:18.0.0-6639.g7b3143ba.el8.x86_64 2023-10-08T09:04:41.214 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-08T09:04:41.214 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T09:04:41.214 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-08T09:04:41.214 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-08T09:04:41.214 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T09:04:41.215 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-08T09:04:41.215 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:04:41.215 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2023-10-08T09:04:41.372 DEBUG:teuthology.parallel:result is None 2023-10-08T09:04:41.372 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:04:41.585 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T09:04:41.609 INFO:teuthology.orchestra.run.smithi049.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T09:04:41.610 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T09:04:41.610 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T09:04:41.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:04:41.828 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T09:04:41.856 INFO:teuthology.orchestra.run.smithi132.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T09:04:41.856 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T09:04:41.856 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T09:04:41.856 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:04:42.067 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T09:04:42.096 INFO:teuthology.orchestra.run.smithi184.stdout:18.0.0-6639.g7b3143ba.el8 2023-10-08T09:04:42.096 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6639.g7b3143ba.el8 2023-10-08T09:04:42.096 INFO:teuthology.task.install:The correct ceph version 18.0.0-6639.g7b3143ba is installed. 2023-10-08T09:04:42.096 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T09:04:42.097 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:04:42.097 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T09:04:42.142 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:04:42.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T09:04:42.190 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:04:42.190 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T09:04:42.240 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T09:04:42.240 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:04:42.240 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T09:04:42.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T09:04:42.338 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:04:42.338 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T09:04:42.369 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T09:04:42.438 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:04:42.438 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T09:04:42.471 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T09:04:42.544 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T09:04:42.544 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:04:42.544 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T09:04:42.574 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T09:04:42.643 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:04:42.643 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T09:04:42.673 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T09:04:42.743 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:04:42.743 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T09:04:42.775 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T09:04:42.847 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T09:04:42.848 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:04:42.848 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T09:04:42.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T09:04:42.946 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:04:42.946 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T09:04:42.976 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T09:04:43.045 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:04:43.045 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T09:04:43.077 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T09:04:43.150 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-08T09:04:43.232 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-10-08T09:04:43.232 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:04:43.232 INFO:tasks.cephadm:Cluster fsid is b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:04:43.233 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-08T09:04:43.233 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.49', 'mon.b': '172.21.15.132', 'mon.c': '172.21.15.184'} 2023-10-08T09:04:43.233 INFO:tasks.cephadm:First mon is mon.a on smithi049 2023-10-08T09:04:43.233 INFO:tasks.cephadm:First mgr is x 2023-10-08T09:04:43.233 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-08T09:04:43.234 DEBUG:teuthology.orchestra.run.smithi049:> sudo hostname $(hostname -s) 2023-10-08T09:04:43.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname $(hostname -s) 2023-10-08T09:04:43.298 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname $(hostname -s) 2023-10-08T09:04:43.331 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-08T09:04:43.332 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:04:43.485 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73096/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6639-g7b3143ba', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6639.g7b3143ba'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-07 17:16:01.199993', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20231007.154158', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-08T09:04:43.698 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20231007.154158, sha1 7b3143bae3b6870a749bfb9810ed6e7d0e89d7de from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=7b3143bae3b6870a749bfb9810ed6e7d0e89d7de 2023-10-08T09:04:43.698 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm 2023-10-08T09:04:43.698 DEBUG:teuthology.orchestra.run.smithi049:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:44.271 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 09:04 /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:44.271 DEBUG:teuthology.orchestra.run.smithi132:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:44.848 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 09:04 /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:44.848 DEBUG:teuthology.orchestra.run.smithi184:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231007.154158/7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:45.435 INFO:teuthology.orchestra.run.smithi184.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 8 09:04 /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:45.436 DEBUG:teuthology.orchestra.run.smithi049:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:45.458 DEBUG:teuthology.orchestra.run.smithi132:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:45.479 DEBUG:teuthology.orchestra.run.smithi184:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-08T09:04:45.515 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de on all hosts... 2023-10-08T09:04:45.515 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T09:04:45.518 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T09:04:45.521 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de pull 2023-10-08T09:04:47.719 INFO:teuthology.orchestra.run.smithi184.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:04:47.873 INFO:teuthology.orchestra.run.smithi132.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:04:50.619 INFO:teuthology.orchestra.run.smithi049.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:05:19.258 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-10-08T09:05:19.259 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:05:19.259 INFO:teuthology.orchestra.run.smithi132.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T09:05:19.259 INFO:teuthology.orchestra.run.smithi132.stdout: "repo_digests": [ 2023-10-08T09:05:19.259 INFO:teuthology.orchestra.run.smithi132.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T09:05:19.259 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2023-10-08T09:05:19.259 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-10-08T09:05:19.318 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-10-08T09:05:19.318 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:05:19.319 INFO:teuthology.orchestra.run.smithi184.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T09:05:19.319 INFO:teuthology.orchestra.run.smithi184.stdout: "repo_digests": [ 2023-10-08T09:05:19.319 INFO:teuthology.orchestra.run.smithi184.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T09:05:19.319 INFO:teuthology.orchestra.run.smithi184.stdout: ] 2023-10-08T09:05:19.319 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-10-08T09:05:26.044 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:05:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:05:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "image_id": "09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492", 2023-10-08T09:05:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "repo_digests": [ 2023-10-08T09:05:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:75b5faa12438709dee04dcec4961d708afb26e7c47ec80421dede7e5d40615e1" 2023-10-08T09:05:26.044 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:05:26.044 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:05:26.064 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph 2023-10-08T09:05:26.095 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph 2023-10-08T09:05:26.133 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph 2023-10-08T09:05:26.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /etc/ceph 2023-10-08T09:05:26.198 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /etc/ceph 2023-10-08T09:05:26.229 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /etc/ceph 2023-10-08T09:05:26.261 INFO:tasks.cephadm:Writing seed config... 2023-10-08T09:05:26.262 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-08T09:05:26.263 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-08T09:05:26.264 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-08T09:05:26.265 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-08T09:05:26.265 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-08T09:05:26.265 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-08T09:05:26.265 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-08T09:05:26.266 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:05:26.266 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-08T09:05:26.283 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b8694e94-65b9-11ee-8db5-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2023-10-08T09:05:26.284 DEBUG:teuthology.orchestra.run.smithi049:mon.a> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a.service 2023-10-08T09:05:26.327 DEBUG:teuthology.orchestra.run.smithi049:mgr.x> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.x.service 2023-10-08T09:05:26.353 INFO:journalctl@ceph.mon.a.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:05:26.354 INFO:tasks.cephadm:Bootstrapping... 2023-10-08T09:05:26.354 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de -v bootstrap --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.49 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:05:26.380 INFO:journalctl@ceph.mgr.x.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:05:26.526 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2023-10-08T09:05:26.526 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', '-v', 'bootstrap', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.49', '--skip-admin-label'] 2023-10-08T09:05:26.671 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.1.1 2023-10-08T09:05:26.672 INFO:teuthology.orchestra.run.smithi049.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-08T09:05:26.672 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying podman|docker is present... 2023-10-08T09:05:26.819 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.1.1 2023-10-08T09:05:26.820 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying lvm2 is present... 2023-10-08T09:05:26.820 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying time synchronization is in place... 2023-10-08T09:05:26.827 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-08T09:05:26.827 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-08T09:05:26.832 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-08T09:05:26.833 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2023-10-08T09:05:26.838 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2023-10-08T09:05:26.845 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2023-10-08T09:05:26.845 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2023-10-08T09:05:26.845 INFO:teuthology.orchestra.run.smithi049.stdout:Repeating the final host check... 2023-10-08T09:05:26.989 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 4.1.1 2023-10-08T09:05:26.990 INFO:teuthology.orchestra.run.smithi049.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-08T09:05:26.990 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl is present 2023-10-08T09:05:26.990 INFO:teuthology.orchestra.run.smithi049.stdout:lvcreate is present 2023-10-08T09:05:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-08T09:05:26.996 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-08T09:05:27.001 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-08T09:05:27.002 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2023-10-08T09:05:27.007 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout enabled 2023-10-08T09:05:27.014 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout active 2023-10-08T09:05:27.014 INFO:teuthology.orchestra.run.smithi049.stdout:Unit chronyd.service is enabled and running 2023-10-08T09:05:27.014 INFO:teuthology.orchestra.run.smithi049.stdout:Host looks OK 2023-10-08T09:05:27.014 INFO:teuthology.orchestra.run.smithi049.stdout:Cluster fsid: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:27.014 INFO:teuthology.orchestra.run.smithi049.stdout:Acquiring lock 140019308657128 on /run/cephadm/b8694e94-65b9-11ee-8db5-212e2dc638e7.lock 2023-10-08T09:05:27.014 INFO:teuthology.orchestra.run.smithi049.stdout:Lock 140019308657128 acquired on /run/cephadm/b8694e94-65b9-11ee-8db5-212e2dc638e7.lock 2023-10-08T09:05:27.015 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 3300 ... 2023-10-08T09:05:27.015 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying IP 172.21.15.49 port 6789 ... 2023-10-08T09:05:27.015 INFO:teuthology.orchestra.run.smithi049.stdout:Base mon IP(s) is [172.21.15.49:3300, 172.21.15.49:6789], mon addrv is [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2023-10-08T09:05:27.018 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-10-08T09:05:27.019 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.49 metric 100 2023-10-08T09:05:27.021 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-08T09:05:27.021 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-10-08T09:05:27.021 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1780sec hoplimit 64 pref medium 2023-10-08T09:05:27.024 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-08T09:05:27.024 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-08T09:05:27.024 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-08T09:05:27.024 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-10-08T09:05:27.024 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-10-08T09:05:27.024 INFO:teuthology.orchestra.run.smithi049.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-08T09:05:27.025 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2023-10-08T09:05:27.025 INFO:teuthology.orchestra.run.smithi049.stdout:Mon IP `172.21.15.49` is in CIDR network `172.21.0.0/20` 2023-10-08T09:05:27.025 INFO:teuthology.orchestra.run.smithi049.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-08T09:05:27.026 INFO:teuthology.orchestra.run.smithi049.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-08T09:05:27.027 INFO:teuthology.orchestra.run.smithi049.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:05:27.469 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stdout 09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492 2023-10-08T09:05:27.469 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de... 2023-10-08T09:05:27.469 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Getting image source signatures 2023-10-08T09:05:27.470 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:1016a809630cb60ade9603521911a0725a35bac1f67b6518836ae2dd33bb1dad 2023-10-08T09:05:27.470 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying blob sha256:be9ed931e0f930233626f1c4d225977fbcc33e056e1a8fc67c7f63553d9455a2 2023-10-08T09:05:27.470 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Copying config sha256:09a1f3752b5d744decb566ac63bdba7c424478910988ca16cd520bc6ac368492 2023-10-08T09:05:27.470 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-08T09:05:27.470 INFO:teuthology.orchestra.run.smithi049.stdout:/bin/podman: stderr Storing signatures 2023-10-08T09:05:28.322 INFO:teuthology.orchestra.run.smithi049.stdout:ceph: stdout ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev) 2023-10-08T09:05:28.323 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph version: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev) 2023-10-08T09:05:28.323 INFO:teuthology.orchestra.run.smithi049.stdout:Extracting ceph user uid/gid from container image... 2023-10-08T09:05:28.975 INFO:teuthology.orchestra.run.smithi049.stdout:stat: stdout 167 167 2023-10-08T09:05:28.976 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial keys... 2023-10-08T09:05:30.225 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQDZcCJlWpxWFRAA8t+HNFePcLtha57szzU+fg== 2023-10-08T09:05:30.977 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQDacCJlxhTeIhAAVrii3Z+SrHUzIBOszGq7BQ== 2023-10-08T09:05:31.822 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph-authtool: stdout AQDbcCJl1RfiExAAVN13T9fXh3rSnaTPZFVgQA== 2023-10-08T09:05:31.823 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial monmap... 2023-10-08T09:05:32.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-08T09:05:32.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-08T09:05:32.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:32.773 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-08T09:05:32.773 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool for a [v2:172.21.15.49:3300,v1:172.21.15.49:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-08T09:05:32.773 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = pacific 2023-10-08T09:05:32.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: set fsid to b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:32.774 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-08T09:05:32.774 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:05:32.774 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mon... 2023-10-08T09:05:33.624 INFO:teuthology.orchestra.run.smithi049.stdout:create mon.a on 2023-10-08T09:05:33.748 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-08T09:05:33.838 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-08T09:05:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7.target. 2023-10-08T09:05:33.960 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7.target → /etc/systemd/system/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7.target. 2023-10-08T09:05:34.143 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a 2023-10-08T09:05:34.143 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a.service: Unit ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a.service not loaded. 2023-10-08T09:05:34.231 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7.target.wants/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@.service. 2023-10-08T09:05:34.541 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:34 smithi049 systemd[1]: Starting Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:05:34.828 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:34 smithi049 podman[98945]: 2023-10-08T09:05:34.829 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:34 smithi049 bash[98945]: ddb10a7d39ef8f96670c7977ebf963d219e9f7d7912090f3efb15c5f930defb8 2023-10-08T09:05:34.829 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:34 smithi049 ceph-mon[98985]: mkfs b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:34.829 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:34 smithi049 ceph-mon[98985]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T09:05:34.841 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:05:34.841 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2023-10-08T09:05:34.849 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:05:34.849 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2023-10-08T09:05:34.849 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2023-10-08T09:05:34.849 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2023-10-08T09:05:34.849 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon to start... 2023-10-08T09:05:34.849 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mon... 2023-10-08T09:05:35.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:34 smithi049 systemd[1]: Started Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:05:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:35 smithi049 ceph-mon[98985]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T09:05:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:35 smithi049 ceph-mon[98985]: monmap e1: 1 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0]} removed_ranks: {} 2023-10-08T09:05:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:35 smithi049 ceph-mon[98985]: fsmap 2023-10-08T09:05:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:35 smithi049 ceph-mon[98985]: osdmap e1: 0 total, 0 up, 0 in 2023-10-08T09:05:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:35 smithi049 ceph-mon[98985]: mgrmap e1: no daemons active 2023-10-08T09:05:37.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:36 smithi049 ceph-mon[98985]: from='client.? 172.21.15.49:0/1916633901' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-08T09:05:37.701 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout cluster: 2023-10-08T09:05:37.703 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout id: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:37.703 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-08T09:05:37.703 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:37.703 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout services: 2023-10-08T09:05:37.703 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 1.60182s) 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout data: 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout pgs: 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:mon is available 2023-10-08T09:05:37.704 INFO:teuthology.orchestra.run.smithi049.stdout:Assimilating anything we can from ceph.conf... 2023-10-08T09:05:38.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:38 smithi049 ceph-mon[98985]: from='client.? 172.21.15.49:0/69498817' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-08T09:05:38.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:38 smithi049 ceph-mon[98985]: from='client.? 172.21.15.49:0/69498817' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.49:3300,v1:172.21.15.49:6789] 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-08T09:05:38.753 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-08T09:05:38.754 INFO:teuthology.orchestra.run.smithi049.stdout:Generating new minimal ceph.conf... 2023-10-08T09:05:39.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:39 smithi049 ceph-mon[98985]: from='client.? 172.21.15.49:0/82052039' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:05:39.693 INFO:teuthology.orchestra.run.smithi049.stdout:Restarting the monitor... 2023-10-08T09:05:40.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:39 smithi049 systemd[1]: Stopping Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:05:40.279 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:40 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-a[98961]: 2023-10-08T09:05:40.024+0000 7fd0b3b16700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:05:40.279 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:40 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-a[98961]: 2023-10-08T09:05:40.024+0000 7fd0b3b16700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-08T09:05:40.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:40 smithi049 bash[99320]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-a 2023-10-08T09:05:40.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:40 smithi049 systemd[1]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-08T09:05:40.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:40 smithi049 systemd[1]: Stopped Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:05:40.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:40 smithi049 systemd[1]: Starting Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:05:41.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 podman[99445]: 2023-10-08T09:05:41.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: set uid:gid to 167:167 (ceph:ceph) 2023-10-08T09:05:41.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev), process ceph-mon, pid 7 2023-10-08T09:05:41.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: pidfile_write: ignore empty --pid-file 2023-10-08T09:05:41.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 bash[99445]: f54a132ddc754ded0f9f6b8b06bee8f7fdefa74d5edad07755b8ef83c6823f42 2023-10-08T09:05:41.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: load: jerasure load: lrc 2023-10-08T09:05:41.365 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: RocksDB version: 7.9.2 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Git sha 0 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Compile date 2023-10-07 16:15:44 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: DB SUMMARY 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: DB Session ID: ZF18YERD5N3KZML6L4VW 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: CURRENT file: CURRENT 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: IDENTITY file: IDENTITY 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 90000 ; 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.error_if_exists: 0 2023-10-08T09:05:41.366 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.create_if_missing: 0 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.paranoid_checks: 1 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.env: 0x55696d4aeda0 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.fs: PosixFileSystem 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.info_log: 0x55696e2d3640 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_file_opening_threads: 16 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.statistics: (nil) 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.use_fsync: 0 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_log_file_size: 0 2023-10-08T09:05:41.367 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.keep_log_file_num: 1000 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.recycle_log_file_num: 0 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.allow_fallocate: 1 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.allow_mmap_reads: 0 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.allow_mmap_writes: 0 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.use_direct_reads: 0 2023-10-08T09:05:41.368 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.create_missing_column_families: 0 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.db_log_dir: 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.wal_dir: 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-08T09:05:41.369 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.advise_random_on_open: 1 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.db_write_buffer_size: 0 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.write_buffer_manager: 0x55696e28b360 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.rate_limiter: (nil) 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.wal_recovery_mode: 2 2023-10-08T09:05:41.370 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.enable_thread_tracking: 0 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.enable_pipelined_write: 0 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.unordered_write: 0 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.row_cache: None 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.wal_filter: None 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.allow_ingest_behind: 0 2023-10-08T09:05:41.371 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.two_write_queues: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.manual_wal_flush: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.wal_compression: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.atomic_flush: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.log_readahead_size: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.best_efforts_recovery: 0 2023-10-08T09:05:41.372 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.allow_data_in_errors: 0 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.db_host_id: __hostname__ 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_background_jobs: 2 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_background_compactions: -1 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_subcompactions: 1 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-08T09:05:41.373 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_total_wal_size: 0 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_open_files: -1 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bytes_per_sync: 0 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-08T09:05:41.374 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_readahead_size: 0 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_background_flushes: -1 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Compression algorithms supported: 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kZSTD supported: 0 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kXpressCompression supported: 0 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kLZ4HCCompression supported: 1 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kZlibCompression supported: 1 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kSnappyCompression supported: 1 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kLZ4Compression supported: 1 2023-10-08T09:05:41.375 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: kBZip2Compression supported: 0 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.merge_operator: 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_filter: None 2023-10-08T09:05:41.376 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_filter_factory: None 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.sst_partitioner_factory: None 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55696e2d3740) 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks: 1 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-08T09:05:41.377 INFO:journalctl@ceph.mon.a.smithi049.stdout: pin_top_level_index_and_filter: 1 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_type: 0 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_index_type: 0 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_shortening: 1 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: checksum: 4 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: no_block_cache: 0 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache: 0x55696e3291f0 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_name: BinnedLRUCache 2023-10-08T09:05:41.378 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_options: 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: capacity : 536870912 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_shard_bits : 4 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: strict_capacity_limit : 0 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: high_pri_pool_ratio: 0.000 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_cache_compressed: (nil) 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: persistent_cache: (nil) 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size: 4096 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_size_deviation: 10 2023-10-08T09:05:41.379 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_restart_interval: 16 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: index_block_restart_interval: 1 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: metadata_block_size: 4096 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: partition_filters: 0 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: use_delta_encoding: 1 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: filter_policy: bloomfilter 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: whole_key_filtering: 1 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: verify_compression: 0 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: read_amp_bytes_per_bit: 0 2023-10-08T09:05:41.380 INFO:journalctl@ceph.mon.a.smithi049.stdout: format_version: 5 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_index_compression: 1 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout: block_align: 0 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout: max_auto_readahead_size: 262144 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout: prepopulate_block_cache: 0 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout: initial_auto_readahead_size: 8192 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout: num_file_reads_for_auto_readahead: 2 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.write_buffer_size: 33554432 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_write_buffer_number: 2 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression: NoCompression 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression: Disabled 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.prefix_extractor: nullptr 2023-10-08T09:05:41.381 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.num_levels: 7 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-08T09:05:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.level: 32767 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.strategy: 0 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.enabled: false 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:05:41.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.target_file_size_base: 67108864 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-08T09:05:41.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.arena_block_size: 1048576 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.disable_auto_compactions: 0 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-08T09:05:41.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-08T09:05:41.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-08T09:05:41.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-08T09:05:41.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-08T09:05:41.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-08T09:05:41.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-08T09:05:41.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-08T09:05:41.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.table_properties_collectors: 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.inplace_update_support: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.bloom_locality: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.max_successive_merges: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.paranoid_file_checks: 0 2023-10-08T09:05:41.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.force_consistency_checks: 1 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.report_bg_io_stats: 0 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.ttl: 2592000 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.enable_blob_files: false 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.min_blob_size: 0 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.blob_file_size: 268435456 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-08T09:05:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.blob_file_starting_level: 0 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: acd1e0aa-7677-4009-883f-f6ebdf95fd55 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696755941287938, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696755941289380, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85870, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 287, "table_properties": {"data_size": 83955, "index_size": 261, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11277, "raw_average_key_size": 48, "raw_value_size": 77586, "raw_average_value_size": 331, "num_data_blocks": 11, "num_entries": 234, "num_filter_entries": 234, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696755941, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "acd1e0aa-7677-4009-883f-f6ebdf95fd55", "db_session_id": "ZF18YERD5N3KZML6L4VW", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-08T09:05:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696755941289489, "job": 1, "event": "recovery_finished"} 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55696e3c6000 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: DB pointer 0x55696e3b2000 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** DB Stats ** 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:05:41.391 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: L0 2/0 85.69 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: Sum 2/0 85.69 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** Compaction Stats [default] ** 2023-10-08T09:05:41.392 INFO:journalctl@ceph.mon.a.smithi049.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 96.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-08T09:05:41.393 INFO:journalctl@ceph.mon.a.smithi049.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout: Cumulative compaction: 0.00 GB write, 18.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout: Interval compaction: 0.00 GB write, 18.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache BinnedLRUCache@0x55696e3291f0#7 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: starting mon.a rank 0 at public addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] at bind addrs [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???) e1 preinit fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:41.394 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???).mds e1 new map 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???).mds e1 print_map 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout: e1 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout: legacy client fscid: -1 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout: 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout: No filesystems configured 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:05:41.395 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-08T09:05:41.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-08T09:05:41.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: monmap e1: 1 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0]} removed_ranks: {} 2023-10-08T09:05:41.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: fsmap 2023-10-08T09:05:41.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: osdmap e1: 0 total, 0 up, 0 in 2023-10-08T09:05:41.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 ceph-mon[99485]: mgrmap e1: no daemons active 2023-10-08T09:05:41.396 INFO:teuthology.orchestra.run.smithi049.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-08T09:05:41.616 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:41 smithi049 systemd[1]: Started Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:05:42.422 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-08T09:05:42.449 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:05:42.449 INFO:teuthology.orchestra.run.smithi049.stdout:Creating mgr... 2023-10-08T09:05:42.449 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-08T09:05:42.450 INFO:teuthology.orchestra.run.smithi049.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-08T09:05:42.659 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.x 2023-10-08T09:05:42.659 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.x.service: Unit ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.x.service not loaded. 2023-10-08T09:05:42.758 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7.target.wants/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@.service. 2023-10-08T09:05:43.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:42 smithi049 systemd[1]: Starting Ceph mgr.x for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:05:43.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:43 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/305511876' entity='client.admin' 2023-10-08T09:05:43.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:43 smithi049 podman[99717]: 2023-10-08T09:05:43.467 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:05:43.467 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2023-10-08T09:05:43.474 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:05:43.475 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2023-10-08T09:05:43.475 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2023-10-08T09:05:43.475 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to enable service . firewalld.service is not available 2023-10-08T09:05:43.483 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:05:43.483 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2023-10-08T09:05:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:05:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2023-10-08T09:05:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2023-10-08T09:05:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-08T09:05:43.491 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr to start... 2023-10-08T09:05:43.492 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr... 2023-10-08T09:05:43.718 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:43 smithi049 bash[99717]: c5c33412444564618d27621a4ea3d8856a76108cdf213f3393ea7ddb7941d2a2 2023-10-08T09:05:43.718 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:43 smithi049 systemd[1]: Started Ceph mgr.x for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:05:43.719 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:43.508+0000 7fe57f7da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:05:43.719 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:43.606+0000 7fe57f7da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:05:43.719 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:43.717+0000 7fe57f7da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:05:44.054 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:43.833+0000 7fe57f7da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:05:44.333 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:44 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/203248123' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:05:44.333 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:44.052+0000 7fe57f7da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:05:44.596 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:44.542+0000 7fe57f7da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:05:44.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:44.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:05:44.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:05:44.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:05:44.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:05:44.642 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:05:44.643 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:05:44.644 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:05:44.647 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:05:44.648 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:05:44.649 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:05:34.761207+0000", 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:44.650 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:05:44.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:05:44.651 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (1/15)... 2023-10-08T09:05:44.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:44.694+0000 7fe57f7da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:05:44.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:44.803+0000 7fe57f7da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:05:45.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:44.918+0000 7fe57f7da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:05:45.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:45 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:45.429+0000 7fe57f7da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:05:46.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:46 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:46.224+0000 7fe57f7da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:05:46.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:46 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:46.322+0000 7fe57f7da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:05:47.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:47 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1563974401' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:05:47.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:47 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:47.336+0000 7fe57f7da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:05:47.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:47 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:47.451+0000 7fe57f7da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:05:47.798 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:05:47.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:05:47.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:05:47.801 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:05:47.804 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:47.805 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:05:34.761207+0000", 2023-10-08T09:05:47.806 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:47.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:47.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:05:47.807 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:05:47.807 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (2/15)... 2023-10-08T09:05:48.030 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:47 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:47.675+0000 7fe57f7da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:05:48.285 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:48.029+0000 7fe57f7da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:05:48.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:48.284+0000 7fe57f7da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:05:48.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:48.467+0000 7fe57f7da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:05:49.132 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:48.797+0000 7fe57f7da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:05:49.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:49 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:49.623+0000 7fe57f7da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:05:49.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:49 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:49.782+0000 7fe57f7da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:05:50.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:50.035+0000 7fe57f7da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:05:50.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:50.173+0000 7fe57f7da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:05:50.804 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:50.450+0000 7fe57f7da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:05:51.881 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:05:51 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:05:51.511+0000 7fe57f7da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:05:52.556 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: Activating manager daemon x 2023-10-08T09:05:52.556 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: mgrmap e2: x(active, starting, since 0.00281342s) 2023-10-08T09:05:52.556 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T09:05:52.556 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: Manager daemon x is now available 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='mgr.14100 172.21.15.49:0/4146629389' entity='mgr.x' 2023-10-08T09:05:52.557 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:52 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2306099427' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:05:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:53 smithi049 ceph-mon[99485]: mgrmap e3: x(active, since 1.00772s) 2023-10-08T09:05:54.151 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:54.156 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:05:54.156 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:05:54.157 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:05:54.158 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:05:54.162 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:05:54.163 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:05:34.761207+0000", 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:54.164 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:05:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:05:54.165 INFO:teuthology.orchestra.run.smithi049.stdout:mgr not available, waiting (3/15)... 2023-10-08T09:05:54.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:54 smithi049 ceph-mon[99485]: mgrmap e4: x(active, since 2s) 2023-10-08T09:05:57.615 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "health": { 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-08T09:05:57.616 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 0 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "a" 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:57.617 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-08T09:05:57.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-08T09:05:57.618 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-08T09:05:57.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-08T09:05:57.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-08T09:05:57.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-08T09:05:57.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:57.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-08T09:05:57.621 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-08T09:05:57.622 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "restful" 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ], 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:57.623 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-08T09:05:57.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-08T09:05:57.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "modified": "2023-10-08T09:05:34.761207+0000", 2023-10-08T09:05:57.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-08T09:05:57.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout }, 2023-10-08T09:05:57.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-08T09:05:57.624 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:05:57.624 INFO:teuthology.orchestra.run.smithi049.stdout:mgr is available 2023-10-08T09:05:57.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:57 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1673969627' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-08T09:05:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:05:58 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1606671601' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-08T09:05:58.962 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:58.962 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [global] 2023-10-08T09:05:58.962 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout fsid = b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout [osd] 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-08T09:05:58.963 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-08T09:05:58.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-08T09:05:58.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-08T09:05:58.964 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-08T09:05:58.964 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling cephadm module... 2023-10-08T09:06:00.333 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1452323332' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-08T09:06:00.333 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: ignoring --setuser ceph since I am not root 2023-10-08T09:06:00.333 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: ignoring --setgroup ceph since I am not root 2023-10-08T09:06:00.333 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:00.236+0000 7fcf6e1e6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:06:00.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:00.332+0000 7fcf6e1e6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:06:00.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:00.442+0000 7fcf6e1e6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:06:00.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:00.559+0000 7fcf6e1e6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:06:01.051 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:00.774+0000 7fcf6e1e6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:06:01.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:01 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1452323332' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-08T09:06:01.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:01 smithi049 ceph-mon[99485]: mgrmap e5: x(active, since 8s) 2023-10-08T09:06:01.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:01.253+0000 7fcf6e1e6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2023-10-08T09:06:01.848 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 5... 2023-10-08T09:06:01.877 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:01.406+0000 7fcf6e1e6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:06:01.877 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:01.516+0000 7fcf6e1e6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:06:01.877 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:01.637+0000 7fcf6e1e6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:06:02.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:02 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4007787288' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-08T09:06:02.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:02.164+0000 7fcf6e1e6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:06:03.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:02.953+0000 7fcf6e1e6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:06:03.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:03 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:03.052+0000 7fcf6e1e6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:06:04.359 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:04.025+0000 7fcf6e1e6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:06:04.359 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:04.137+0000 7fcf6e1e6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:06:04.963 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:04.359+0000 7fcf6e1e6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:06:04.963 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:04.710+0000 7fcf6e1e6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:06:04.963 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:04.962+0000 7fcf6e1e6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:06:05.381 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:05.145+0000 7fcf6e1e6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:06:05.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:05.477+0000 7fcf6e1e6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:06:06.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:06 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:06.301+0000 7fcf6e1e6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:06:06.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:06 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:06.459+0000 7fcf6e1e6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:06:07.119 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:06 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:06.705+0000 7fcf6e1e6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:06:07.119 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:06 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:06.843+0000 7fcf6e1e6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:06:07.120 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:07 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:07.118+0000 7fcf6e1e6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:06:08.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:08.165+0000 7fcf6e1e6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:06:08.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: Active manager daemon x restarted 2023-10-08T09:06:08.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: Activating manager daemon x 2023-10-08T09:06:08.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: osdmap e2: 0 total, 0 up, 0 in 2023-10-08T09:06:08.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: mgrmap e6: x(active, starting, since 0.00315584s) 2023-10-08T09:06:08.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: Manager daemon x is now available 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:08.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:08 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:09.587 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:09 smithi049 ceph-mon[99485]: Found migration_current of "None". Setting to last migration. 2023-10-08T09:06:09.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:09 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:09.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:09 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T09:06:09.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:09 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T09:06:09.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:09 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:09.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:09 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:09.588 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:09 smithi049 ceph-mon[99485]: mgrmap e7: x(active, since 1.00517s) 2023-10-08T09:06:09.606 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:06:09.607 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-08T09:06:09.607 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-08T09:06:09.607 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:06:09.607 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 5 is available 2023-10-08T09:06:09.607 INFO:teuthology.orchestra.run.smithi049.stdout:Setting orchestrator backend to cephadm... 2023-10-08T09:06:10.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:10 smithi049 ceph-mon[99485]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-08T09:06:10.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:10 smithi049 ceph-mon[99485]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-08T09:06:10.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:10 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:09] ENGINE Bus STARTING 2023-10-08T09:06:10.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:10 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:09] ENGINE Serving on https://172.21.15.49:7150 2023-10-08T09:06:10.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:10 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:09] ENGINE Serving on http://172.21.15.49:8765 2023-10-08T09:06:10.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:10 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:09] ENGINE Bus STARTED 2023-10-08T09:06:10.522 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:10 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:11 smithi049 ceph-mon[99485]: mgrmap e8: x(active, since 2s) 2023-10-08T09:06:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:11 smithi049 ceph-mon[99485]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:11 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:11 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-08T09:06:12.192 INFO:teuthology.orchestra.run.smithi049.stdout:Generating ssh key... 2023-10-08T09:06:12.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:12 smithi049 ceph-mon[99485]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:13.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-mon[99485]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:13.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-mon[99485]: Generating ssh key... 2023-10-08T09:06:14.034 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: Generating public/private rsa key pair. 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: Your identification has been saved in /tmp/tmpdvoyq1ug/key. 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: Your public key has been saved in /tmp/tmpdvoyq1ug/key.pub. 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: The key fingerprint is: 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: SHA256:YXmDQfal+IitJ3Y3aF5AeMrocv3IwxOvyTgRMIkOISo ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: The key's randomart image is: 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: +---[RSA 3072]----+ 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: |oo . .+ . | 2023-10-08T09:06:14.035 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: |= + o * o | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: |E o . O = | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: |.. .o B = . | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: | ..+ S . | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: | ..... o | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: | . oo=o= + | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: | o.=+X.o . | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: | ..*+o | 2023-10-08T09:06:14.036 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:13 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: +----[SHA256]-----+ 2023-10-08T09:06:15.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:14 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:15.063 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:14 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:15.475 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD27PCv976eK9FjOiB5XztrcPZCz6M0SUHjlZbfoVAEmb3MFG1xhJHH3zaa0Vb3DZQM/Tzer6JydoASb6zMBtHykPYJFL6LPoTm66vPlESxr5Kn08acWkyDGjdjpa9snsH4ZXUVHRkRBE0++E/5ETZmRIgIowP6RuHyBrFqzpGc5CNdOy/VB/rMvA+ljx0u5rttsftrdwuRE1ZkZcizGSDxT6BO09Jjx9qdQWDHdwbzKrvJLydFZOp4urxDIF5Q9K4v5ZBKxjMx4eHJBBalsJCEpfD/UCkX/swbTcy+AU0NQLB63NDCIl+27wh6XkkeBrSN2ERMyx8IXaBPdXOHCs1wB3GQZzUvyxNwNxTUNgNZwqBsmwnl0aKiSjoZlXGpY4pYphOhibFbB27CelxhFRv5Xjw73KBvSpoHcPD4t4UNd3WXnZuDSgJq0KH1S7zaTVkuxCh3CLrbagDI+8adyJjpFH8H8cjFsU4rIJBJ0V9GtZnepqWuooEI2BSC4MHrG7k= ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:06:15.475 INFO:teuthology.orchestra.run.smithi049.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-08T09:06:15.476 INFO:teuthology.orchestra.run.smithi049.stdout:Adding key to root@localhost authorized_keys... 2023-10-08T09:06:15.477 INFO:teuthology.orchestra.run.smithi049.stdout:Adding host smithi049... 2023-10-08T09:06:15.731 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:15 smithi049 ceph-mon[99485]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:17.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:17 smithi049 ceph-mon[99485]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi049", "addr": "172.21.15.49", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:17.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:17 smithi049 ceph-mon[99485]: Deploying cephadm binary to smithi049 2023-10-08T09:06:19.724 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Added host 'smithi049' with addr '172.21.15.49' 2023-10-08T09:06:19.725 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mon service... 2023-10-08T09:06:20.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:20 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:20.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:20 smithi049 ceph-mon[99485]: Added host smithi049 2023-10-08T09:06:20.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:20 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:21.269 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-08T09:06:21.269 INFO:teuthology.orchestra.run.smithi049.stdout:Deploying unmanaged mgr service... 2023-10-08T09:06:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:21 smithi049 ceph-mon[99485]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:21 smithi049 ceph-mon[99485]: Saving service mon spec with placement count:5 2023-10-08T09:06:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:21 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:21 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:23.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:23 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:23.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:23 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:23.266 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-08T09:06:24.352 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:24 smithi049 ceph-mon[99485]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:24.353 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:24 smithi049 ceph-mon[99485]: Saving service mgr spec with placement count:2 2023-10-08T09:06:24.353 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:24 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:25.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:25 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3926482048' entity='client.admin' 2023-10-08T09:06:25.582 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:25 smithi049 ceph-mon[99485]: from='mgr.14120 172.21.15.49:0/653098461' entity='mgr.x' 2023-10-08T09:06:26.059 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling the dashboard module... 2023-10-08T09:06:26.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:26 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1231357958' entity='client.admin' 2023-10-08T09:06:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:27 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1065601056' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-08T09:06:27.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: ignoring --setuser ceph since I am not root 2023-10-08T09:06:27.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: ignoring --setgroup ceph since I am not root 2023-10-08T09:06:27.883 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:27.797+0000 7fe44646c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:06:28.198 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:27.893+0000 7fe44646c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:06:28.198 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:28.004+0000 7fe44646c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:06:28.199 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:28.125+0000 7fe44646c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:06:28.586 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:28.341+0000 7fe44646c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:06:28.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:28.822+0000 7fe44646c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:06:28.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:28 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1065601056' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-08T09:06:28.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:28 smithi049 ceph-mon[99485]: mgrmap e9: x(active, since 19s) 2023-10-08T09:06:29.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:28.978+0000 7fe44646c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:06:29.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:29.092+0000 7fe44646c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:06:29.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:29.209+0000 7fe44646c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:06:29.441 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:06:29.441 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-08T09:06:29.441 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-08T09:06:29.442 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-08T09:06:29.442 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-08T09:06:29.442 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:06:29.442 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for the mgr to restart... 2023-10-08T09:06:29.442 INFO:teuthology.orchestra.run.smithi049.stdout:Waiting for mgr epoch 9... 2023-10-08T09:06:29.723 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:29 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1532625109' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-08T09:06:30.132 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:29.738+0000 7fe44646c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:06:30.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:30 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:30.527+0000 7fe44646c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:06:30.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:30 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:30.626+0000 7fe44646c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:06:31.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:31.603+0000 7fe44646c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:06:31.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:31.714+0000 7fe44646c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:06:32.295 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:31.937+0000 7fe44646c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:06:32.545 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:32 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:32.291+0000 7fe44646c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:06:32.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:32 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:32.544+0000 7fe44646c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:06:32.882 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:32 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:32.728+0000 7fe44646c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:06:33.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:33 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:33.062+0000 7fe44646c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:06:34.296 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:33 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:33.889+0000 7fe44646c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:06:34.297 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:34.047+0000 7fe44646c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:06:34.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:34.296+0000 7fe44646c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:06:34.632 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:34.434+0000 7fe44646c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:06:35.132 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:34.709+0000 7fe44646c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:06:36.132 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:06:35.757+0000 7fe44646c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:06:36.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: Active manager daemon x restarted 2023-10-08T09:06:36.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: Activating manager daemon x 2023-10-08T09:06:36.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: osdmap e3: 0 total, 0 up, 0 in 2023-10-08T09:06:36.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: mgrmap e10: x(active, starting, since 0.00303968s) 2023-10-08T09:06:36.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:06:36.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-08T09:06:36.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-08T09:06:36.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-08T09:06:36.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-08T09:06:36.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: Manager daemon x is now available 2023-10-08T09:06:36.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:36.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:37.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-08T09:06:37.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-08T09:06:37.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:36 smithi049 ceph-mon[99485]: mgrmap e11: x(active, since 1.006s) 2023-10-08T09:06:37.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout { 2023-10-08T09:06:37.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-08T09:06:37.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-08T09:06:37.207 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout } 2023-10-08T09:06:37.207 INFO:teuthology.orchestra.run.smithi049.stdout:mgr epoch 9 is available 2023-10-08T09:06:37.207 INFO:teuthology.orchestra.run.smithi049.stdout:Generating a dashboard self-signed certificate... 2023-10-08T09:06:38.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:37 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:37] ENGINE Bus STARTING 2023-10-08T09:06:38.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:37 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:37] ENGINE Serving on http://172.21.15.49:8765 2023-10-08T09:06:38.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:37 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:37] ENGINE Serving on https://172.21.15.49:7150 2023-10-08T09:06:38.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:37 smithi049 ceph-mon[99485]: [08/Oct/2023:09:06:37] ENGINE Bus STARTED 2023-10-08T09:06:38.900 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-08T09:06:38.901 INFO:teuthology.orchestra.run.smithi049.stdout:Creating initial admin user... 2023-10-08T09:06:39.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:38 smithi049 ceph-mon[99485]: mgrmap e12: x(active, since 2s) 2023-10-08T09:06:39.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:39.080 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:38 smithi049 ceph-mon[99485]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:39.081 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:39.081 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:39.081 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:40.651 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$yoc.Y19H9019u2SJIDulyOmElW5bv/SeK4khYw1cV63JHYwfdOiAm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1696756000, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-08T09:06:40.651 INFO:teuthology.orchestra.run.smithi049.stdout:Fetching dashboard port number... 2023-10-08T09:06:41.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:41 smithi049 ceph-mon[99485]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:41.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:41.892 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stdout 8443 2023-10-08T09:06:41.902 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-08T09:06:41.902 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout disabled 2023-10-08T09:06:41.909 INFO:teuthology.orchestra.run.smithi049.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-08T09:06:41.909 INFO:teuthology.orchestra.run.smithi049.stdout:systemctl: stdout inactive 2023-10-08T09:06:41.909 INFO:teuthology.orchestra.run.smithi049.stdout:firewalld.service is not enabled 2023-10-08T09:06:41.909 INFO:teuthology.orchestra.run.smithi049.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-08T09:06:41.911 INFO:teuthology.orchestra.run.smithi049.stdout:Ceph Dashboard is now available at: 2023-10-08T09:06:41.911 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:41.911 INFO:teuthology.orchestra.run.smithi049.stdout: URL: https://smithi049:8443/ 2023-10-08T09:06:41.912 INFO:teuthology.orchestra.run.smithi049.stdout: User: admin 2023-10-08T09:06:41.912 INFO:teuthology.orchestra.run.smithi049.stdout: Password: i2wes5u4g8 2023-10-08T09:06:41.912 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:41.912 INFO:teuthology.orchestra.run.smithi049.stdout:Saving cluster configuration to /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config directory 2023-10-08T09:06:41.912 INFO:teuthology.orchestra.run.smithi049.stdout:Enabling autotune for osd_memory_target 2023-10-08T09:06:42.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:42 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1266820108' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-08T09:06:43.513 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:43 smithi049 ceph-mon[99485]: mgrmap e13: x(active, since 6s) 2023-10-08T09:06:44.799 INFO:teuthology.orchestra.run.smithi049.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-08T09:06:44.799 INFO:teuthology.orchestra.run.smithi049.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-08T09:06:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout:Or, if you are only running a single cluster on this host: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: ceph telemetry on 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.800 INFO:teuthology.orchestra.run.smithi049.stdout:For more information see: 2023-10-08T09:06:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-08T09:06:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:44.801 INFO:teuthology.orchestra.run.smithi049.stdout:Bootstrap complete. 2023-10-08T09:06:44.840 INFO:tasks.cephadm:Fetching config... 2023-10-08T09:06:44.840 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:06:44.840 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-08T09:06:44.857 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-08T09:06:44.857 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:06:44.857 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-08T09:06:44.916 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-08T09:06:44.916 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:06:44.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-08T09:06:44.984 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-08T09:06:44.984 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:06:44.984 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-08T09:06:45.041 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-08T09:06:45.042 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD27PCv976eK9FjOiB5XztrcPZCz6M0SUHjlZbfoVAEmb3MFG1xhJHH3zaa0Vb3DZQM/Tzer6JydoASb6zMBtHykPYJFL6LPoTm66vPlESxr5Kn08acWkyDGjdjpa9snsH4ZXUVHRkRBE0++E/5ETZmRIgIowP6RuHyBrFqzpGc5CNdOy/VB/rMvA+ljx0u5rttsftrdwuRE1ZkZcizGSDxT6BO09Jjx9qdQWDHdwbzKrvJLydFZOp4urxDIF5Q9K4v5ZBKxjMx4eHJBBalsJCEpfD/UCkX/swbTcy+AU0NQLB63NDCIl+27wh6XkkeBrSN2ERMyx8IXaBPdXOHCs1wB3GQZzUvyxNwNxTUNgNZwqBsmwnl0aKiSjoZlXGpY4pYphOhibFbB27CelxhFRv5Xjw73KBvSpoHcPD4t4UNd3WXnZuDSgJq0KH1S7zaTVkuxCh3CLrbagDI+8adyJjpFH8H8cjFsU4rIJBJ0V9GtZnepqWuooEI2BSC4MHrG7k= ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T09:06:45.120 INFO:teuthology.orchestra.run.smithi049.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD27PCv976eK9FjOiB5XztrcPZCz6M0SUHjlZbfoVAEmb3MFG1xhJHH3zaa0Vb3DZQM/Tzer6JydoASb6zMBtHykPYJFL6LPoTm66vPlESxr5Kn08acWkyDGjdjpa9snsH4ZXUVHRkRBE0++E/5ETZmRIgIowP6RuHyBrFqzpGc5CNdOy/VB/rMvA+ljx0u5rttsftrdwuRE1ZkZcizGSDxT6BO09Jjx9qdQWDHdwbzKrvJLydFZOp4urxDIF5Q9K4v5ZBKxjMx4eHJBBalsJCEpfD/UCkX/swbTcy+AU0NQLB63NDCIl+27wh6XkkeBrSN2ERMyx8IXaBPdXOHCs1wB3GQZzUvyxNwNxTUNgNZwqBsmwnl0aKiSjoZlXGpY4pYphOhibFbB27CelxhFRv5Xjw73KBvSpoHcPD4t4UNd3WXnZuDSgJq0KH1S7zaTVkuxCh3CLrbagDI+8adyJjpFH8H8cjFsU4rIJBJ0V9GtZnepqWuooEI2BSC4MHrG7k= ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:06:45.134 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD27PCv976eK9FjOiB5XztrcPZCz6M0SUHjlZbfoVAEmb3MFG1xhJHH3zaa0Vb3DZQM/Tzer6JydoASb6zMBtHykPYJFL6LPoTm66vPlESxr5Kn08acWkyDGjdjpa9snsH4ZXUVHRkRBE0++E/5ETZmRIgIowP6RuHyBrFqzpGc5CNdOy/VB/rMvA+ljx0u5rttsftrdwuRE1ZkZcizGSDxT6BO09Jjx9qdQWDHdwbzKrvJLydFZOp4urxDIF5Q9K4v5ZBKxjMx4eHJBBalsJCEpfD/UCkX/swbTcy+AU0NQLB63NDCIl+27wh6XkkeBrSN2ERMyx8IXaBPdXOHCs1wB3GQZzUvyxNwNxTUNgNZwqBsmwnl0aKiSjoZlXGpY4pYphOhibFbB27CelxhFRv5Xjw73KBvSpoHcPD4t4UNd3WXnZuDSgJq0KH1S7zaTVkuxCh3CLrbagDI+8adyJjpFH8H8cjFsU4rIJBJ0V9GtZnepqWuooEI2BSC4MHrG7k= ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T09:06:45.178 INFO:teuthology.orchestra.run.smithi132.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD27PCv976eK9FjOiB5XztrcPZCz6M0SUHjlZbfoVAEmb3MFG1xhJHH3zaa0Vb3DZQM/Tzer6JydoASb6zMBtHykPYJFL6LPoTm66vPlESxr5Kn08acWkyDGjdjpa9snsH4ZXUVHRkRBE0++E/5ETZmRIgIowP6RuHyBrFqzpGc5CNdOy/VB/rMvA+ljx0u5rttsftrdwuRE1ZkZcizGSDxT6BO09Jjx9qdQWDHdwbzKrvJLydFZOp4urxDIF5Q9K4v5ZBKxjMx4eHJBBalsJCEpfD/UCkX/swbTcy+AU0NQLB63NDCIl+27wh6XkkeBrSN2ERMyx8IXaBPdXOHCs1wB3GQZzUvyxNwNxTUNgNZwqBsmwnl0aKiSjoZlXGpY4pYphOhibFbB27CelxhFRv5Xjw73KBvSpoHcPD4t4UNd3WXnZuDSgJq0KH1S7zaTVkuxCh3CLrbagDI+8adyJjpFH8H8cjFsU4rIJBJ0V9GtZnepqWuooEI2BSC4MHrG7k= ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:06:45.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD27PCv976eK9FjOiB5XztrcPZCz6M0SUHjlZbfoVAEmb3MFG1xhJHH3zaa0Vb3DZQM/Tzer6JydoASb6zMBtHykPYJFL6LPoTm66vPlESxr5Kn08acWkyDGjdjpa9snsH4ZXUVHRkRBE0++E/5ETZmRIgIowP6RuHyBrFqzpGc5CNdOy/VB/rMvA+ljx0u5rttsftrdwuRE1ZkZcizGSDxT6BO09Jjx9qdQWDHdwbzKrvJLydFZOp4urxDIF5Q9K4v5ZBKxjMx4eHJBBalsJCEpfD/UCkX/swbTcy+AU0NQLB63NDCIl+27wh6XkkeBrSN2ERMyx8IXaBPdXOHCs1wB3GQZzUvyxNwNxTUNgNZwqBsmwnl0aKiSjoZlXGpY4pYphOhibFbB27CelxhFRv5Xjw73KBvSpoHcPD4t4UNd3WXnZuDSgJq0KH1S7zaTVkuxCh3CLrbagDI+8adyJjpFH8H8cjFsU4rIJBJ0V9GtZnepqWuooEI2BSC4MHrG7k= ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-08T09:06:45.239 INFO:teuthology.orchestra.run.smithi184.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD27PCv976eK9FjOiB5XztrcPZCz6M0SUHjlZbfoVAEmb3MFG1xhJHH3zaa0Vb3DZQM/Tzer6JydoASb6zMBtHykPYJFL6LPoTm66vPlESxr5Kn08acWkyDGjdjpa9snsH4ZXUVHRkRBE0++E/5ETZmRIgIowP6RuHyBrFqzpGc5CNdOy/VB/rMvA+ljx0u5rttsftrdwuRE1ZkZcizGSDxT6BO09Jjx9qdQWDHdwbzKrvJLydFZOp4urxDIF5Q9K4v5ZBKxjMx4eHJBBalsJCEpfD/UCkX/swbTcy+AU0NQLB63NDCIl+27wh6XkkeBrSN2ERMyx8IXaBPdXOHCs1wB3GQZzUvyxNwNxTUNgNZwqBsmwnl0aKiSjoZlXGpY4pYphOhibFbB27CelxhFRv5Xjw73KBvSpoHcPD4t4UNd3WXnZuDSgJq0KH1S7zaTVkuxCh3CLrbagDI+8adyJjpFH8H8cjFsU4rIJBJ0V9GtZnepqWuooEI2BSC4MHrG7k= ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:06:45.253 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-08T09:06:45.833 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:06:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:45 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3973144102' entity='client.admin' 2023-10-08T09:06:48.083 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-08T09:06:48.083 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-08T09:06:48.767 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:06:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:48 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1416261057' entity='client.admin' 2023-10-08T09:06:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi049", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:06:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:50.728 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi132 2023-10-08T09:06:50.729 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:06:50.729 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.conf 2023-10-08T09:06:50.750 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:06:50.750 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:06:50.810 INFO:tasks.cephadm:Adding host smithi132 to orchestrator... 2023-10-08T09:06:50.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi132 2023-10-08T09:06:51.385 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:06:51.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:51 smithi049 ceph-mon[99485]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:51.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:51.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:51.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:06:51.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:06:51.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:51 smithi049 ceph-mon[99485]: Updating smithi049:/etc/ceph/ceph.conf 2023-10-08T09:06:51.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:51 smithi049 ceph-mon[99485]: Updating smithi049:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:06:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:52 smithi049 ceph-mon[99485]: Updating smithi049:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:06:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:52 smithi049 ceph-mon[99485]: Updating smithi049:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:06:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:52.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:53 smithi049 ceph-mon[99485]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:06:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:53 smithi049 ceph-mon[99485]: Deploying cephadm binary to smithi132 2023-10-08T09:06:55.815 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi132' with addr '172.21.15.132' 2023-10-08T09:06:56.241 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-08T09:06:56.811 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:06:57.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:56 smithi049 ceph-mon[99485]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:06:57.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:56 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:57.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:56 smithi049 ceph-mon[99485]: Added host smithi132 2023-10-08T09:06:57.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:56 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:06:57.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:56 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:58.364 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:06:58.364 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}] 2023-10-08T09:06:58.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:58.758 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi184 2023-10-08T09:06:58.758 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:06:58.758 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.conf 2023-10-08T09:06:58.779 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:06:58.779 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:06:58.839 INFO:tasks.cephadm:Adding host smithi184 to orchestrator... 2023-10-08T09:06:58.839 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch host add smithi184 2023-10-08T09:06:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:06:59.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:59 smithi049 ceph-mon[99485]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:06:59.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:06:59.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:06:59 smithi049 ceph-mon[99485]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:07:01.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:01.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:01.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:01.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:01.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:01.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:07:01.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:01.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:02.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:02 smithi049 ceph-mon[99485]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:07:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:02 smithi049 ceph-mon[99485]: Updating smithi132:/etc/ceph/ceph.conf 2023-10-08T09:07:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:02 smithi049 ceph-mon[99485]: Updating smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:02 smithi049 ceph-mon[99485]: Deploying cephadm binary to smithi184 2023-10-08T09:07:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:03 smithi049 ceph-mon[99485]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:07:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:03 smithi049 ceph-mon[99485]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:03 smithi049 ceph-mon[99485]: Updating smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:07:03.904 INFO:teuthology.orchestra.run.smithi049.stdout:Added host 'smithi184' with addr '172.21.15.184' 2023-10-08T09:07:04.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-08T09:07:04.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:04 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:04.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:04 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:04.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:04 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:04.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:04 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:04.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:04 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:04.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:04 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:04.938 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:05.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:05 smithi049 ceph-mon[99485]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:05.754 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:05 smithi049 ceph-mon[99485]: Added host smithi184 2023-10-08T09:07:05.755 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:07:06.468 INFO:teuthology.orchestra.run.smithi049.stdout:[{"addr": "172.21.15.49", "hostname": "smithi049", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}, {"addr": "172.21.15.184", "hostname": "smithi184", "labels": [], "status": ""}] 2023-10-08T09:07:06.902 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-08T09:07:06.902 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd crush tunables default 2023-10-08T09:07:07.465 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:07.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:07 smithi049 ceph-mon[99485]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:07.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:07 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:07.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:07 smithi049 ceph-mon[99485]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:07:09.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:09.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:09 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/863269128' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-08T09:07:10.117 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2023-10-08T09:07:10.519 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: Updating smithi184:/etc/ceph/ceph.conf 2023-10-08T09:07:10.519 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: Updating smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:10.519 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:07:10.519 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:10.519 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:10.519 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:10.520 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/863269128' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-08T09:07:10.520 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:10 smithi049 ceph-mon[99485]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:10.526 INFO:tasks.cephadm:Adding mon.a on smithi049 2023-10-08T09:07:10.526 INFO:tasks.cephadm:Adding mon.b on smithi132 2023-10-08T09:07:10.526 INFO:tasks.cephadm:Adding mon.c on smithi184 2023-10-08T09:07:10.526 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch apply mon '3;smithi049:172.21.15.49=a;smithi132:172.21.15.132=b;smithi184:172.21.15.184=c' 2023-10-08T09:07:11.102 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:11 smithi049 ceph-mon[99485]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:11 smithi049 ceph-mon[99485]: Updating smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:07:12.570 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mon update... 2023-10-08T09:07:12.970 DEBUG:teuthology.orchestra.run.smithi132:mon.b> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.b.service 2023-10-08T09:07:12.973 DEBUG:teuthology.orchestra.run.smithi184:mon.c> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.c.service 2023-10-08T09:07:12.976 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T09:07:12.977 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T09:07:13.006 INFO:journalctl@ceph.mon.c.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:07:13.008 INFO:journalctl@ceph.mon.b.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi132:172.21.15.132=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi132:172.21.15.132=b;smithi184:172.21.15.184=c;count:3 2023-10-08T09:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:13 smithi049 ceph-mon[99485]: Deploying daemon mon.c on smithi184 2023-10-08T09:07:13.898 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:15 smithi049 ceph-mon[99485]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:15.924 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:07:15.925 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":1,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","modified":"2023-10-08T09:05:32.295520Z","created":"2023-10-08T09:05:32.295520Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-08T09:07:15.926 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 1 2023-10-08T09:07:16.580 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:16 smithi184 ceph-mon[103675]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-08T09:07:16.893 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:16 smithi184 systemd[1]: Started Ceph mon.c for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:07:17.895 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T09:07:17.896 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T09:07:18.473 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:07:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: mon.a calling monitor election 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: mon.c calling monitor election 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: monmap e2: 2 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: fsmap 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: mgrmap e13: x(active, since 45s) 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: overall HEALTH_OK 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:21.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:07:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: mon.a calling monitor election 2023-10-08T09:07:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: mon.c calling monitor election 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: monmap e2: 2 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: fsmap 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: mgrmap e13: x(active, since 45s) 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: overall HEALTH_OK 2023-10-08T09:07:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:22.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:22.942 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:07:22.942 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":2,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","modified":"2023-10-08T09:07:16.547129Z","created":"2023-10-08T09:05:32.295520Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-08T09:07:22.945 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 2 2023-10-08T09:07:22.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:22.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:22.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:22.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:22 smithi184 ceph-mon[103675]: Deploying daemon mon.b on smithi132 2023-10-08T09:07:22.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:23.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:23.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:23.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:23.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:22 smithi049 ceph-mon[99485]: Deploying daemon mon.b on smithi132 2023-10-08T09:07:23.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:23 smithi184 ceph-mon[103675]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:23 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/1300254597' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:07:24.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:23 smithi049 ceph-mon[99485]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:23 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/1300254597' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:07:24.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 systemd[1]: Starting Ceph mon.b for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:07:24.333 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-08T09:07:24.334 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph mon dump -f json 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 podman[103349]: 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 bash[103349]: dc5533f275d6d7ee4008d8cfe50aa85df417d6911bde9567c21e306e9113236c 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: set uid:gid to 167:167 (ceph:ceph) 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev), process ceph-mon, pid 7 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pidfile_write: ignore empty --pid-file 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: load: jerasure load: lrc 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: RocksDB version: 7.9.2 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Git sha 0 2023-10-08T09:07:24.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Compile date 2023-10-07 16:15:44 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: DB SUMMARY 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: DB Session ID: NNON4VERJ73XCACV3MJH 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: CURRENT file: CURRENT 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: IDENTITY file: IDENTITY 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.error_if_exists: 0 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.create_if_missing: 0 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.paranoid_checks: 1 2023-10-08T09:07:24.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.env: 0x55ce0e9eeda0 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.fs: PosixFileSystem 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.info_log: 0x55ce108af640 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_file_opening_threads: 16 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.statistics: (nil) 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.use_fsync: 0 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_log_file_size: 0 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-08T09:07:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.keep_log_file_num: 1000 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.recycle_log_file_num: 0 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.allow_fallocate: 1 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.allow_mmap_reads: 0 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.allow_mmap_writes: 0 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.use_direct_reads: 0 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.create_missing_column_families: 0 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.db_log_dir: 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.wal_dir: 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-08T09:07:24.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.advise_random_on_open: 1 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.db_write_buffer_size: 0 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.write_buffer_manager: 0x55ce10867360 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.rate_limiter: (nil) 2023-10-08T09:07:24.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.wal_recovery_mode: 2 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.enable_thread_tracking: 0 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.enable_pipelined_write: 0 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.unordered_write: 0 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.row_cache: None 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.wal_filter: None 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.allow_ingest_behind: 0 2023-10-08T09:07:24.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.two_write_queues: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.manual_wal_flush: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.wal_compression: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.atomic_flush: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.log_readahead_size: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.best_efforts_recovery: 0 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-08T09:07:24.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.allow_data_in_errors: 0 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.db_host_id: __hostname__ 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_background_jobs: 2 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_background_compactions: -1 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_subcompactions: 1 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_total_wal_size: 0 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-08T09:07:24.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_open_files: -1 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bytes_per_sync: 0 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_readahead_size: 0 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_background_flushes: -1 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Compression algorithms supported: 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kZSTD supported: 0 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kXpressCompression supported: 0 2023-10-08T09:07:24.806 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kLZ4HCCompression supported: 1 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kZlibCompression supported: 1 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kSnappyCompression supported: 1 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kLZ4Compression supported: 1 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: kBZip2Compression supported: 0 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.merge_operator: 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_filter: None 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_filter_factory: None 2023-10-08T09:07:24.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.sst_partitioner_factory: None 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ce108af740) 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: cache_index_and_filter_blocks: 1 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: pin_top_level_index_and_filter: 1 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: index_type: 0 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: data_block_index_type: 0 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: index_shortening: 1 2023-10-08T09:07:24.808 INFO:journalctl@ceph.mon.b.smithi132.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: checksum: 4 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: no_block_cache: 0 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache: 0x55ce109051f0 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache_name: BinnedLRUCache 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache_options: 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: capacity : 536870912 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: num_shard_bits : 4 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: strict_capacity_limit : 0 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: high_pri_pool_ratio: 0.000 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_cache_compressed: (nil) 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: persistent_cache: (nil) 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_size: 4096 2023-10-08T09:07:24.809 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_size_deviation: 10 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_restart_interval: 16 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: index_block_restart_interval: 1 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: metadata_block_size: 4096 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: partition_filters: 0 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: use_delta_encoding: 1 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: filter_policy: bloomfilter 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: whole_key_filtering: 1 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: verify_compression: 0 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: read_amp_bytes_per_bit: 0 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: format_version: 5 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: enable_index_compression: 1 2023-10-08T09:07:24.810 INFO:journalctl@ceph.mon.b.smithi132.stdout: block_align: 0 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout: max_auto_readahead_size: 262144 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout: prepopulate_block_cache: 0 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout: initial_auto_readahead_size: 8192 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout: num_file_reads_for_auto_readahead: 2 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.write_buffer_size: 33554432 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_write_buffer_number: 2 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression: NoCompression 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression: Disabled 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.prefix_extractor: nullptr 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.num_levels: 7 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-08T09:07:24.811 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-08T09:07:24.812 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-08T09:07:24.812 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-08T09:07:24.812 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-08T09:07:24.812 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-08T09:07:24.813 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-08T09:07:24.813 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:07:24.813 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-08T09:07:24.813 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-08T09:07:24.813 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.level: 32767 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.strategy: 0 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.enabled: false 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-08T09:07:24.814 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.target_file_size_base: 67108864 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-08T09:07:24.815 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.arena_block_size: 1048576 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.disable_auto_compactions: 0 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-08T09:07:24.816 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.table_properties_collectors: 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.inplace_update_support: 0 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-08T09:07:24.817 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.bloom_locality: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.max_successive_merges: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.paranoid_file_checks: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.force_consistency_checks: 1 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.report_bg_io_stats: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.ttl: 2592000 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-08T09:07:24.818 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.enable_blob_files: false 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.min_blob_size: 0 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.blob_file_size: 268435456 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.blob_file_starting_level: 0 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-08T09:07:24.819 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4a151d4c-4d33-4f72-b05b-cb99d86caf80 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696756044670064, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696756044670639, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1696756044, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4a151d4c-4d33-4f72-b05b-cb99d86caf80", "db_session_id": "NNON4VERJ73XCACV3MJH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: EVENT_LOG_v1 {"time_micros": 1696756044670729, "job": 1, "event": "recovery_finished"} 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ce109a2000 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: DB pointer 0x55ce1098e000 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** DB Stats ** 2023-10-08T09:07:24.820 INFO:journalctl@ceph.mon.b.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** Compaction Stats [default] ** 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:07:24.821 INFO:journalctl@ceph.mon.b.smithi132.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** Compaction Stats [default] ** 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-08T09:07:24.822 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: Block cache BinnedLRUCache@0x55ce109051f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-08T09:07:24.823 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: using public_addr v2:172.21.15.132:0/0 -> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: starting mon.b rank -1 at public addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] at bind addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(???) e0 preinit fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 systemd[1]: Started Ceph mon.b for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).mds e1 new map 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout: e1 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout: legacy client fscid: -1 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout: 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout: No filesystems configured 2023-10-08T09:07:24.824 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.825 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi184", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi132:/etc/ceph/ceph.conf 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Deploying cephadm binary to smithi184 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:07:24.826 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Added host smithi184 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:07:24.827 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi184", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/863269128' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi184:/etc/ceph/ceph.conf 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi184:/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/863269128' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Updating smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi049:172.21.15.49=a;smithi132:172.21.15.132=b;smithi184:172.21.15.184=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Saving service mon spec with placement smithi049:172.21.15.49=a;smithi132:172.21.15.132=b;smithi184:172.21.15.184=c;count:3 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:24.829 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Deploying daemon mon.c on smithi184 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.a calling monitor election 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.830 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.c calling monitor election 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: monmap e2: 2 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: fsmap 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:24.831 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mgrmap e13: x(active, since 45s) 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: overall HEALTH_OK 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: Deploying daemon mon.b on smithi132 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/1300254597' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:07:24.832 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:24 smithi132 ceph-mon[103389]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-08T09:07:24.902 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:07:30.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:07:30.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: mon.c calling monitor election 2023-10-08T09:07:30.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: mon.a calling monitor election 2023-10-08T09:07:30.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:30.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:30.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:30.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:07:30.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:30.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: fsmap 2023-10-08T09:07:30.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:30.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: mgrmap e13: x(active, since 53s) 2023-10-08T09:07:30.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-08T09:07:30.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-08T09:07:30.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-08T09:07:30.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2023-10-08T09:07:30.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:30.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:07:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: mon.c calling monitor election 2023-10-08T09:07:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: mon.a calling monitor election 2023-10-08T09:07:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: fsmap 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:30.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: mgrmap e13: x(active, since 53s) 2023-10-08T09:07:30.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-08T09:07:30.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-08T09:07:30.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-08T09:07:30.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2023-10-08T09:07:30.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:30.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:30.746 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T09:07:30.746 INFO:teuthology.orchestra.run.smithi184.stdout:{"epoch":3,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","modified":"2023-10-08T09:07:24.701230Z","created":"2023-10-08T09:05:32.295520Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:3300","nonce":0},{"type":"v1","addr":"172.21.15.49:6789","nonce":0}]},"addr":"172.21.15.49:6789/0","public_addr":"172.21.15.49:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:3300","nonce":0},{"type":"v1","addr":"172.21.15.184:6789","nonce":0}]},"addr":"172.21.15.184:6789/0","public_addr":"172.21.15.184:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-08T09:07:30.749 INFO:teuthology.orchestra.run.smithi184.stderr:dumped monmap epoch 3 2023-10-08T09:07:31.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:30 smithi049 ceph-mon[99485]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:31.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:31.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:31.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:31.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:31.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:30 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/1734838442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:07:31.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:30 smithi184 ceph-mon[103675]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:31.136 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:31.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:31.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:31.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:31.137 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:30 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/1734838442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:07:31.159 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-08T09:07:31.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-08T09:07:31.744 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:07:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: mon.c calling monitor election 2023-10-08T09:07:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: mon.a calling monitor election 2023-10-08T09:07:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-08T09:07:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: fsmap 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: mgrmap e13: x(active, since 53s) 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: mon.b (rank 2) addr [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] is down (out of quorum) 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:32.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:32.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:32.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/1734838442' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-08T09:07:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: mon.b calling monitor election 2023-10-08T09:07:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: mon.b calling monitor election 2023-10-08T09:07:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: mon.c calling monitor election 2023-10-08T09:07:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: mon.a calling monitor election 2023-10-08T09:07:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T09:07:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: fsmap 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: mgrmap e13: x(active, since 55s) 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: Cluster is now healthy 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: overall HEALTH_OK 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:32.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: mon.b calling monitor election 2023-10-08T09:07:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: mon.b calling monitor election 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: mon.c calling monitor election 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: mon.a calling monitor election 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: fsmap 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: mgrmap e13: x(active, since 55s) 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: Cluster is now healthy 2023-10-08T09:07:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: overall HEALTH_OK 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:32.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:33.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:33.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:33.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: mon.b calling monitor election 2023-10-08T09:07:33.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: mon.b calling monitor election 2023-10-08T09:07:33.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: mon.c calling monitor election 2023-10-08T09:07:33.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: mon.a calling monitor election 2023-10-08T09:07:33.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-08T09:07:33.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: monmap e3: 3 mons at {a=[v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0],b=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0],c=[v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0]} removed_ranks: {} 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: fsmap 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: osdmap e4: 0 total, 0 up, 0 in 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: mgrmap e13: x(active, since 55s) 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: Cluster is now healthy 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: overall HEALTH_OK 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:33.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:33.344 INFO:teuthology.orchestra.run.smithi049.stdout:# minimal ceph.conf for b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:07:33.345 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2023-10-08T09:07:33.345 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:07:33.345 INFO:teuthology.orchestra.run.smithi049.stdout: mon_host = [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] 2023-10-08T09:07:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: Updating smithi049:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: Updating smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: Updating smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T09:07:33.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: Reconfiguring daemon mon.a on smithi049 2023-10-08T09:07:33.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:33.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:33 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4142449213' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: Updating smithi049:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: Updating smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: Updating smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T09:07:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: Reconfiguring daemon mon.a on smithi049 2023-10-08T09:07:33.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:33.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:33 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4142449213' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: Updating smithi049:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: Updating smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: Updating smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/config/ceph.conf 2023-10-08T09:07:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: Reconfiguring mon.a (unknown last config time)... 2023-10-08T09:07:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: Reconfiguring daemon mon.a on smithi049 2023-10-08T09:07:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:07:34.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:33 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4142449213' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:34.115 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-08T09:07:34.115 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:07:34.115 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T09:07:34.145 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:07:34.145 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:07:34.212 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:07:34.212 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T09:07:34.243 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:07:34.243 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:07:34.309 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:07:34.309 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.conf 2023-10-08T09:07:34.347 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:07:34.347 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-08T09:07:34.418 INFO:tasks.cephadm:Adding mgr.x on smithi049 2023-10-08T09:07:34.419 INFO:tasks.cephadm:Adding mgr.y on smithi132 2023-10-08T09:07:34.419 INFO:tasks.cephadm:Adding mgr.z on smithi184 2023-10-08T09:07:34.419 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch apply mgr '3;smithi049=x;smithi132=y;smithi184=z' 2023-10-08T09:07:35.016 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:07:35.230 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:35.231 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:35.231 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:35.231 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: Reconfiguring mon.b (monmap changed)... 2023-10-08T09:07:35.231 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:35.231 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:35.231 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:35.231 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:35 smithi132 ceph-mon[103389]: Reconfiguring daemon mon.b on smithi132 2023-10-08T09:07:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: Reconfiguring mon.b (monmap changed)... 2023-10-08T09:07:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:35.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:35 smithi049 ceph-mon[99485]: Reconfiguring daemon mon.b on smithi132 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: Reconfiguring mon.b (monmap changed)... 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:35.411 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:35 smithi184 ceph-mon[103675]: Reconfiguring daemon mon.b on smithi132 2023-10-08T09:07:36.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:36.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:36.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:36 smithi184 ceph-mon[103675]: Reconfiguring mon.c (monmap changed)... 2023-10-08T09:07:36.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:36.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:36.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:36.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:36 smithi184 ceph-mon[103675]: Reconfiguring daemon mon.c on smithi184 2023-10-08T09:07:36.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:36.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:36.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:36 smithi132 ceph-mon[103389]: Reconfiguring mon.c (monmap changed)... 2023-10-08T09:07:36.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:36.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:36.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:36.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:36 smithi132 ceph-mon[103389]: Reconfiguring daemon mon.c on smithi184 2023-10-08T09:07:36.567 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mgr update... 2023-10-08T09:07:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:36 smithi049 ceph-mon[99485]: Reconfiguring mon.c (monmap changed)... 2023-10-08T09:07:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-08T09:07:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-08T09:07:36.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:36.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:36 smithi049 ceph-mon[99485]: Reconfiguring daemon mon.c on smithi184 2023-10-08T09:07:36.996 DEBUG:teuthology.orchestra.run.smithi132:mgr.y> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.y.service 2023-10-08T09:07:36.999 DEBUG:teuthology.orchestra.run.smithi184:mgr.z> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.z.service 2023-10-08T09:07:37.001 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2023-10-08T09:07:37.001 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set client client_mount_timeout 600 2023-10-08T09:07:37.004 INFO:tasks.cephadm:setting config [client] debug_client = 20 2023-10-08T09:07:37.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set client debug_client 20 2023-10-08T09:07:37.030 INFO:journalctl@ceph.mgr.z.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:07:37.033 INFO:journalctl@ceph.mgr.y.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:07:37.047 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2023-10-08T09:07:37.047 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set client debug_ms 1 2023-10-08T09:07:37.090 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2023-10-08T09:07:37.090 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set client rados_mon_op_timeout 900 2023-10-08T09:07:37.133 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2023-10-08T09:07:37.133 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set client rados_osd_op_timeout 900 2023-10-08T09:07:37.177 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2023-10-08T09:07:37.177 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_mds 20 2023-10-08T09:07:37.221 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2023-10-08T09:07:37.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_mds_balancer 20 2023-10-08T09:07:37.265 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2023-10-08T09:07:37.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds debug_ms 1 2023-10-08T09:07:37.309 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-08T09:07:37.309 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_fragment_size_max 10000 2023-10-08T09:07:37.352 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2023-10-08T09:07:37.352 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_merge_size 5 2023-10-08T09:07:37.396 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2023-10-08T09:07:37.396 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_split_bits 3 2023-10-08T09:07:37.439 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2023-10-08T09:07:37.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_bal_split_size 100 2023-10-08T09:07:37.484 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2023-10-08T09:07:37.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_debug_frag True 2023-10-08T09:07:37.536 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2023-10-08T09:07:37.536 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_debug_scatterstat True 2023-10-08T09:07:37.574 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi132=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: Saving service mgr spec with placement smithi049=x;smithi132=y;smithi184=z;count:3 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.575 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:37.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:07:37.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:37.576 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:37.584 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2023-10-08T09:07:37.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_op_complaint_time 180 2023-10-08T09:07:37.637 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2023-10-08T09:07:37.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds mds_verify_scatter True 2023-10-08T09:07:37.648 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:37.648 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi132=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: Saving service mgr spec with placement smithi049=x;smithi132=y;smithi184=z;count:3 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:37.649 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:37.689 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2023-10-08T09:07:37.689 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds osd_op_complaint_time 180 2023-10-08T09:07:37.721 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:37.742 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2023-10-08T09:07:37.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds rados_mon_op_timeout 900 2023-10-08T09:07:37.743 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:37.796 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2023-10-08T09:07:37.796 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mds rados_osd_op_timeout 900 2023-10-08T09:07:37.874 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2023-10-08T09:07:37.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set mon mon_op_complaint_time 120 2023-10-08T09:07:37.875 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:37.883 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:37.889 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2023-10-08T09:07:37.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config set osd osd_op_complaint_time 180 2023-10-08T09:07:37.937 DEBUG:tasks.cephadm:set 21 configs 2023-10-08T09:07:37.937 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi049') 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi049=x;smithi132=y;smithi184=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: Saving service mgr spec with placement smithi049=x;smithi132=y;smithi184=z;count:3 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:38.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:38.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:38.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:38.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:07:38.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:38.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:38.247 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 systemd[1]: Starting Ceph mgr.z for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:07:38.296 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:38.670 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:38 smithi049 ceph-mon[99485]: Deploying daemon mgr.z on smithi184 2023-10-08T09:07:38.723 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:38.743 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:38 smithi184 ceph-mon[103675]: Deploying daemon mgr.z on smithi184 2023-10-08T09:07:38.743 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 podman[105506]: 2023-10-08T09:07:38.743 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 bash[105506]: fa9061cb56a3fa8d7b86e4a49dc405098cc36a09447f63de1150032c3422c2e2 2023-10-08T09:07:38.743 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 ceph-mgr[105545]: -- 172.21.15.184:0/1615267187 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dc84cd31e0 con 0x55dc84ca9000 2023-10-08T09:07:38.997 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 systemd[1]: Started Ceph mgr.z for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:07:38.997 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:38.820+0000 7fbea7411200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:07:38.997 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:38.899+0000 7fbea7411200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:07:38.997 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:38.988+0000 7fbea7411200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:07:39.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:38 smithi132 ceph-mon[103389]: Deploying daemon mgr.z on smithi184 2023-10-08T09:07:39.248 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:39.250 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:39.265 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:39.497 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:39.081+0000 7fbea7411200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:07:39.497 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:39.248+0000 7fbea7411200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:07:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:07:39.704 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:39.705 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:39.820 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:39.829 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:39.610+0000 7fbea7411200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:07:39.829 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:39.739+0000 7fbea7411200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:07:39.829 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:39.829 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.829 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.830 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:39.830 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:39.830 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-08T09:07:39.830 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:39.830 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:40.247 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:39.828+0000 7fbea7411200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:07:40.247 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:39.923+0000 7fbea7411200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:07:40.547 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:40 smithi132 systemd[1]: Starting Ceph mgr.y for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:07:40.631 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:40.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:40 smithi184 ceph-mon[103675]: Deploying daemon mgr.y on smithi132 2023-10-08T09:07:40.747 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:40 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:40.342+0000 7fbea7411200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:07:40.868 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:40 smithi132 ceph-mon[103389]: Deploying daemon mgr.y on smithi132 2023-10-08T09:07:40.868 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:40 smithi132 podman[104556]: 2023-10-08T09:07:40.868 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:40 smithi132 bash[104556]: 9341b0223e017c59208914fff267f7c7b49e6d12bd916f4084e67ee9b55e999d 2023-10-08T09:07:40.868 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:40 smithi132 ceph-mgr[104595]: -- 172.21.15.132:0/2126280132 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557858aa91e0 con 0x557858a7ec00 2023-10-08T09:07:40.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:40 smithi049 ceph-mon[99485]: Deploying daemon mgr.y on smithi132 2023-10-08T09:07:40.964 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:41.044 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:41.216 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:40 smithi132 systemd[1]: Started Ceph mgr.y for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:07:41.216 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:40 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:40.944+0000 7f5922e4a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T09:07:41.216 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:41.025+0000 7f5922e4a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T09:07:41.216 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:41.116+0000 7f5922e4a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T09:07:41.216 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:41.214+0000 7f5922e4a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T09:07:41.497 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:41.007+0000 7fbea7411200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:07:41.497 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:41.090+0000 7fbea7411200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:07:41.547 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:41.396+0000 7f5922e4a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T09:07:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:41 smithi049 ceph-mon[99485]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:41.910 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-mon[103675]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:41.911 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.911 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.911 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.911 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.911 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:41.922 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:41.785+0000 7f5922e4a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T09:07:41.922 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-mon[103389]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:41.922 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.922 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.922 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.922 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:41.923 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:42.192 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:41.909+0000 7fbea7411200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:07:42.192 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:42.006+0000 7fbea7411200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:07:42.297 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:41.920+0000 7f5922e4a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T09:07:42.297 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:42.015+0000 7f5922e4a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T09:07:42.297 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:42.114+0000 7f5922e4a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T09:07:42.487 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:42.190+0000 7fbea7411200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:07:42.747 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:42.486+0000 7fbea7411200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:07:42.747 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:42.726+0000 7fbea7411200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:07:43.047 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:42.559+0000 7f5922e4a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T09:07:43.140 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:42.874+0000 7fbea7411200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:07:43.547 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:43.228+0000 7f5922e4a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T09:07:43.547 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:43.309+0000 7f5922e4a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T09:07:43.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-mon[103389]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:43.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4078959002' entity='client.admin' 2023-10-08T09:07:43.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:43 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3131626696' entity='client.admin' 2023-10-08T09:07:43.747 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:43.138+0000 7fbea7411200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:07:43.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-mon[103675]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:43.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4078959002' entity='client.admin' 2023-10-08T09:07:43.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3131626696' entity='client.admin' 2023-10-08T09:07:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:43 smithi049 ceph-mon[99485]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:43 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4078959002' entity='client.admin' 2023-10-08T09:07:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:43 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3131626696' entity='client.admin' 2023-10-08T09:07:44.165 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:43.837+0000 7fbea7411200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:07:44.165 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:43.969+0000 7fbea7411200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:07:44.343 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:44.077+0000 7f5922e4a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:07:44.343 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:44.170+0000 7f5922e4a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T09:07:44.456 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:44.164+0000 7fbea7411200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:07:44.456 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:44.277+0000 7fbea7411200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:07:44.605 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:44.605 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:44.605 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:44.606 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:44.606 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:44.606 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:44.606 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:44 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1998476562' entity='client.admin' 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:44.342+0000 7f5922e4a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:44.621 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1998476562' entity='client.admin' 2023-10-08T09:07:44.672 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi049') 2023-10-08T09:07:44.747 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:44.493+0000 7fbea7411200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:07:44.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:44.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:44.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:44.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-08T09:07:44.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-08T09:07:44.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:44.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:44 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1998476562' entity='client.admin' 2023-10-08T09:07:44.993 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:44.619+0000 7f5922e4a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T09:07:44.993 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:44.846+0000 7f5922e4a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T09:07:45.252 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:44.991+0000 7f5922e4a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T09:07:45.547 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:45.250+0000 7f5922e4a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T09:07:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T09:07:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: Reconfiguring daemon mgr.x on smithi049 2023-10-08T09:07:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: Standby manager daemon z started 2023-10-08T09:07:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T09:07:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:07:45.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T09:07:45.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: Reconfiguring daemon mgr.x on smithi049 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: Standby manager daemon z started 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T09:07:45.599 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:45 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:07:45.747 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z[105521]: 2023-10-08T09:07:45.351+0000 7fbea7411200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:07:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: Reconfiguring mgr.x (unknown last config time)... 2023-10-08T09:07:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: Reconfiguring daemon mgr.x on smithi049 2023-10-08T09:07:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: Standby manager daemon z started 2023-10-08T09:07:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-08T09:07:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:07:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-08T09:07:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:45 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.184:0/3095234328' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:07:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:46.270 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:45.947+0000 7f5922e4a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T09:07:46.270 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:46.078+0000 7f5922e4a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T09:07:46.547 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:46.268+0000 7f5922e4a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T09:07:46.547 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:46.382+0000 7f5922e4a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T09:07:46.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-mon[103389]: mgrmap e14: x(active, since 69s), standbys: z 2023-10-08T09:07:46.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T09:07:46.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:46.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3801052074' entity='client.admin' 2023-10-08T09:07:46.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1419013088' entity='client.admin' 2023-10-08T09:07:46.742 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:46 smithi049 ceph-mon[99485]: mgrmap e14: x(active, since 69s), standbys: z 2023-10-08T09:07:46.742 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:46 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T09:07:46.742 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:46 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:46.742 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:46 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3801052074' entity='client.admin' 2023-10-08T09:07:46.742 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:46 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1419013088' entity='client.admin' 2023-10-08T09:07:46.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:46 smithi184 ceph-mon[103675]: mgrmap e14: x(active, since 69s), standbys: z 2023-10-08T09:07:46.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:46 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-08T09:07:46.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:46 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:46.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:46 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3801052074' entity='client.admin' 2023-10-08T09:07:46.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:46 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1419013088' entity='client.admin' 2023-10-08T09:07:46.892 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi049') 2023-10-08T09:07:47.047 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:46.598+0000 7f5922e4a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T09:07:47.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.678+0000 7fa0cd7fa700 1 -- 172.21.15.49:0/1085058456 <== mon.2 v2:172.21.15.132:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v16)=0 v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa0c005aae0 con 0x7fa0d00fe8b0 2023-10-08T09:07:47.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 -- 172.21.15.49:0/1085058456 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0b806cf70 msgr2=0x7fa0b806f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:47.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 --2- 172.21.15.49:0/1085058456 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0b806cf70 0x7fa0b806f430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009c60 tx=0x7fa0c4009260 comp rx=0 tx=0).stop 2023-10-08T09:07:47.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 -- 172.21.15.49:0/1085058456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0d00fe8b0 msgr2=0x7fa0d01968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:47.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 --2- 172.21.15.49:0/1085058456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0d00fe8b0 0x7fa0d01968a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000e9f0 tx=0x7fa0c000ed00 comp rx=0 tx=0).stop 2023-10-08T09:07:47.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 -- 172.21.15.49:0/1085058456 shutdown_connections 2023-10-08T09:07:47.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 --2- 172.21.15.49:0/1085058456 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0b806cf70 0x7fa0b806f430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 --2- 172.21.15.49:0/1085058456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa0d0101ff0 0x7fa0d019de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 --2- 172.21.15.49:0/1085058456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa0d00fe8b0 0x7fa0d01968a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 --2- 172.21.15.49:0/1085058456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0d00ff270 0x7fa0d0196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 -- 172.21.15.49:0/1085058456 >> 172.21.15.49:0/1085058456 conn(0x7fa0d00fa490 msgr2=0x7fa0d0104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:47.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 -- 172.21.15.49:0/1085058456 shutdown_connections 2023-10-08T09:07:47.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.681+0000 7fa0d69e0700 1 -- 172.21.15.49:0/1085058456 wait complete. 2023-10-08T09:07:47.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.729+0000 7f0c27c6e700 1 -- 172.21.15.49:0/4181870059 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7f0c201147d0 con 0x7f0c2010b230 2023-10-08T09:07:47.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.734+0000 7f0c16ffd700 1 -- 172.21.15.49:0/4181870059 <== mon.2 v2:172.21.15.132:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v17)=0 v17) v1 ==== 108+0+0 (secure 0 0 0) 0x7f0c1009a800 con 0x7f0c2010b230 2023-10-08T09:07:47.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 -- 172.21.15.49:0/4181870059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c0c0a14f0 msgr2=0x7f0c0c0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:47.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 --2- 172.21.15.49:0/4181870059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c0c0a14f0 0x7f0c0c0a39b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c201a0270 tx=0x7f0c1c009260 comp rx=0 tx=0).stop 2023-10-08T09:07:47.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 -- 172.21.15.49:0/4181870059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c2010b230 msgr2=0x7f0c2019f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:47.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 --2- 172.21.15.49:0/4181870059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c2010b230 0x7f0c2019f320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c100098e0 tx=0x7f0c1000f720 comp rx=0 tx=0).stop 2023-10-08T09:07:47.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 -- 172.21.15.49:0/4181870059 shutdown_connections 2023-10-08T09:07:47.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 --2- 172.21.15.49:0/4181870059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c0c0a14f0 0x7f0c0c0a39b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 --2- 172.21.15.49:0/4181870059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c20108900 0x7f0c2019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 --2- 172.21.15.49:0/4181870059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c2010dbb0 0x7f0c201a6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 --2- 172.21.15.49:0/4181870059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c2010b230 0x7f0c2019f320 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:47.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 -- 172.21.15.49:0/4181870059 >> 172.21.15.49:0/4181870059 conn(0x7f0c2006d8a0 msgr2=0x7f0c20109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:47.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 -- 172.21.15.49:0/4181870059 shutdown_connections 2023-10-08T09:07:47.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:47.738+0000 7f0c27c6e700 1 -- 172.21.15.49:0/4181870059 wait complete. 2023-10-08T09:07:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:47 smithi184 ceph-mon[103675]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:47 smithi184 ceph-mon[103675]: Standby manager daemon y started 2023-10-08T09:07:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:47 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T09:07:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:47 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:07:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:47 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T09:07:47.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:47 smithi184 ceph-mon[103675]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:07:47.764 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:47 smithi049 ceph-mon[99485]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:47.764 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:47 smithi049 ceph-mon[99485]: Standby manager daemon y started 2023-10-08T09:07:47.764 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:47 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T09:07:47.764 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:47 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:07:47.764 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:47 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T09:07:47.764 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:47 smithi049 ceph-mon[99485]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:07:47.797 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:07:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y[104571]: 2023-10-08T09:07:47.450+0000 7f5922e4a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T09:07:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:47 smithi132 ceph-mon[103389]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:47 smithi132 ceph-mon[103389]: Standby manager daemon y started 2023-10-08T09:07:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:47 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-08T09:07:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:47 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-08T09:07:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:47 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-08T09:07:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:47 smithi132 ceph-mon[103389]: from='mgr.? 172.21.15.132:0/1991835162' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-08T09:07:48.222 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:48.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:48 smithi049 ceph-mon[99485]: mgrmap e15: x(active, since 71s), standbys: z, y 2023-10-08T09:07:48.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T09:07:48.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:48 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/419929698' entity='client.admin' 2023-10-08T09:07:48.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:48 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' 2023-10-08T09:07:48.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:48 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' 2023-10-08T09:07:48.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 -- 172.21.15.49:0/4007303877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c81012c0 msgr2=0x7fb5c8107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:48.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/4007303877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c81012c0 0x7fb5c8107b70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000b0d0 tx=0x7fb5c000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 -- 172.21.15.49:0/4007303877 shutdown_connections 2023-10-08T09:07:48.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/4007303877 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c81012c0 0x7fb5c8107b70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/4007303877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5c80ff720 0x7fb5c80ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/4007303877 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c8100920 0x7fb5c8100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 -- 172.21.15.49:0/4007303877 >> 172.21.15.49:0/4007303877 conn(0x7fb5c80face0 msgr2=0x7fb5c80fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 -- 172.21.15.49:0/4007303877 shutdown_connections 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.714+0000 7fb5cf4da700 1 -- 172.21.15.49:0/4007303877 wait complete. 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 Processor -- start 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 -- start start 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c80ff720 0x7fb5c8196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c8100920 0x7fb5c8196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5c81012c0 0x7fb5c819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:48.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c819e390 con 0x7fb5c8100920 2023-10-08T09:07:48.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c819e4d0 con 0x7fb5c81012c0 2023-10-08T09:07:48.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cf4da700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c819e660 con 0x7fb5c80ff720 2023-10-08T09:07:48.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.715+0000 7fb5cca75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c8100920 0x7fb5c8196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:48.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c8100920 0x7fb5c8196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46774/0 (socket says 172.21.15.49:46774) 2023-10-08T09:07:48.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 -- 172.21.15.49:0/2490921161 learned_addr learned my addr 172.21.15.49:0/2490921161 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:48.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cd276700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c80ff720 0x7fb5c8196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:48.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cda77700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5c81012c0 0x7fb5c819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:48.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 -- 172.21.15.49:0/2490921161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c80ff720 msgr2=0x7fb5c8196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:48.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c80ff720 0x7fb5c8196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 -- 172.21.15.49:0/2490921161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5c81012c0 msgr2=0x7fb5c819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:48.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5c81012c0 0x7fb5c819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 -- 172.21.15.49:0/2490921161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009d00 con 0x7fb5c8100920 2023-10-08T09:07:48.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cca75700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c8100920 0x7fb5c8196d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800d980 tx=0x7fb5b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:48.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5be7fc700 1 -- 172.21.15.49:0/2490921161 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b80098d0 con 0x7fb5c8100920 2023-10-08T09:07:48.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c819e940 con 0x7fb5c8100920 2023-10-08T09:07:48.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5be7fc700 1 -- 172.21.15.49:0/2490921161 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b8010460 con 0x7fb5c8100920 2023-10-08T09:07:48.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5be7fc700 1 -- 172.21.15.49:0/2490921161 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800f620 con 0x7fb5c8100920 2023-10-08T09:07:48.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.716+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c819ee90 con 0x7fb5c8100920 2023-10-08T09:07:48.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.717+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c804e4c0 con 0x7fb5c8100920 2023-10-08T09:07:48.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.719+0000 7fb5be7fc700 1 -- 172.21.15.49:0/2490921161 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fb5b8010860 con 0x7fb5c8100920 2023-10-08T09:07:48.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.719+0000 7fb5be7fc700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5b40a1470 0x7fb5b40a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:48.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.719+0000 7fb5cd276700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5b40a1470 0x7fb5b40a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:48.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.719+0000 7fb5be7fc700 1 -- 172.21.15.49:0/2490921161 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb5b80cb0d0 con 0x7fb5c8100920 2023-10-08T09:07:48.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.720+0000 7fb5cd276700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5b40a1470 0x7fb5b40a3930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8100780 tx=0x7fb5c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:48.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.724+0000 7fb5be7fc700 1 -- 172.21.15.49:0/2490921161 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb5b8096400 con 0x7fb5c8100920 2023-10-08T09:07:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:48 smithi184 ceph-mon[103675]: mgrmap e15: x(active, since 71s), standbys: z, y 2023-10-08T09:07:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T09:07:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:48 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/419929698' entity='client.admin' 2023-10-08T09:07:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:48 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' 2023-10-08T09:07:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:48 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' 2023-10-08T09:07:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:48 smithi132 ceph-mon[103389]: mgrmap e15: x(active, since 71s), standbys: z, y 2023-10-08T09:07:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-08T09:07:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:48 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/419929698' entity='client.admin' 2023-10-08T09:07:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:48 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' 2023-10-08T09:07:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:48 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' 2023-10-08T09:07:48.862 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:48.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.966+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7fb5c810af80 con 0x7fb5c8100920 2023-10-08T09:07:48.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.970+0000 7fb5be7fc700 1 -- 172.21.15.49:0/2490921161 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v18)=0 v18) v1 ==== 117+0+0 (secure 0 0 0) 0x7fb5b8020020 con 0x7fb5c8100920 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5b40a1470 msgr2=0x7fb5b40a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5b40a1470 0x7fb5b40a3930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8100780 tx=0x7fb5c400b410 comp rx=0 tx=0).stop 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c8100920 msgr2=0x7fb5c8196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c8100920 0x7fb5c8196d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800d980 tx=0x7fb5b800dc90 comp rx=0 tx=0).stop 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 shutdown_connections 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5b40a1470 0x7fb5b40a3930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5c80ff720 0x7fb5c8196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5c81012c0 0x7fb5c819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 --2- 172.21.15.49:0/2490921161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5c8100920 0x7fb5c8196d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:48.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 >> 172.21.15.49:0/2490921161 conn(0x7fb5c80face0 msgr2=0x7fb5c8068e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:48.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 shutdown_connections 2023-10-08T09:07:48.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.972+0000 7fb5cf4da700 1 -- 172.21.15.49:0/2490921161 wait complete. 2023-10-08T09:07:49.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 -- 172.21.15.49:0/2391425159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 msgr2=0x7f4378108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:49.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 --2- 172.21.15.49:0/2391425159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f4378108980 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f437000b0a0 tx=0x7f437000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 -- 172.21.15.49:0/2391425159 shutdown_connections 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 --2- 172.21.15.49:0/2391425159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43781012d0 0x7f43781036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 --2- 172.21.15.49:0/2391425159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f4378108980 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 --2- 172.21.15.49:0/2391425159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4378103c00 0x7f4378105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 -- 172.21.15.49:0/2391425159 >> 172.21.15.49:0/2391425159 conn(0x7f43780facc0 msgr2=0x7f43780fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 -- 172.21.15.49:0/2391425159 shutdown_connections 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.907+0000 7f437d0d7700 1 -- 172.21.15.49:0/2391425159 wait complete. 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 Processor -- start 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 -- start start 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43781012d0 0x7f4378196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4378103c00 0x7f4378196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f437819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819e380 con 0x7f4378106530 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819e4c0 con 0x7f4378103c00 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437d0d7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437819e650 con 0x7f43781012d0 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f437819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f437819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46790/0 (socket says 172.21.15.49:46790) 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f437759e700 1 -- 172.21.15.49:0/1241133700 learned_addr learned my addr 172.21.15.49:0/1241133700 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.908+0000 7f4376d9d700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43781012d0 0x7f4378196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437659c700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4378103c00 0x7f4378196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437759e700 1 -- 172.21.15.49:0/1241133700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43781012d0 msgr2=0x7f4378196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437759e700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43781012d0 0x7f4378196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437759e700 1 -- 172.21.15.49:0/1241133700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4378103c00 msgr2=0x7f4378196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437759e700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4378103c00 0x7f4378196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437759e700 1 -- 172.21.15.49:0/1241133700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009d00 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437759e700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f437819ddb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4370004c80 tx=0x7f4370004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f435ffff700 1 -- 172.21.15.49:0/1241133700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000e040 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437819e8d0 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f435ffff700 1 -- 172.21.15.49:0/1241133700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f437000cec0 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f435ffff700 1 -- 172.21.15.49:0/1241133700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437001b9b0 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.909+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437819ed90 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.911+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43780fc840 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.912+0000 7f435ffff700 1 -- 172.21.15.49:0/1241133700 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f437001dc50 con 0x7f4378106530 2023-10-08T09:07:49.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.912+0000 7f435ffff700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43600a13f0 0x7f43600a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.913+0000 7f435ffff700 1 -- 172.21.15.49:0/1241133700 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f43700ccc40 con 0x7f4378106530 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.913+0000 7f4376d9d700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43600a13f0 0x7f43600a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.913+0000 7f4376d9d700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43600a13f0 0x7f43600a38b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4368007c60 tx=0x7f43680073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:48.917+0000 7f435ffff700 1 -- 172.21.15.49:0/1241133700 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f43700d3080 con 0x7f4378106530 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.130+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f43780fc220 con 0x7f4378106530 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.133+0000 7f435ffff700 1 -- 172.21.15.49:0/1241133700 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v19)=0 v19) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4370098000 con 0x7f4378106530 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43600a13f0 msgr2=0x7f43600a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43600a13f0 0x7f43600a38b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4368007c60 tx=0x7f43680073d0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 msgr2=0x7f437819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f437819ddb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4370004c80 tx=0x7f4370004cb0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 shutdown_connections 2023-10-08T09:07:49.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43600a13f0 0x7f43600a38b0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4368007c60 tx=0x7f43680073d0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43781012d0 0x7f4378196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4378106530 0x7f437819ddb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 --2- 172.21.15.49:0/1241133700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4378103c00 0x7f4378196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:49.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 >> 172.21.15.49:0/1241133700 conn(0x7f43780facc0 msgr2=0x7f4378101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:49.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.135+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 shutdown_connections 2023-10-08T09:07:49.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:49.136+0000 7f437d0d7700 1 -- 172.21.15.49:0/1241133700 wait complete. 2023-10-08T09:07:49.623 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:49.623 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:49.653 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi049') 2023-10-08T09:07:49.654 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi049') 2023-10-08T09:07:49.654 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi049') 2023-10-08T09:07:49.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:49 smithi049 ceph-mon[99485]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:49.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:49 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2490921161' entity='client.admin' 2023-10-08T09:07:49.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:49 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1241133700' entity='client.admin' 2023-10-08T09:07:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:49 smithi184 ceph-mon[103675]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:49 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2490921161' entity='client.admin' 2023-10-08T09:07:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:49 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1241133700' entity='client.admin' 2023-10-08T09:07:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:49 smithi132 ceph-mon[103389]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:49 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2490921161' entity='client.admin' 2023-10-08T09:07:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:49 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1241133700' entity='client.admin' 2023-10-08T09:07:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/3264354518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc100930 msgr2=0x7f8fcc100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/3264354518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc100930 0x7f8fcc100d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40099c0 tx=0x7f8fb4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/3264354518 shutdown_connections 2023-10-08T09:07:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/3264354518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fcc0ff730 0x7f8fcc0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/3264354518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc1012d0 0x7f8fcc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/3264354518 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc100930 0x7f8fcc100d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/3264354518 >> 172.21.15.49:0/3264354518 conn(0x7f8fcc0facf0 msgr2=0x7f8fcc0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:50.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/3264354518 shutdown_connections 2023-10-08T09:07:50.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.074+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/3264354518 wait complete. 2023-10-08T09:07:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.075+0000 7f8fd2ed6700 1 Processor -- start 2023-10-08T09:07:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.075+0000 7f8fd2ed6700 1 -- start start 2023-10-08T09:07:50.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.075+0000 7f8fd2ed6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc0ff730 0x7f8fcc196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.075+0000 7f8fd2ed6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fcc100930 0x7f8fcc196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd2ed6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc1012d0 0x7f8fcc19dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd2ed6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc19e2b0 con 0x7f8fcc0ff730 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd2ed6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc19e3f0 con 0x7f8fcc1012d0 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd2ed6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc19e580 con 0x7f8fcc100930 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc0ff730 0x7f8fcc196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc0ff730 0x7f8fcc196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46798/0 (socket says 172.21.15.49:46798) 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 -- 172.21.15.49:0/2143451880 learned_addr learned my addr 172.21.15.49:0/2143451880 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fcbfff700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fcc100930 0x7f8fcc196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd1473700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc1012d0 0x7f8fcc19dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 -- 172.21.15.49:0/2143451880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fcc100930 msgr2=0x7f8fcc196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fcc100930 0x7f8fcc196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 -- 172.21.15.49:0/2143451880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc1012d0 msgr2=0x7f8fcc19dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc1012d0 0x7f8fcc19dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 -- 172.21.15.49:0/2143451880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40095f0 con 0x7f8fcc0ff730 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd1473700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc1012d0 0x7f8fcc19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd0c72700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc0ff730 0x7f8fcc196770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00add0 tx=0x7f8fbc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fc9ffb700 1 -- 172.21.15.49:0/2143451880 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00f070 con 0x7f8fcc0ff730 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc108670 con 0x7f8fcc0ff730 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fc9ffb700 1 -- 172.21.15.49:0/2143451880 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fbc007be0 con 0x7f8fcc0ff730 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.076+0000 7f8fc9ffb700 1 -- 172.21.15.49:0/2143451880 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc004280 con 0x7f8fcc0ff730 2023-10-08T09:07:50.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.077+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc108b70 con 0x7f8fcc0ff730 2023-10-08T09:07:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.077+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc106bc0 con 0x7f8fcc0ff730 2023-10-08T09:07:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.079+0000 7f8fc9ffb700 1 -- 172.21.15.49:0/2143451880 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f8fbc003a20 con 0x7f8fcc0ff730 2023-10-08T09:07:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.079+0000 7f8fc9ffb700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8fb80a13f0 0x7f8fb80a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.079+0000 7f8fcbfff700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8fb80a13f0 0x7f8fb80a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.079+0000 7f8fc9ffb700 1 -- 172.21.15.49:0/2143451880 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8fbc0ca590 con 0x7f8fcc0ff730 2023-10-08T09:07:50.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.080+0000 7f8fcbfff700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8fb80a13f0 0x7f8fb80a38b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb40037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:50.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.084+0000 7f8fc9ffb700 1 -- 172.21.15.49:0/2143451880 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8fbc095950 con 0x7f8fcc0ff730 2023-10-08T09:07:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.313+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7f8fcc10c330 con 0x7f8fcc0ff730 2023-10-08T09:07:50.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.316+0000 7f8fc9ffb700 1 -- 172.21.15.49:0/2143451880 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v20)=0 v20) v1 ==== 106+0+0 (secure 0 0 0) 0x7f8fbc098f70 con 0x7f8fcc0ff730 2023-10-08T09:07:50.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8fb80a13f0 msgr2=0x7f8fb80a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8fb80a13f0 0x7f8fb80a38b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4006010 tx=0x7f8fb40037c0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc0ff730 msgr2=0x7f8fcc196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc0ff730 0x7f8fcc196770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00add0 tx=0x7f8fbc00c330 comp rx=0 tx=0).stop 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 shutdown_connections 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8fb80a13f0 0x7f8fb80a38b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8fcc1012d0 0x7f8fcc19dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8fcc0ff730 0x7f8fcc196770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 --2- 172.21.15.49:0/2143451880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8fcc100930 0x7f8fcc196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 >> 172.21.15.49:0/2143451880 conn(0x7f8fcc0facf0 msgr2=0x7f8fcc103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 shutdown_connections 2023-10-08T09:07:50.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.319+0000 7f8fd2ed6700 1 -- 172.21.15.49:0/2143451880 wait complete. 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 -- 172.21.15.49:0/1551582344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 msgr2=0x7ff640108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 --2- 172.21.15.49:0/1551582344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640108c90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b0d0 tx=0x7ff63800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 -- 172.21.15.49:0/1551582344 shutdown_connections 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 --2- 172.21.15.49:0/1551582344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff640100840 0x7ff640100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 --2- 172.21.15.49:0/1551582344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640108c90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 --2- 172.21.15.49:0/1551582344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff640101a40 0x7ff640101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 -- 172.21.15.49:0/1551582344 >> 172.21.15.49:0/1551582344 conn(0x7ff6400fbe00 msgr2=0x7ff6400fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 -- 172.21.15.49:0/1551582344 shutdown_connections 2023-10-08T09:07:50.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.394+0000 7ff6479a1700 1 -- 172.21.15.49:0/1551582344 wait complete. 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 Processor -- start 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 -- start start 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff640100840 0x7ff6400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff640101a40 0x7ff640072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640073800 con 0x7ff6401023e0 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640073970 con 0x7ff640101a40 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff6479a1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401a0890 con 0x7ff640100840 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff645f3e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff645f3e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46816/0 (socket says 172.21.15.49:46816) 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff645f3e700 1 -- 172.21.15.49:0/2469170215 learned_addr learned my addr 172.21.15.49:0/2469170215 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:50.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.395+0000 7ff644f3c700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff640101a40 0x7ff640072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff64573d700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff640100840 0x7ff6400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff645f3e700 1 -- 172.21.15.49:0/2469170215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff640100840 msgr2=0x7ff6400744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff645f3e700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff640100840 0x7ff6400744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff645f3e700 1 -- 172.21.15.49:0/2469170215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff640101a40 msgr2=0x7ff640072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff645f3e700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff640101a40 0x7ff640072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff645f3e700 1 -- 172.21.15.49:0/2469170215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638009d00 con 0x7ff6401023e0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff645f3e700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640073120 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff638007bd0 tx=0x7ff638009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff6327fc700 1 -- 172.21.15.49:0/2469170215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63800e040 con 0x7ff6401023e0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6401a0a90 con 0x7ff6401023e0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff6327fc700 1 -- 172.21.15.49:0/2469170215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff63801dd50 con 0x7ff6401023e0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff6327fc700 1 -- 172.21.15.49:0/2469170215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63801b450 con 0x7ff6401023e0 2023-10-08T09:07:50.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.396+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401a0f00 con 0x7ff6401023e0 2023-10-08T09:07:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.397+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff640107c00 con 0x7ff6401023e0 2023-10-08T09:07:50.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.398+0000 7ff6327fc700 1 -- 172.21.15.49:0/2469170215 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ff63801d890 con 0x7ff6401023e0 2023-10-08T09:07:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.399+0000 7ff6327fc700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff62c0a13f0 0x7ff62c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.399+0000 7ff64573d700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff62c0a13f0 0x7ff62c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.399+0000 7ff6327fc700 1 -- 172.21.15.49:0/2469170215 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff6380ccf60 con 0x7ff6401023e0 2023-10-08T09:07:50.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.399+0000 7ff64573d700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff62c0a13f0 0x7ff62c0a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:50.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.403+0000 7ff6327fc700 1 -- 172.21.15.49:0/2469170215 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff638098320 con 0x7ff6401023e0 2023-10-08T09:07:50.603 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:50.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.627+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7ff64010c280 con 0x7ff6401023e0 2023-10-08T09:07:50.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.632+0000 7ff6327fc700 1 -- 172.21.15.49:0/2469170215 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff63809b940 con 0x7ff6401023e0 2023-10-08T09:07:50.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.636+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff62c0a13f0 msgr2=0x7ff62c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff62c0a13f0 0x7ff62c0a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c005e50 comp rx=0 tx=0).stop 2023-10-08T09:07:50.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 msgr2=0x7ff640073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:50.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640073120 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff638007bd0 tx=0x7ff638009440 comp rx=0 tx=0).stop 2023-10-08T09:07:50.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 shutdown_connections 2023-10-08T09:07:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff62c0a13f0 0x7ff62c0a38b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff640100840 0x7ff6400744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6401023e0 0x7ff640073120 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 --2- 172.21.15.49:0/2469170215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff640101a40 0x7ff640072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.637+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 >> 172.21.15.49:0/2469170215 conn(0x7ff6400fbe00 msgr2=0x7ff640104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:50.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.638+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 shutdown_connections 2023-10-08T09:07:50.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:50.638+0000 7ff6479a1700 1 -- 172.21.15.49:0/2469170215 wait complete. 2023-10-08T09:07:50.745 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:50.844 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi049') 2023-10-08T09:07:50.920 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi049') 2023-10-08T09:07:51.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:51 smithi049 ceph-mon[99485]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:51.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:51 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2143451880' entity='client.admin' 2023-10-08T09:07:51.460 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:51 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2469170215' entity='client.admin' 2023-10-08T09:07:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:51 smithi184 ceph-mon[103675]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:51 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2143451880' entity='client.admin' 2023-10-08T09:07:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:51 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2469170215' entity='client.admin' 2023-10-08T09:07:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:51 smithi132 ceph-mon[103389]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:51 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2143451880' entity='client.admin' 2023-10-08T09:07:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:51 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2469170215' entity='client.admin' 2023-10-08T09:07:52.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 -- 172.21.15.49:0/4025057133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 msgr2=0x7ffb7c108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:52.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 --2- 172.21.15.49:0/4025057133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c108990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400b0a0 tx=0x7ffb7400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 -- 172.21.15.49:0/4025057133 shutdown_connections 2023-10-08T09:07:52.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 --2- 172.21.15.49:0/4025057133 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7c101320 0x7ffb7c103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 --2- 172.21.15.49:0/4025057133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c108990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 --2- 172.21.15.49:0/4025057133 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb7c103c50 0x7ffb7c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 -- 172.21.15.49:0/4025057133 >> 172.21.15.49:0/4025057133 conn(0x7ffb7c0facf0 msgr2=0x7ffb7c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 -- 172.21.15.49:0/4025057133 shutdown_connections 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.364+0000 7ffb82910700 1 -- 172.21.15.49:0/4025057133 wait complete. 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 Processor -- start 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 -- start start 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb7c101320 0x7ffb7c196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7c103c50 0x7ffb7c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:52.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c19e370 con 0x7ffb7c106580 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c19e4b0 con 0x7ffb7c101320 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb82910700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c19e640 con 0x7ffb7c103c50 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb80ead700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb80ead700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46830/0 (socket says 172.21.15.49:46830) 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb80ead700 1 -- 172.21.15.49:0/3365248069 learned_addr learned my addr 172.21.15.49:0/3365248069 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb7bfff700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb7c101320 0x7ffb7c196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb7b7fe700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7c103c50 0x7ffb7c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb80ead700 1 -- 172.21.15.49:0/3365248069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7c103c50 msgr2=0x7ffb7c196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.365+0000 7ffb80ead700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7c103c50 0x7ffb7c196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb80ead700 1 -- 172.21.15.49:0/3365248069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb7c101320 msgr2=0x7ffb7c196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb80ead700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb7c101320 0x7ffb7c196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb80ead700 1 -- 172.21.15.49:0/3365248069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74009d00 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb80ead700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c19dda0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffb740094c0 tx=0x7ffb740094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb797fa700 1 -- 172.21.15.49:0/3365248069 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400e040 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb797fa700 1 -- 172.21.15.49:0/3365248069 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7401d940 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c19e8c0 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb797fa700 1 -- 172.21.15.49:0/3365248069 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7401ba40 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.366+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c19ece0 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.367+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c0fc870 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.368+0000 7ffb797fa700 1 -- 172.21.15.49:0/3365248069 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ffb7401dab0 con 0x7ffb7c106580 2023-10-08T09:07:52.706 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.368+0000 7ffb797fa700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb680a13f0 0x7ffb680a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.369+0000 7ffb797fa700 1 -- 172.21.15.49:0/3365248069 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffb740cce40 con 0x7ffb7c106580 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.369+0000 7ffb7bfff700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb680a13f0 0x7ffb680a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.369+0000 7ffb7bfff700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb680a13f0 0x7ffb680a38b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005950 tx=0x7ffb6c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.373+0000 7ffb797fa700 1 -- 172.21.15.49:0/3365248069 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffb74098200 con 0x7ffb7c106580 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.576+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7ffb7c04e4c0 con 0x7ffb7c106580 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.579+0000 7ffb797fa700 1 -- 172.21.15.49:0/3365248069 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v22)=0 v22) v1 ==== 116+0+0 (secure 0 0 0) 0x7ffb7409b820 con 0x7ffb7c106580 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb680a13f0 msgr2=0x7ffb680a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb680a13f0 0x7ffb680a38b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005950 tx=0x7ffb6c00a300 comp rx=0 tx=0).stop 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 msgr2=0x7ffb7c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c19dda0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffb740094c0 tx=0x7ffb740094f0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.707 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 shutdown_connections 2023-10-08T09:07:52.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb680a13f0 0x7ffb680a38b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb7c106580 0x7ffb7c19dda0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb7c101320 0x7ffb7c196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 --2- 172.21.15.49:0/3365248069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb7c103c50 0x7ffb7c196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:52.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 >> 172.21.15.49:0/3365248069 conn(0x7ffb7c0facf0 msgr2=0x7ffb7c101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:52.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 shutdown_connections 2023-10-08T09:07:52.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:52.582+0000 7ffb82910700 1 -- 172.21.15.49:0/3365248069 wait complete. 2023-10-08T09:07:53.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:53.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:53.225 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:53.225 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:53.225 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.225 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:53 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3365248069' entity='client.admin' 2023-10-08T09:07:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:53.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:53 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3365248069' entity='client.admin' 2023-10-08T09:07:53.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:53.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:07:53.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:07:53.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:07:53.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:07:53.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:53 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3365248069' entity='client.admin' 2023-10-08T09:07:53.915 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:53.952 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi049') 2023-10-08T09:07:54.091 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:07:54.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.330+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/1490912140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 msgr2=0x7f1d28072ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.330+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/1490912140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 0x7f1d28072ad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009f20 tx=0x7f1d2000b490 comp rx=0 tx=0).stop 2023-10-08T09:07:54.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.330+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/1490912140 shutdown_connections 2023-10-08T09:07:54.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.330+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/1490912140 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d28077f70 0x7f1d2807a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.330+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/1490912140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 0x7f1d28072ad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.330+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/1490912140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d280755a0 0x7f1d28077a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.330+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/1490912140 >> 172.21.15.49:0/1490912140 conn(0x7f1d2806d8a0 msgr2=0x7f1d2806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/1490912140 shutdown_connections 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/1490912140 wait complete. 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 Processor -- start 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 -- start start 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 0x7f1d281b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d280755a0 0x7f1d281b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d28077f70 0x7f1d281b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d281b7a10 con 0x7f1d280755a0 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d281b7b80 con 0x7f1d280726b0 2023-10-08T09:07:54.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.331+0000 7f1d2ebd4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d281b7d40 con 0x7f1d28077f70 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d280755a0 0x7f1d281b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d280755a0 0x7f1d281b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46842/0 (socket says 172.21.15.49:46842) 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 -- 172.21.15.49:0/2429147340 learned_addr learned my addr 172.21.15.49:0/2429147340 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d2c970700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 0x7f1d281b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d2d171700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d28077f70 0x7f1d281b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 -- 172.21.15.49:0/2429147340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d28077f70 msgr2=0x7f1d281b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d28077f70 0x7f1d281b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 -- 172.21.15.49:0/2429147340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 msgr2=0x7f1d281b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 0x7f1d281b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 -- 172.21.15.49:0/2429147340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d20009b80 con 0x7f1d280755a0 2023-10-08T09:07:54.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d27fff700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d280755a0 0x7f1d281b4c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800fab0 tx=0x7f1d1800fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:54.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d25ffb700 1 -- 172.21.15.49:0/2429147340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18009790 con 0x7f1d280755a0 2023-10-08T09:07:54.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d25ffb700 1 -- 172.21.15.49:0/2429147340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d1800dce0 con 0x7f1d280755a0 2023-10-08T09:07:54.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d281ae570 con 0x7f1d280755a0 2023-10-08T09:07:54.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d25ffb700 1 -- 172.21.15.49:0/2429147340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1801c7b0 con 0x7f1d280755a0 2023-10-08T09:07:54.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.332+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d281aea70 con 0x7f1d280755a0 2023-10-08T09:07:54.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.333+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d28070430 con 0x7f1d280755a0 2023-10-08T09:07:54.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.335+0000 7f1d25ffb700 1 -- 172.21.15.49:0/2429147340 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f1d18014450 con 0x7f1d280755a0 2023-10-08T09:07:54.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.335+0000 7f1d25ffb700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1d100a13f0 0x7f1d100a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.335+0000 7f1d2c970700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1d100a13f0 0x7f1d100a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.335+0000 7f1d25ffb700 1 -- 172.21.15.49:0/2429147340 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1d180cf300 con 0x7f1d280755a0 2023-10-08T09:07:54.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.336+0000 7f1d2c970700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1d100a13f0 0x7f1d100a38b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d200079d0 tx=0x7f1d20007ac0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:54.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.340+0000 7f1d25ffb700 1 -- 172.21.15.49:0/2429147340 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1d1809a630 con 0x7f1d280755a0 2023-10-08T09:07:54.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.542+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f1d2804e4c0 con 0x7f1d280755a0 2023-10-08T09:07:54.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.545+0000 7f1d25ffb700 1 -- 172.21.15.49:0/2429147340 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v23)=0 v23) v1 ==== 120+0+0 (secure 0 0 0) 0x7f1d1809dc50 con 0x7f1d280755a0 2023-10-08T09:07:54.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1d100a13f0 msgr2=0x7f1d100a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1d100a13f0 0x7f1d100a38b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d200079d0 tx=0x7f1d20007ac0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d280755a0 msgr2=0x7f1d281b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d280755a0 0x7f1d281b4c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800fab0 tx=0x7f1d1800fdc0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 shutdown_connections 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1d100a13f0 0x7f1d100a38b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1d28077f70 0x7f1d281b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d280726b0 0x7f1d281b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 --2- 172.21.15.49:0/2429147340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d280755a0 0x7f1d281b4c80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 >> 172.21.15.49:0/2429147340 conn(0x7f1d2806d8a0 msgr2=0x7f1d28076c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 shutdown_connections 2023-10-08T09:07:54.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.548+0000 7f1d2ebd4700 1 -- 172.21.15.49:0/2429147340 wait complete. 2023-10-08T09:07:54.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 -- 172.21.15.49:0/1938589863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26700ff730 msgr2=0x7f26700ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1938589863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26700ff730 0x7f26700ffb50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26600099c0 tx=0x7f2660009cd0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 -- 172.21.15.49:0/1938589863 shutdown_connections 2023-10-08T09:07:54.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1938589863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26701012d0 0x7f2670107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1938589863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26700ff730 0x7f26700ffb50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1938589863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2670100930 0x7f2670100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 -- 172.21.15.49:0/1938589863 >> 172.21.15.49:0/1938589863 conn(0x7f26700facf0 msgr2=0x7f26700fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:54.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 -- 172.21.15.49:0/1938589863 shutdown_connections 2023-10-08T09:07:54.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.878+0000 7f2675bf9700 1 -- 172.21.15.49:0/1938589863 wait complete. 2023-10-08T09:07:54.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 Processor -- start 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 -- start start 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26700ff730 0x7f2670068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2670100930 0x7f267019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26701012d0 0x7f267019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267019e270 con 0x7f26701012d0 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267019e3e0 con 0x7f26700ff730 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.879+0000 7f2675bf9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267019e5a0 con 0x7f2670100930 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26701012d0 0x7f267019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26701012d0 0x7f267019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46858/0 (socket says 172.21.15.49:46858) 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 -- 172.21.15.49:0/1546641635 learned_addr learned my addr 172.21.15.49:0/1546641635 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266f7fe700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26700ff730 0x7f2670068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266effd700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2670100930 0x7f267019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 -- 172.21.15.49:0/1546641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2670100930 msgr2=0x7f267019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2670100930 0x7f267019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 -- 172.21.15.49:0/1546641635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26700ff730 msgr2=0x7f2670068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26700ff730 0x7f2670068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 -- 172.21.15.49:0/1546641635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26600095f0 con 0x7f26701012d0 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266ffff700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26701012d0 0x7f267019dad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2668007ee0 tx=0x7f266800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266cff9700 1 -- 172.21.15.49:0/1546641635 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800dd20 con 0x7f26701012d0 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266cff9700 1 -- 172.21.15.49:0/1546641635 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f266800f040 con 0x7f26701012d0 2023-10-08T09:07:54.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f266cff9700 1 -- 172.21.15.49:0/1546641635 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680048a0 con 0x7f26701012d0 2023-10-08T09:07:54.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267019e880 con 0x7f26701012d0 2023-10-08T09:07:54.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.880+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267019ed30 con 0x7f26701012d0 2023-10-08T09:07:54.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.881+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2670106cb0 con 0x7f26701012d0 2023-10-08T09:07:54.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.882+0000 7f266cff9700 1 -- 172.21.15.49:0/1546641635 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f2668003a00 con 0x7f26701012d0 2023-10-08T09:07:54.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.883+0000 7f266cff9700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26580a1470 0x7f26580a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:54.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.883+0000 7f266f7fe700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26580a1470 0x7f26580a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:54.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.883+0000 7f266cff9700 1 -- 172.21.15.49:0/1546641635 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f26680ca760 con 0x7f26701012d0 2023-10-08T09:07:54.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.883+0000 7f266f7fe700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26580a1470 0x7f26580a3930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2660006010 tx=0x7f266000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:54.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:54.887+0000 7f266cff9700 1 -- 172.21.15.49:0/1546641635 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f266804f100 con 0x7f26701012d0 2023-10-08T09:07:55.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.099+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f267004e4c0 con 0x7f26701012d0 2023-10-08T09:07:55.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.102+0000 7f266cff9700 1 -- 172.21.15.49:0/1546641635 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v24)=0 v24) v1 ==== 119+0+0 (secure 0 0 0) 0x7f2668095a90 con 0x7f26701012d0 2023-10-08T09:07:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26580a1470 msgr2=0x7f26580a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26580a1470 0x7f26580a3930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2660006010 tx=0x7f266000b540 comp rx=0 tx=0).stop 2023-10-08T09:07:55.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26701012d0 msgr2=0x7f267019dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26701012d0 0x7f267019dad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2668007ee0 tx=0x7f266800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 shutdown_connections 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26580a1470 0x7f26580a3930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26701012d0 0x7f267019dad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26700ff730 0x7f2670068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 --2- 172.21.15.49:0/1546641635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2670100930 0x7f267019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 >> 172.21.15.49:0/1546641635 conn(0x7f26700facf0 msgr2=0x7f2670103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 shutdown_connections 2023-10-08T09:07:55.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.105+0000 7f2675bf9700 1 -- 172.21.15.49:0/1546641635 wait complete. 2023-10-08T09:07:55.620 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:55 smithi049 ceph-mon[99485]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:55.621 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:55 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2429147340' entity='client.admin' 2023-10-08T09:07:55.621 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:55 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1546641635' entity='client.admin' 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.976+0000 7fc4df959700 1 -- 172.21.15.49:0/515510818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d81003a0 msgr2=0x7fc4d8107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.976+0000 7fc4df959700 1 --2- 172.21.15.49:0/515510818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d81003a0 0x7fc4d8107c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000b0a0 tx=0x7fc4d000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.976+0000 7fc4df959700 1 -- 172.21.15.49:0/515510818 shutdown_connections 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.976+0000 7fc4df959700 1 --2- 172.21.15.49:0/515510818 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d81003a0 0x7fc4d8107c60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.976+0000 7fc4df959700 1 --2- 172.21.15.49:0/515510818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4d8100cc0 0x7fc4d81010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.976+0000 7fc4df959700 1 --2- 172.21.15.49:0/515510818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 0x7fc4d80ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.976+0000 7fc4df959700 1 -- 172.21.15.49:0/515510818 >> 172.21.15.49:0/515510818 conn(0x7fc4d80fa450 msgr2=0x7fc4d80fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:55.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.977+0000 7fc4df959700 1 -- 172.21.15.49:0/515510818 shutdown_connections 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.977+0000 7fc4df959700 1 -- 172.21.15.49:0/515510818 wait complete. 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 Processor -- start 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 -- start start 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 0x7fc4d8196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4d81003a0 0x7fc4d8196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d8100cc0 0x7fc4d819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d819e370 con 0x7fc4d80ffa00 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d819e4b0 con 0x7fc4d81003a0 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4df959700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d819e640 con 0x7fc4d8100cc0 2023-10-08T09:07:55.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4dd6f5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 0x7fc4d8196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4dd6f5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 0x7fc4d8196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46868/0 (socket says 172.21.15.49:46868) 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4dd6f5700 1 -- 172.21.15.49:0/2092314395 learned_addr learned my addr 172.21.15.49:0/2092314395 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.978+0000 7fc4dcef4700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4d81003a0 0x7fc4d8196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4ddef6700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d8100cc0 0x7fc4d819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4dd6f5700 1 -- 172.21.15.49:0/2092314395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d8100cc0 msgr2=0x7fc4d819dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4dd6f5700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d8100cc0 0x7fc4d819dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4dd6f5700 1 -- 172.21.15.49:0/2092314395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4d81003a0 msgr2=0x7fc4d8196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4dd6f5700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4d81003a0 0x7fc4d8196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4dd6f5700 1 -- 172.21.15.49:0/2092314395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0009d00 con 0x7fc4d80ffa00 2023-10-08T09:07:55.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4dcef4700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4d81003a0 0x7fc4d8196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:07:55.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4dd6f5700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 0x7fc4d8196880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9a0 tx=0x7fc4d400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:55.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4ce7fc700 1 -- 172.21.15.49:0/2092314395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4009790 con 0x7fc4d80ffa00 2023-10-08T09:07:55.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4ce7fc700 1 -- 172.21.15.49:0/2092314395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4d40044e0 con 0x7fc4d80ffa00 2023-10-08T09:07:55.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4ce7fc700 1 -- 172.21.15.49:0/2092314395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4003d70 con 0x7fc4d80ffa00 2023-10-08T09:07:55.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d819e920 con 0x7fc4d80ffa00 2023-10-08T09:07:55.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.979+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d819edd0 con 0x7fc4d80ffa00 2023-10-08T09:07:55.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.981+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d8106bd0 con 0x7fc4d80ffa00 2023-10-08T09:07:55.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.982+0000 7fc4ce7fc700 1 -- 172.21.15.49:0/2092314395 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fc4d4003ef0 con 0x7fc4d80ffa00 2023-10-08T09:07:55.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.982+0000 7fc4ce7fc700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4c40a1470 0x7fc4c40a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:55.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.983+0000 7fc4dcef4700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4c40a1470 0x7fc4c40a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:55.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.983+0000 7fc4ce7fc700 1 -- 172.21.15.49:0/2092314395 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc4d4014070 con 0x7fc4d80ffa00 2023-10-08T09:07:55.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.983+0000 7fc4dcef4700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4c40a1470 0x7fc4c40a3930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009cb0 tx=0x7fc4c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:55.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:55.990+0000 7fc4ce7fc700 1 -- 172.21.15.49:0/2092314395 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc4d40967e0 con 0x7fc4d80ffa00 2023-10-08T09:07:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:55 smithi184 ceph-mon[103675]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:55 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2429147340' entity='client.admin' 2023-10-08T09:07:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:55 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1546641635' entity='client.admin' 2023-10-08T09:07:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:55 smithi132 ceph-mon[103389]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:55 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2429147340' entity='client.admin' 2023-10-08T09:07:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:55 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1546641635' entity='client.admin' 2023-10-08T09:07:56.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.222+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7fc4d804e4c0 con 0x7fc4d80ffa00 2023-10-08T09:07:56.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.227+0000 7fc4ce7fc700 1 -- 172.21.15.49:0/2092314395 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc4d4099e00 con 0x7fc4d80ffa00 2023-10-08T09:07:56.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4c40a1470 msgr2=0x7fc4c40a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4c40a1470 0x7fc4c40a3930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009cb0 tx=0x7fc4c8009260 comp rx=0 tx=0).stop 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 msgr2=0x7fc4d8196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 0x7fc4d8196880 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9a0 tx=0x7fc4d400ed60 comp rx=0 tx=0).stop 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 shutdown_connections 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4c40a1470 0x7fc4c40a3930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4d8100cc0 0x7fc4d819dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4d80ffa00 0x7fc4d8196880 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 --2- 172.21.15.49:0/2092314395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4d81003a0 0x7fc4d8196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 >> 172.21.15.49:0/2092314395 conn(0x7fc4d80fa450 msgr2=0x7fc4d8103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:56.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 shutdown_connections 2023-10-08T09:07:56.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.229+0000 7fc4df959700 1 -- 172.21.15.49:0/2092314395 wait complete. 2023-10-08T09:07:56.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 -- 172.21.15.49:0/1373269193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 msgr2=0x7f3ec4106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:56.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 --2- 172.21.15.49:0/1373269193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4106040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009990 tx=0x7f3eb4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 -- 172.21.15.49:0/1373269193 shutdown_connections 2023-10-08T09:07:56.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 --2- 172.21.15.49:0/1373269193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ec4101320 0x7f3ec4103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 --2- 172.21.15.49:0/1373269193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec4106580 0x7f3ec4108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 --2- 172.21.15.49:0/1373269193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4106040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 -- 172.21.15.49:0/1373269193 >> 172.21.15.49:0/1373269193 conn(0x7f3ec40facf0 msgr2=0x7f3ec40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:56.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 -- 172.21.15.49:0/1373269193 shutdown_connections 2023-10-08T09:07:56.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.386+0000 7f3ecb683700 1 -- 172.21.15.49:0/1373269193 wait complete. 2023-10-08T09:07:56.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 Processor -- start 2023-10-08T09:07:56.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 -- start start 2023-10-08T09:07:56.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec4101320 0x7f3ec41967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:56.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:56.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ec4106580 0x7f3ec419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:56.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e300 con 0x7f3ec4103c50 2023-10-08T09:07:56.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e440 con 0x7f3ec4101320 2023-10-08T09:07:56.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ecb683700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e5d0 con 0x7f3ec4106580 2023-10-08T09:07:56.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec8c1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:56.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec8c1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46884/0 (socket says 172.21.15.49:46884) 2023-10-08T09:07:56.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec8c1e700 1 -- 172.21.15.49:0/556738705 learned_addr learned my addr 172.21.15.49:0/556738705 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:56.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec941f700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec4101320 0x7f3ec41967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:56.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec9c20700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ec4106580 0x7f3ec419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec8c1e700 1 -- 172.21.15.49:0/556738705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ec4106580 msgr2=0x7f3ec419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec8c1e700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ec4106580 0x7f3ec419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.387+0000 7f3ec8c1e700 1 -- 172.21.15.49:0/556738705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec4101320 msgr2=0x7f3ec41967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3ec8c1e700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec4101320 0x7f3ec41967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3ec8c1e700 1 -- 172.21.15.49:0/556738705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb40095f0 con 0x7f3ec4103c50 2023-10-08T09:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3ec941f700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec4101320 0x7f3ec41967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3ec8c1e700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4196d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4005910 tx=0x7f3eb4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3eba7fc700 1 -- 172.21.15.49:0/556738705 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb401d070 con 0x7f3ec4103c50 2023-10-08T09:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3eba7fc700 1 -- 172.21.15.49:0/556738705 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb4022470 con 0x7f3ec4103c50 2023-10-08T09:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec419e850 con 0x7f3ec4103c50 2023-10-08T09:07:56.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3eba7fc700 1 -- 172.21.15.49:0/556738705 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4017650 con 0x7f3ec4103c50 2023-10-08T09:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.388+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec419ecc0 con 0x7f3ec4103c50 2023-10-08T09:07:56.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.389+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec40fc870 con 0x7f3ec4103c50 2023-10-08T09:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.391+0000 7f3eba7fc700 1 -- 172.21.15.49:0/556738705 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f3eb400fad0 con 0x7f3ec4103c50 2023-10-08T09:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.391+0000 7f3eba7fc700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3eb00a1470 0x7f3eb00a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.392+0000 7f3ec941f700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3eb00a1470 0x7f3eb00a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:56.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.392+0000 7f3eba7fc700 1 -- 172.21.15.49:0/556738705 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3eb40cc8e0 con 0x7f3ec4103c50 2023-10-08T09:07:56.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.392+0000 7f3ec941f700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3eb00a1470 0x7f3eb00a3930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009cb0 tx=0x7f3ec0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:56.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.396+0000 7f3eba7fc700 1 -- 172.21.15.49:0/556738705 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3eb4097ca0 con 0x7f3ec4103c50 2023-10-08T09:07:56.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.597+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f3ec404e4c0 con 0x7f3ec4103c50 2023-10-08T09:07:56.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.600+0000 7f3eba7fc700 1 -- 172.21.15.49:0/556738705 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v26)=0 v26) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3eb409b2c0 con 0x7f3ec4103c50 2023-10-08T09:07:56.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3eb00a1470 msgr2=0x7f3eb00a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:56.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3eb00a1470 0x7f3eb00a3930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009cb0 tx=0x7f3ec0009260 comp rx=0 tx=0).stop 2023-10-08T09:07:56.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 msgr2=0x7f3ec4196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:56.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4196d00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4005910 tx=0x7f3eb4009580 comp rx=0 tx=0).stop 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 shutdown_connections 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3eb00a1470 0x7f3eb00a3930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ec4106580 0x7f3ec419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ec4101320 0x7f3ec41967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 --2- 172.21.15.49:0/556738705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec4103c50 0x7f3ec4196d00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 >> 172.21.15.49:0/556738705 conn(0x7f3ec40facf0 msgr2=0x7f3ec41031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 shutdown_connections 2023-10-08T09:07:56.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:56.602+0000 7f3ecb683700 1 -- 172.21.15.49:0/556738705 wait complete. 2023-10-08T09:07:57.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:57 smithi049 ceph-mon[99485]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:57.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:57 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2092314395' entity='client.admin' 2023-10-08T09:07:57.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:57 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/556738705' entity='client.admin' 2023-10-08T09:07:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:57 smithi184 ceph-mon[103675]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:57 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2092314395' entity='client.admin' 2023-10-08T09:07:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:57 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/556738705' entity='client.admin' 2023-10-08T09:07:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.416+0000 7f6c563e2700 1 -- 172.21.15.49:0/2314734719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 msgr2=0x7f6c501060a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.416+0000 7f6c563e2700 1 --2- 172.21.15.49:0/2314734719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c501060a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009990 tx=0x7f6c44009ca0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.416+0000 7f6c563e2700 1 -- 172.21.15.49:0/2314734719 shutdown_connections 2023-10-08T09:07:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.416+0000 7f6c563e2700 1 --2- 172.21.15.49:0/2314734719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c50101380 0x7f6c50103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.416+0000 7f6c563e2700 1 --2- 172.21.15.49:0/2314734719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c501065e0 0x7f6c501089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.416+0000 7f6c563e2700 1 --2- 172.21.15.49:0/2314734719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c501060a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.416+0000 7f6c563e2700 1 -- 172.21.15.49:0/2314734719 >> 172.21.15.49:0/2314734719 conn(0x7f6c500facf0 msgr2=0x7f6c500fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.417+0000 7f6c563e2700 1 -- 172.21.15.49:0/2314734719 shutdown_connections 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.417+0000 7f6c563e2700 1 -- 172.21.15.49:0/2314734719 wait complete. 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.417+0000 7f6c563e2700 1 Processor -- start 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.417+0000 7f6c563e2700 1 -- start start 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.417+0000 7f6c563e2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c50101380 0x7f6c50196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.417+0000 7f6c563e2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c50196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.417+0000 7f6c563e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c501065e0 0x7f6c5019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c563e2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5019e240 con 0x7f6c50103cb0 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c563e2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5019e380 con 0x7f6c501065e0 2023-10-08T09:07:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c563e2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5019e510 con 0x7f6c50101380 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c50196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c50196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46898/0 (socket says 172.21.15.49:46898) 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 -- 172.21.15.49:0/3891121975 learned_addr learned my addr 172.21.15.49:0/3891121975 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c5497f700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c501065e0 0x7f6c5019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4ffff700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c50101380 0x7f6c50196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 -- 172.21.15.49:0/3891121975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c50101380 msgr2=0x7f6c50196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c50101380 0x7f6c50196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 -- 172.21.15.49:0/3891121975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c501065e0 msgr2=0x7f6c5019dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c501065e0 0x7f6c5019dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 -- 172.21.15.49:0/3891121975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440095f0 con 0x7f6c50103cb0 2023-10-08T09:07:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4f7fe700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c50196c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400b5c0 tx=0x7f6c4400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4d7fa700 1 -- 172.21.15.49:0/3891121975 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4401d070 con 0x7f6c50103cb0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4d7fa700 1 -- 172.21.15.49:0/3891121975 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c44022470 con 0x7f6c50103cb0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5019e790 con 0x7f6c50103cb0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c4d7fa700 1 -- 172.21.15.49:0/3891121975 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c44017650 con 0x7f6c50103cb0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.418+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5019eba0 con 0x7f6c50103cb0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.419+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c500fc870 con 0x7f6c50103cb0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.420+0000 7f6c4d7fa700 1 -- 172.21.15.49:0/3891121975 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f6c440225e0 con 0x7f6c50103cb0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.421+0000 7f6c4d7fa700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6c380a13f0 0x7f6c380a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.421+0000 7f6c4ffff700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6c380a13f0 0x7f6c380a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.421+0000 7f6c4d7fa700 1 -- 172.21.15.49:0/3891121975 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6c440cc550 con 0x7f6c50103cb0 2023-10-08T09:07:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.421+0000 7f6c4ffff700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6c380a13f0 0x7f6c380a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005950 tx=0x7f6c400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.425+0000 7f6c4d7fa700 1 -- 172.21.15.49:0/3891121975 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6c44097a30 con 0x7f6c50103cb0 2023-10-08T09:07:57.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:57 smithi132 ceph-mon[103389]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:57.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:57 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2092314395' entity='client.admin' 2023-10-08T09:07:57.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:57 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/556738705' entity='client.admin' 2023-10-08T09:07:57.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.632+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f6c5004e4c0 con 0x7f6c50103cb0 2023-10-08T09:07:57.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.635+0000 7f6c4d7fa700 1 -- 172.21.15.49:0/3891121975 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v27)=0 v27) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6c440270c0 con 0x7f6c50103cb0 2023-10-08T09:07:57.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.637+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6c380a13f0 msgr2=0x7f6c380a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.637+0000 7f6c563e2700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6c380a13f0 0x7f6c380a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6c40005950 tx=0x7f6c400058e0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.637+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 msgr2=0x7f6c50196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.637+0000 7f6c563e2700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c50196c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400b5c0 tx=0x7f6c4400f690 comp rx=0 tx=0).stop 2023-10-08T09:07:57.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.637+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 shutdown_connections 2023-10-08T09:07:57.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.638+0000 7f6c563e2700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6c380a13f0 0x7f6c380a38b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.638+0000 7f6c563e2700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c50101380 0x7f6c50196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.638+0000 7f6c563e2700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c501065e0 0x7f6c5019dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.638+0000 7f6c563e2700 1 --2- 172.21.15.49:0/3891121975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6c50103cb0 0x7f6c50196c40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.638+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 >> 172.21.15.49:0/3891121975 conn(0x7f6c500facf0 msgr2=0x7f6c50101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:57.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.638+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 shutdown_connections 2023-10-08T09:07:57.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.638+0000 7f6c563e2700 1 -- 172.21.15.49:0/3891121975 wait complete. 2023-10-08T09:07:57.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.679+0000 7f5cba989700 1 -- 172.21.15.49:0/2139905424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb4101a40 msgr2=0x7f5cb4101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.679+0000 7f5cba989700 1 --2- 172.21.15.49:0/2139905424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb4101a40 0x7f5cb4101ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80099c0 tx=0x7f5ca8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.679+0000 7f5cba989700 1 -- 172.21.15.49:0/2139905424 shutdown_connections 2023-10-08T09:07:57.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.679+0000 7f5cba989700 1 --2- 172.21.15.49:0/2139905424 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb4100840 0x7f5cb4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.679+0000 7f5cba989700 1 --2- 172.21.15.49:0/2139905424 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cb41023e0 0x7f5cb4108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.679+0000 7f5cba989700 1 --2- 172.21.15.49:0/2139905424 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb4101a40 0x7f5cb4101ea0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.679+0000 7f5cba989700 1 -- 172.21.15.49:0/2139905424 >> 172.21.15.49:0/2139905424 conn(0x7f5cb40fbe00 msgr2=0x7f5cb40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.680+0000 7f5cba989700 1 -- 172.21.15.49:0/2139905424 shutdown_connections 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.680+0000 7f5cba989700 1 -- 172.21.15.49:0/2139905424 wait complete. 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.680+0000 7f5cba989700 1 Processor -- start 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.680+0000 7f5cba989700 1 -- start start 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cba989700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb4100840 0x7f5cb4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cba989700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cb4101a40 0x7f5cb4198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cba989700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb41023e0 0x7f5cb419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cba989700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41a0450 con 0x7f5cb41023e0 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cba989700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41a0590 con 0x7f5cb4101a40 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cba989700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41a0720 con 0x7f5cb4100840 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb41023e0 0x7f5cb419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb41023e0 0x7f5cb419fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46914/0 (socket says 172.21.15.49:46914) 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 -- 172.21.15.49:0/1792691983 learned_addr learned my addr 172.21.15.49:0/1792691983 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb3fff700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb4100840 0x7f5cb4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb37fe700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cb4101a40 0x7f5cb4198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 -- 172.21.15.49:0/1792691983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb4100840 msgr2=0x7f5cb4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb4100840 0x7f5cb4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 -- 172.21.15.49:0/1792691983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cb4101a40 msgr2=0x7f5cb4198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cb4101a40 0x7f5cb4198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 -- 172.21.15.49:0/1792691983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca80095f0 con 0x7f5cb41023e0 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb8f26700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb41023e0 0x7f5cb419fe80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007f10 tx=0x7f5cac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb17fa700 1 -- 172.21.15.49:0/1792691983 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac017070 con 0x7f5cb41023e0 2023-10-08T09:07:57.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb41a0a00 con 0x7f5cb41023e0 2023-10-08T09:07:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb17fa700 1 -- 172.21.15.49:0/1792691983 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cac00f040 con 0x7f5cb41023e0 2023-10-08T09:07:57.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.681+0000 7f5cb17fa700 1 -- 172.21.15.49:0/1792691983 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac005610 con 0x7f5cb41023e0 2023-10-08T09:07:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.682+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb41a0eb0 con 0x7f5cb41023e0 2023-10-08T09:07:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.683+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb404e4c0 con 0x7f5cb41023e0 2023-10-08T09:07:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.684+0000 7f5cb17fa700 1 -- 172.21.15.49:0/1792691983 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5cac004ad0 con 0x7f5cb41023e0 2023-10-08T09:07:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.684+0000 7f5cb17fa700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c9c0a13f0 0x7f5c9c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.684+0000 7f5cb17fa700 1 -- 172.21.15.49:0/1792691983 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5cac0ca730 con 0x7f5cb41023e0 2023-10-08T09:07:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.684+0000 7f5cb3fff700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c9c0a13f0 0x7f5c9c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:57.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.685+0000 7f5cb3fff700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c9c0a13f0 0x7f5c9c0a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5cb41018a0 tx=0x7f5ca400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:57.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.689+0000 7f5cb17fa700 1 -- 172.21.15.49:0/1792691983 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5cac095a60 con 0x7f5cb41023e0 2023-10-08T09:07:57.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.908+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7f5cb41a11a0 con 0x7f5cb41023e0 2023-10-08T09:07:57.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.911+0000 7f5cb17fa700 1 -- 172.21.15.49:0/1792691983 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v28)=0 v28) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5cac099080 con 0x7f5cb41023e0 2023-10-08T09:07:57.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c9c0a13f0 msgr2=0x7f5c9c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c9c0a13f0 0x7f5c9c0a38b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5cb41018a0 tx=0x7f5ca400a3b0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb41023e0 msgr2=0x7f5cb419fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:57.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb41023e0 0x7f5cb419fe80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5cac007f10 tx=0x7f5cac00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 shutdown_connections 2023-10-08T09:07:57.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c9c0a13f0 0x7f5c9c0a38b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cb4100840 0x7f5cb4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cb41023e0 0x7f5cb419fe80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 --2- 172.21.15.49:0/1792691983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cb4101a40 0x7f5cb4198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:57.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.913+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 >> 172.21.15.49:0/1792691983 conn(0x7f5cb40fbe00 msgr2=0x7f5cb4072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:57.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.914+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 shutdown_connections 2023-10-08T09:07:57.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:57.914+0000 7f5cba989700 1 -- 172.21.15.49:0/1792691983 wait complete. 2023-10-08T09:07:58.529 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi049') 2023-10-08T09:07:58.529 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi049') 2023-10-08T09:07:58.529 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi049') 2023-10-08T09:07:58.530 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi049') 2023-10-08T09:07:58.530 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi049') 2023-10-08T09:07:58.530 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi049') 2023-10-08T09:07:58.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:58 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3891121975' entity='client.admin' 2023-10-08T09:07:58.744 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:58 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1792691983' entity='client.admin' 2023-10-08T09:07:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 -- 172.21.15.49:0/2244553772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78103c50 msgr2=0x7f3e78106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/2244553772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78103c50 0x7f3e78106040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009990 tx=0x7f3e6c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:07:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 -- 172.21.15.49:0/2244553772 shutdown_connections 2023-10-08T09:07:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/2244553772 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e78106580 0x7f3e78108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/2244553772 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e78101320 0x7f3e78103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/2244553772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78103c50 0x7f3e78106040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 -- 172.21.15.49:0/2244553772 >> 172.21.15.49:0/2244553772 conn(0x7f3e780facf0 msgr2=0x7f3e780fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 -- 172.21.15.49:0/2244553772 shutdown_connections 2023-10-08T09:07:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 -- 172.21.15.49:0/2244553772 wait complete. 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.970+0000 7f3e7cea5700 1 Processor -- start 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7cea5700 1 -- start start 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7cea5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e78101320 0x7f3e78100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7cea5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e78103c50 0x7f3e780ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7cea5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78106580 0x7f3e780ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7cea5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e780fffb0 con 0x7f3e78106580 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7cea5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e78100120 con 0x7f3e78103c50 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7cea5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e781a0800 con 0x7f3e78101320 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78106580 0x7f3e780ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78106580 0x7f3e780ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46928/0 (socket says 172.21.15.49:46928) 2023-10-08T09:07:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 -- 172.21.15.49:0/3360654087 learned_addr learned my addr 172.21.15.49:0/3360654087 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e75d9b700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e78103c50 0x7f3e780ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e7659c700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e78101320 0x7f3e78100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 -- 172.21.15.49:0/3360654087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e78101320 msgr2=0x7f3e78100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e78101320 0x7f3e78100c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 -- 172.21.15.49:0/3360654087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e78103c50 msgr2=0x7f3e780ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e78103c50 0x7f3e780ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 -- 172.21.15.49:0/3360654087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0095f0 con 0x7f3e78106580 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e76d9d700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78106580 0x7f3e780ff810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000ef90 tx=0x7f3e7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.971+0000 7f3e677fe700 1 -- 172.21.15.49:0/3360654087 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70018070 con 0x7f3e78106580 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.972+0000 7f3e677fe700 1 -- 172.21.15.49:0/3360654087 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7000f040 con 0x7f3e78106580 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.972+0000 7f3e677fe700 1 -- 172.21.15.49:0/3360654087 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000cda0 con 0x7f3e78106580 2023-10-08T09:07:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.972+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e781a0a30 con 0x7f3e78106580 2023-10-08T09:07:58.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.972+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e781a0ee0 con 0x7f3e78106580 2023-10-08T09:07:58.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.973+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e780fc870 con 0x7f3e78106580 2023-10-08T09:07:58.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.974+0000 7f3e677fe700 1 -- 172.21.15.49:0/3360654087 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f3e70003a00 con 0x7f3e78106580 2023-10-08T09:07:58.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.974+0000 7f3e677fe700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3e600a1470 0x7f3e600a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:58.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.974+0000 7f3e7659c700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3e600a1470 0x7f3e600a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:58.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.974+0000 7f3e677fe700 1 -- 172.21.15.49:0/3360654087 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e700caae0 con 0x7f3e78106580 2023-10-08T09:07:58.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.975+0000 7f3e7659c700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3e600a1470 0x7f3e600a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005950 tx=0x7f3e6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:58.979+0000 7f3e677fe700 1 -- 172.21.15.49:0/3360654087 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e70095ea0 con 0x7f3e78106580 2023-10-08T09:07:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:58 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3891121975' entity='client.admin' 2023-10-08T09:07:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:58 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1792691983' entity='client.admin' 2023-10-08T09:07:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:58 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3891121975' entity='client.admin' 2023-10-08T09:07:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:58 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1792691983' entity='client.admin' 2023-10-08T09:07:59.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.189+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f3e7810c300 con 0x7f3e78106580 2023-10-08T09:07:59.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.192+0000 7f3e677fe700 1 -- 172.21.15.49:0/3360654087 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v29)=0 v29) v1 ==== 120+0+0 (secure 0 0 0) 0x7f3e700994c0 con 0x7f3e78106580 2023-10-08T09:07:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3e600a1470 msgr2=0x7f3e600a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3e600a1470 0x7f3e600a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e68005950 tx=0x7f3e6800b410 comp rx=0 tx=0).stop 2023-10-08T09:07:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78106580 msgr2=0x7f3e780ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78106580 0x7f3e780ff810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000ef90 tx=0x7f3e7000c3a0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 shutdown_connections 2023-10-08T09:07:59.198 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3e600a1470 0x7f3e600a3930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e78101320 0x7f3e78100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e78106580 0x7f3e780ff810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 --2- 172.21.15.49:0/3360654087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e78103c50 0x7f3e780ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.194+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 >> 172.21.15.49:0/3360654087 conn(0x7f3e780facf0 msgr2=0x7f3e781031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.195+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 shutdown_connections 2023-10-08T09:07:59.199 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.195+0000 7f3e7cea5700 1 -- 172.21.15.49:0/3360654087 wait complete. 2023-10-08T09:07:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 -- 172.21.15.49:0/3469882074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464103c30 msgr2=0x7f6464106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 --2- 172.21.15.49:0/3469882074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464103c30 0x7f6464106020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6458009990 tx=0x7f6458009ca0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 -- 172.21.15.49:0/3469882074 shutdown_connections 2023-10-08T09:07:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 --2- 172.21.15.49:0/3469882074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6464106560 0x7f6464108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 --2- 172.21.15.49:0/3469882074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464101300 0x7f64641036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 --2- 172.21.15.49:0/3469882074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464103c30 0x7f6464106020 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 -- 172.21.15.49:0/3469882074 >> 172.21.15.49:0/3469882074 conn(0x7f64640facf0 msgr2=0x7f64640fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.210+0000 7f646a9b9700 1 -- 172.21.15.49:0/3469882074 shutdown_connections 2023-10-08T09:07:59.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.211+0000 7f646a9b9700 1 -- 172.21.15.49:0/3469882074 wait complete. 2023-10-08T09:07:59.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.211+0000 7f646a9b9700 1 Processor -- start 2023-10-08T09:07:59.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.211+0000 7f646a9b9700 1 -- start start 2023-10-08T09:07:59.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.211+0000 7f646a9b9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464101300 0x7f6464100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:59.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f646a9b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464103c30 0x7f64640ff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:59.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f646a9b9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6464106560 0x7f64640ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:59.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f646a9b9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64640fff90 con 0x7f6464101300 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f646a9b9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6464100100 con 0x7f6464103c30 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f646a9b9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64641a0800 con 0x7f6464106560 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464101300 0x7f6464100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464101300 0x7f6464100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46938/0 (socket says 172.21.15.49:46938) 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 -- 172.21.15.49:0/412062576 learned_addr learned my addr 172.21.15.49:0/412062576 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f64637fe700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464103c30 0x7f64640ff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6468f56700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6464106560 0x7f64640ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 -- 172.21.15.49:0/412062576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6464106560 msgr2=0x7f64640ff7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6464106560 0x7f64640ff7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 -- 172.21.15.49:0/412062576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464103c30 msgr2=0x7f64640ff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464103c30 0x7f64640ff2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 -- 172.21.15.49:0/412062576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580095f0 con 0x7f6464101300 2023-10-08T09:07:59.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6468f56700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6464106560 0x7f64640ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:07:59.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f6463fff700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464101300 0x7f6464100c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f645400d950 tx=0x7f645400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:59.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f64617fa700 1 -- 172.21.15.49:0/412062576 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454009940 con 0x7f6464101300 2023-10-08T09:07:59.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64641a0a90 con 0x7f6464101300 2023-10-08T09:07:59.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f64617fa700 1 -- 172.21.15.49:0/412062576 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6454010460 con 0x7f6464101300 2023-10-08T09:07:59.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f64617fa700 1 -- 172.21.15.49:0/412062576 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645400f620 con 0x7f6464101300 2023-10-08T09:07:59.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.212+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64641a0f90 con 0x7f6464101300 2023-10-08T09:07:59.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.213+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64640fc870 con 0x7f6464101300 2023-10-08T09:07:59.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.214+0000 7f64617fa700 1 -- 172.21.15.49:0/412062576 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f64540108d0 con 0x7f6464101300 2023-10-08T09:07:59.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.215+0000 7f64617fa700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f644c0a13f0 0x7f644c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:07:59.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.215+0000 7f64637fe700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f644c0a13f0 0x7f644c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:07:59.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.215+0000 7f64617fa700 1 -- 172.21.15.49:0/412062576 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f64540caf20 con 0x7f6464101300 2023-10-08T09:07:59.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.216+0000 7f64637fe700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f644c0a13f0 0x7f644c0a38b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f645800b5c0 tx=0x7f6458003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:07:59.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.219+0000 7f64617fa700 1 -- 172.21.15.49:0/412062576 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f64540961b0 con 0x7f6464101300 2023-10-08T09:07:59.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.427+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f646410c330 con 0x7f6464101300 2023-10-08T09:07:59.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.431+0000 7f64617fa700 1 -- 172.21.15.49:0/412062576 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v30)=0 v30) v1 ==== 121+0+0 (secure 0 0 0) 0x7f64540200c0 con 0x7f6464101300 2023-10-08T09:07:59.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f644c0a13f0 msgr2=0x7f644c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f644c0a13f0 0x7f644c0a38b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f645800b5c0 tx=0x7f6458003790 comp rx=0 tx=0).stop 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464101300 msgr2=0x7f6464100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464101300 0x7f6464100c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f645400d950 tx=0x7f645400dc60 comp rx=0 tx=0).stop 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 shutdown_connections 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f644c0a13f0 0x7f644c0a38b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6464106560 0x7f64640ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6464101300 0x7f6464100c10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 --2- 172.21.15.49:0/412062576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6464103c30 0x7f64640ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:07:59.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 >> 172.21.15.49:0/412062576 conn(0x7f64640facf0 msgr2=0x7f6464101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:07:59.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 shutdown_connections 2023-10-08T09:07:59.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:07:59.433+0000 7f646a9b9700 1 -- 172.21.15.49:0/412062576 wait complete. 2023-10-08T09:07:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:59 smithi049 ceph-mon[99485]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:59 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3360654087' entity='client.admin' 2023-10-08T09:07:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:07:59 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/412062576' entity='client.admin' 2023-10-08T09:07:59.941 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi049') 2023-10-08T09:07:59.942 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi049') 2023-10-08T09:07:59.942 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi049') 2023-10-08T09:07:59.942 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi049') 2023-10-08T09:07:59.942 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi049') 2023-10-08T09:07:59.942 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'b8694e94-65b9-11ee-8db5-212e2dc638e7', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi049') 2023-10-08T09:07:59.942 INFO:tasks.cephadm:Deploying OSDs... 2023-10-08T09:07:59.943 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:07:59.943 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T09:07:59.961 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T09:07:59.962 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2023-10-08T09:07:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:59 smithi184 ceph-mon[103675]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:07:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:59 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3360654087' entity='client.admin' 2023-10-08T09:07:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:07:59 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/412062576' entity='client.admin' 2023-10-08T09:08:00.019 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T09:08:00.021 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:00.021 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 394468 Links: 1 2023-10-08T09:08:00.021 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:00.021 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:00.021 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-10-08 09:06:47.499873191 +0000 2023-10-08T09:08:00.021 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-10-08 09:00:56.271793191 +0000 2023-10-08T09:08:00.021 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-10-08 09:00:56.271793191 +0000 2023-10-08T09:08:00.022 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-10-08T09:08:00.022 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T09:08:00.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:59 smithi132 ceph-mon[103389]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:00.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:59 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3360654087' entity='client.admin' 2023-10-08T09:08:00.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:07:59 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/412062576' entity='client.admin' 2023-10-08T09:08:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-10-08T09:08:00.087 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-10-08T09:08:00.088 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000183683 s, 2.8 MB/s 2023-10-08T09:08:00.088 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T09:08:00.147 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 395760 Links: 1 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-10-08 09:06:47.542872014 +0000 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-10-08 09:00:56.789778929 +0000 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-10-08 09:00:56.789778929 +0000 2023-10-08T09:08:00.205 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-10-08T09:08:00.206 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T09:08:00.272 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-10-08T09:08:00.272 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-10-08T09:08:00.272 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000104101 s, 4.9 MB/s 2023-10-08T09:08:00.273 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T09:08:00.332 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2023-10-08T09:08:00.389 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 391818 Links: 1 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-10-08 09:06:47.583870892 +0000 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-10-08 09:00:57.309764611 +0000 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-10-08 09:00:57.309764611 +0000 2023-10-08T09:08:00.390 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-10-08T09:08:00.391 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T09:08:00.456 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-10-08T09:08:00.456 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-10-08T09:08:00.456 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.8447e-05 s, 5.2 MB/s 2023-10-08T09:08:00.457 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T09:08:00.515 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2023-10-08T09:08:00.572 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 391904 Links: 1 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2023-10-08 09:06:47.625869743 +0000 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2023-10-08 09:00:57.821750514 +0000 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2023-10-08 09:00:57.821750514 +0000 2023-10-08T09:08:00.573 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2023-10-08T09:08:00.574 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T09:08:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2023-10-08T09:08:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2023-10-08T09:08:00.639 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000108618 s, 4.7 MB/s 2023-10-08T09:08:00.640 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T09:08:00.699 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2023-10-08T09:08:00.699 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T09:08:00.721 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T09:08:00.721 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2023-10-08T09:08:00.781 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T09:08:00.781 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:00.782 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 386627 Links: 1 2023-10-08T09:08:00.782 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:00.782 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:00.782 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-08 09:07:40.237664421 +0000 2023-10-08T09:08:00.782 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-08 09:00:40.208570242 +0000 2023-10-08T09:08:00.782 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-08 09:00:40.208570242 +0000 2023-10-08T09:08:00.782 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-08T09:08:00.783 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T09:08:00.855 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-08T09:08:00.855 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-08T09:08:00.856 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000262657 s, 1.9 MB/s 2023-10-08T09:08:00.857 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T09:08:00.917 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 386705 Links: 1 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-08 09:07:40.237664421 +0000 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-08 09:00:40.714557428 +0000 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-08 09:00:40.714557428 +0000 2023-10-08T09:08:00.976 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-08T09:08:00.977 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T09:08:01.043 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-08T09:08:01.044 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-08T09:08:01.044 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000243576 s, 2.1 MB/s 2023-10-08T09:08:01.045 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T09:08:01.103 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2023-10-08T09:08:01.160 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 389105 Links: 1 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-08 09:07:40.237664421 +0000 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-08 09:00:41.228544413 +0000 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-08 09:00:41.228544413 +0000 2023-10-08T09:08:01.161 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-08T09:08:01.161 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T09:08:01.226 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-08T09:08:01.227 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-08T09:08:01.227 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00022665 s, 2.3 MB/s 2023-10-08T09:08:01.228 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T09:08:01.286 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2023-10-08T09:08:01.343 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T09:08:01.343 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:01.344 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 6h/6d Inode: 386900 Links: 1 2023-10-08T09:08:01.344 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:01.344 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:01.344 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2023-10-08 09:07:40.237664421 +0000 2023-10-08T09:08:01.344 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2023-10-08 09:00:41.738531498 +0000 2023-10-08T09:08:01.344 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2023-10-08 09:00:41.738531498 +0000 2023-10-08T09:08:01.344 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2023-10-08T09:08:01.344 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T09:08:01.409 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2023-10-08T09:08:01.409 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2023-10-08T09:08:01.409 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.00017483 s, 2.9 MB/s 2023-10-08T09:08:01.410 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T09:08:01.468 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T09:08:01.468 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T09:08:01.490 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T09:08:01.490 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 386987 Links: 1 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 09:07:38.109045107 +0000 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 09:00:39.131428987 +0000 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 09:00:39.131428987 +0000 2023-10-08T09:08:01.551 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T09:08:01.552 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T09:08:01.623 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T09:08:01.624 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T09:08:01.624 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000259372 s, 2.0 MB/s 2023-10-08T09:08:01.625 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T09:08:01.686 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2023-10-08T09:08:01.744 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T09:08:01.745 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:01.745 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 395306 Links: 1 2023-10-08T09:08:01.745 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:01.745 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:01.745 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 09:07:38.109045107 +0000 2023-10-08T09:08:01.746 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 09:00:39.635418684 +0000 2023-10-08T09:08:01.746 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 09:00:39.635418684 +0000 2023-10-08T09:08:01.746 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T09:08:01.746 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T09:08:01.813 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T09:08:01.813 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T09:08:01.813 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000240659 s, 2.1 MB/s 2023-10-08T09:08:01.814 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T09:08:01.874 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2023-10-08T09:08:01.932 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T09:08:01.932 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:01.932 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 383809 Links: 1 2023-10-08T09:08:01.932 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:01.932 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:01.932 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 09:07:38.109045107 +0000 2023-10-08T09:08:01.932 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 09:00:40.147408218 +0000 2023-10-08T09:08:01.933 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 09:00:40.147408218 +0000 2023-10-08T09:08:01.933 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T09:08:01.933 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T09:08:01.996 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:01 smithi184 ceph-mon[103675]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:01.998 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T09:08:01.998 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T09:08:01.998 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000220637 s, 2.3 MB/s 2023-10-08T09:08:01.999 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T09:08:02.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:01 smithi132 ceph-mon[103389]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:02.057 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2023-10-08T09:08:02.114 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 6h/6d Inode: 394945 Links: 1 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 09:07:38.109045107 +0000 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 09:00:40.669397548 +0000 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 09:00:40.669397548 +0000 2023-10-08T09:08:02.115 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T09:08:02.115 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T09:08:02.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:01 smithi049 ceph-mon[99485]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:02.181 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T09:08:02.181 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T09:08:02.181 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000220097 s, 2.3 MB/s 2023-10-08T09:08:02.182 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T09:08:02.240 INFO:tasks.cephadm:Deploying osd.0 on smithi049 with /dev/vg_nvme/lv_4... 2023-10-08T09:08:02.240 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T09:08:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:08:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:03 smithi049 ceph-mon[99485]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:03.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:03 smithi184 ceph-mon[103675]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:04.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:03 smithi132 ceph-mon[103389]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:04.695 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T09:08:04.695 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T09:08:04.695 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2023-10-08T09:08:04.695 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2023-10-08T09:08:04.695 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00819586 s, 1.3 GB/s 2023-10-08T09:08:04.695 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2023-10-08T09:08:05.079 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi049:vg_nvme/lv_4 2023-10-08T09:08:05.644 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:08:05.922 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:05 smithi049 ceph-mon[99485]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:05 smithi184 ceph-mon[103675]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:05 smithi132 ceph-mon[103389]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:07.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 -- 172.21.15.49:0/2361023267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c1012d0 msgr2=0x7fd58c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 --2- 172.21.15.49:0/2361023267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c1012d0 0x7fd58c107b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd58400b0d0 tx=0x7fd58400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 -- 172.21.15.49:0/2361023267 shutdown_connections 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 --2- 172.21.15.49:0/2361023267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c1012d0 0x7fd58c107b80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 --2- 172.21.15.49:0/2361023267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd58c0ff730 0x7fd58c0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 --2- 172.21.15.49:0/2361023267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd58c100930 0x7fd58c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 -- 172.21.15.49:0/2361023267 >> 172.21.15.49:0/2361023267 conn(0x7fd58c0facf0 msgr2=0x7fd58c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 -- 172.21.15.49:0/2361023267 shutdown_connections 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.011+0000 7fd592737700 1 -- 172.21.15.49:0/2361023267 wait complete. 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.012+0000 7fd592737700 1 Processor -- start 2023-10-08T09:08:07.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.012+0000 7fd592737700 1 -- start start 2023-10-08T09:08:07.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd592737700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd58c0ff730 0x7fd58c068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:07.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd592737700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c100930 0x7fd58c19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:07.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd592737700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd58c1012d0 0x7fd58c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd592737700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c19e2e0 con 0x7fd58c100930 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd592737700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c19e450 con 0x7fd58c1012d0 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd592737700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c19e610 con 0x7fd58c0ff730 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c100930 0x7fd58c19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c100930 0x7fd58c19b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46954/0 (socket says 172.21.15.49:46954) 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 -- 172.21.15.49:0/3994951634 learned_addr learned my addr 172.21.15.49:0/3994951634 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58bfff700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd58c0ff730 0x7fd58c068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 -- 172.21.15.49:0/3994951634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd58c0ff730 msgr2=0x7fd58c068c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd58c0ff730 0x7fd58c068c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd590cd4700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd58c1012d0 0x7fd58c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:07.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 -- 172.21.15.49:0/3994951634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd58c1012d0 msgr2=0x7fd58c19db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd58c1012d0 0x7fd58c19db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 -- 172.21.15.49:0/3994951634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd584009d00 con 0x7fd58c100930 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58bfff700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd58c0ff730 0x7fd58c068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd590cd4700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd58c1012d0 0x7fd58c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd58b7fe700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c100930 0x7fd58c19b5b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b530 tx=0x7fd58000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd5897fa700 1 -- 172.21.15.49:0/3994951634 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580011830 con 0x7fd58c100930 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.013+0000 7fd5897fa700 1 -- 172.21.15.49:0/3994951634 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd580011e50 con 0x7fd58c100930 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.014+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c19e8f0 con 0x7fd58c100930 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.014+0000 7fd5897fa700 1 -- 172.21.15.49:0/3994951634 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58000f2e0 con 0x7fd58c100930 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.014+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c19edf0 con 0x7fd58c100930 2023-10-08T09:08:07.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.014+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c106af0 con 0x7fd58c100930 2023-10-08T09:08:07.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.016+0000 7fd5897fa700 1 -- 172.21.15.49:0/3994951634 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fd580010580 con 0x7fd58c100930 2023-10-08T09:08:07.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.017+0000 7fd5897fa700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5740a13f0 0x7fd5740a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:07.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.017+0000 7fd58bfff700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5740a13f0 0x7fd5740a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:07.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.017+0000 7fd5897fa700 1 -- 172.21.15.49:0/3994951634 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd5800cb130 con 0x7fd58c100930 2023-10-08T09:08:07.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.017+0000 7fd58bfff700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5740a13f0 0x7fd5740a38b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd58c100790 tx=0x7fd57c00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:08:07.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.021+0000 7fd5897fa700 1 -- 172.21.15.49:0/3994951634 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5800963c0 con 0x7fd58c100930 2023-10-08T09:08:07.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:07.228+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd58c02cf80 con 0x7fd5740a13f0 2023-10-08T09:08:07.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:07 smithi184 ceph-mon[103675]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:07.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:07 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:07.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:07 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:07.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:07 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:07 smithi132 ceph-mon[103389]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:07 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:07 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:07 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:08.057 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:07 smithi049 ceph-mon[99485]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:08.057 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:07 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:08.057 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:07 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:08.057 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:07 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:08.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:08 smithi049 ceph-mon[99485]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:08.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:08 smithi184 ceph-mon[103675]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:09.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:08 smithi132 ceph-mon[103389]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:09 smithi184 ceph-mon[103675]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:09 smithi132 ceph-mon[103389]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:09 smithi049 ceph-mon[99485]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:10 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3572436294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78f3f71f-7b0b-4d29-997d-3bca2fd04fd3"}]: dispatch 2023-10-08T09:08:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:10 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3572436294' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78f3f71f-7b0b-4d29-997d-3bca2fd04fd3"}]': finished 2023-10-08T09:08:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:10 smithi184 ceph-mon[103675]: osdmap e5: 1 total, 0 up, 1 in 2023-10-08T09:08:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:10 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:10 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4011951089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:10 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3572436294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78f3f71f-7b0b-4d29-997d-3bca2fd04fd3"}]: dispatch 2023-10-08T09:08:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:10 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3572436294' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78f3f71f-7b0b-4d29-997d-3bca2fd04fd3"}]': finished 2023-10-08T09:08:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:10 smithi132 ceph-mon[103389]: osdmap e5: 1 total, 0 up, 1 in 2023-10-08T09:08:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:10 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:10 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4011951089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3572436294' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78f3f71f-7b0b-4d29-997d-3bca2fd04fd3"}]: dispatch 2023-10-08T09:08:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3572436294' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78f3f71f-7b0b-4d29-997d-3bca2fd04fd3"}]': finished 2023-10-08T09:08:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:10 smithi049 ceph-mon[99485]: osdmap e5: 1 total, 0 up, 1 in 2023-10-08T09:08:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4011951089' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:11 smithi184 ceph-mon[103675]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:11 smithi132 ceph-mon[103389]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:11 smithi049 ceph-mon[99485]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:13.860 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:13 smithi049 ceph-mon[99485]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:13 smithi184 ceph-mon[103675]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:13 smithi132 ceph-mon[103389]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:15 smithi184 ceph-mon[103675]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:15 smithi132 ceph-mon[103389]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:16.092 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:15 smithi049 ceph-mon[99485]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:17 smithi049 ceph-mon[99485]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:17 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T09:08:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:17 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:17 smithi049 ceph-mon[99485]: Deploying daemon osd.0 on smithi049 2023-10-08T09:08:17.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:17 smithi184 ceph-mon[103675]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:17.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:17 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T09:08:17.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:17 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:17.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:17 smithi184 ceph-mon[103675]: Deploying daemon osd.0 on smithi049 2023-10-08T09:08:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:17 smithi132 ceph-mon[103389]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:17 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-08T09:08:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:17 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:17 smithi132 ceph-mon[103389]: Deploying daemon osd.0 on smithi049 2023-10-08T09:08:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:19 smithi049 ceph-mon[99485]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:19 smithi184 ceph-mon[103675]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:19 smithi132 ceph-mon[103389]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:21.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:21 smithi049 ceph-mon[99485]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:21.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:21.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:21.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:21 smithi184 ceph-mon[103675]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:21.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:21 smithi132 ceph-mon[103389]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:23.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:23 smithi049 ceph-mon[99485]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:23.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:23 smithi184 ceph-mon[103675]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:23.785 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 0 on host 'smithi049' 2023-10-08T09:08:23.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.784+0000 7fd5897fa700 1 -- 172.21.15.49:0/3994951634 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd58c02cf80 con 0x7fd5740a13f0 2023-10-08T09:08:23.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5740a13f0 msgr2=0x7fd5740a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:23.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5740a13f0 0x7fd5740a38b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd58c100790 tx=0x7fd57c00a680 comp rx=0 tx=0).stop 2023-10-08T09:08:23.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c100930 msgr2=0x7fd58c19b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:23.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c100930 0x7fd58c19b5b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b530 tx=0x7fd58000b8f0 comp rx=0 tx=0).stop 2023-10-08T09:08:23.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 shutdown_connections 2023-10-08T09:08:23.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5740a13f0 0x7fd5740a38b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:23.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd58c0ff730 0x7fd58c068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:23.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd58c1012d0 0x7fd58c19db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:23.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 --2- 172.21.15.49:0/3994951634 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd58c100930 0x7fd58c19b5b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:23.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.786+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 >> 172.21.15.49:0/3994951634 conn(0x7fd58c0facf0 msgr2=0x7fd58c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:08:23.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.787+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 shutdown_connections 2023-10-08T09:08:23.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:08:23.787+0000 7fd592737700 1 -- 172.21.15.49:0/3994951634 wait complete. 2023-10-08T09:08:23.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:23 smithi132 ceph-mon[103389]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:24.252 DEBUG:teuthology.orchestra.run.smithi049:osd.0> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.0.service 2023-10-08T09:08:24.254 INFO:tasks.cephadm:Deploying osd.1 on smithi132 with /dev/vg_nvme/lv_4... 2023-10-08T09:08:24.255 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T09:08:24.294 INFO:journalctl@ceph.osd.0.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:24.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:24.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:24.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:24.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:24.819 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:08:25.340 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:08:25 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-0[121940]: 2023-10-08T09:08:25.049+0000 7fc3c4ce3640 -1 osd.0 0 log_to_monitors true 2023-10-08T09:08:25.596 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:25 smithi132 ceph-mon[103389]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:25.596 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:25 smithi132 ceph-mon[103389]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T09:08:25.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:25 smithi049 ceph-mon[99485]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:25.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:25 smithi049 ceph-mon[99485]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T09:08:25.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:25 smithi184 ceph-mon[103675]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:25.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:25 smithi184 ceph-mon[103675]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-08T09:08:26.318 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T09:08:26.319 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T09:08:26.319 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-10-08T09:08:26.319 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-10-08T09:08:26.319 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110366 s, 950 MB/s 2023-10-08T09:08:26.319 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-10-08T09:08:26.674 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi132:vg_nvme/lv_4 2023-10-08T09:08:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:26 smithi132 ceph-mon[103389]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T09:08:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:26 smithi132 ceph-mon[103389]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T09:08:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:26 smithi132 ceph-mon[103389]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:08:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:26 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:26 smithi132 ceph-mon[103389]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:27.132 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:08:26 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-0[121940]: 2023-10-08T09:08:26.794+0000 7fc3ba35a700 -1 osd.0 0 waiting for initial osdmap 2023-10-08T09:08:27.132 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:08:26 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-0[121940]: 2023-10-08T09:08:26.797+0000 7fc3b614f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:08:27.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:26 smithi049 ceph-mon[99485]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T09:08:27.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:26 smithi049 ceph-mon[99485]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T09:08:27.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:26 smithi049 ceph-mon[99485]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:08:27.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:26 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:27.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:26 smithi049 ceph-mon[99485]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:27.232 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:08:27.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:26 smithi184 ceph-mon[103675]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-08T09:08:27.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:26 smithi184 ceph-mon[103675]: osdmap e6: 1 total, 0 up, 1 in 2023-10-08T09:08:27.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:26 smithi184 ceph-mon[103675]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:08:27.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:26 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:27.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:26 smithi184 ceph-mon[103675]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi049 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:08:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi049 to 17168M 2023-10-08T09:08:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi049 2023-10-08T09:08:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:08:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T09:08:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:08:28.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi049 to 17168M 2023-10-08T09:08:28.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:28.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:28.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi049 2023-10-08T09:08:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: osdmap e7: 1 total, 0 up, 1 in 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi049 to 17168M 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 -- 172.21.15.132:0/324799096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990101a40 msgr2=0x7ff990101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 --2- 172.21.15.132:0/324799096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990101a40 0x7ff990101ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9800099c0 tx=0x7ff980009cd0 comp rx=0 tx=0).stop 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 -- 172.21.15.132:0/324799096 shutdown_connections 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 --2- 172.21.15.132:0/324799096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9901023e0 0x7ff990108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 --2- 172.21.15.132:0/324799096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff990100840 0x7ff990100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 --2- 172.21.15.132:0/324799096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990101a40 0x7ff990101ea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 -- 172.21.15.132:0/324799096 >> 172.21.15.132:0/324799096 conn(0x7ff9900fbe00 msgr2=0x7ff9900fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 -- 172.21.15.132:0/324799096 shutdown_connections 2023-10-08T09:08:28.444 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 -- 172.21.15.132:0/324799096 wait complete. 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 Processor -- start 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.441+0000 7ff997f47700 1 -- start start 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff997f47700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990100840 0x7ff990196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff997f47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff990101a40 0x7ff990196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff997f47700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9901023e0 0x7ff99019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff997f47700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019e2b0 con 0x7ff990101a40 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff997f47700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019e3f0 con 0x7ff990100840 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff997f47700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99019e580 con 0x7ff9901023e0 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990100840 0x7ff990196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990100840 0x7ff990196770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48820/0 (socket says 172.21.15.132:48820) 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 -- 172.21.15.132:0/1196411904 learned_addr learned my addr 172.21.15.132:0/1196411904 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-08T09:08:28.445 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff9964e4700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9901023e0 0x7ff99019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff9954e2700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff990101a40 0x7ff990196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 -- 172.21.15.132:0/1196411904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9901023e0 msgr2=0x7ff99019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9901023e0 0x7ff99019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 -- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff990101a40 msgr2=0x7ff990196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff990101a40 0x7ff990196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 -- 172.21.15.132:0/1196411904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800095f0 con 0x7ff990100840 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff995ce3700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990100840 0x7ff990196770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d980 tx=0x7ff98c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff986ffd700 1 -- 172.21.15.132:0/1196411904 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c004020 con 0x7ff990100840 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff986ffd700 1 -- 172.21.15.132:0/1196411904 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98c009730 con 0x7ff990100840 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99019e860 con 0x7ff990100840 2023-10-08T09:08:28.446 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.442+0000 7ff986ffd700 1 -- 172.21.15.132:0/1196411904 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c003b20 con 0x7ff990100840 2023-10-08T09:08:28.447 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.443+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99019edb0 con 0x7ff990100840 2023-10-08T09:08:28.447 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.443+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff990107c70 con 0x7ff990100840 2023-10-08T09:08:28.447 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.444+0000 7ff986ffd700 1 -- 172.21.15.132:0/1196411904 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7ff98c0098a0 con 0x7ff990100840 2023-10-08T09:08:28.448 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.445+0000 7ff986ffd700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff97c0a1470 0x7ff97c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:28.449 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.445+0000 7ff986ffd700 1 -- 172.21.15.132:0/1196411904 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7ff98c0cae90 con 0x7ff990100840 2023-10-08T09:08:28.449 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.445+0000 7ff9954e2700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff97c0a1470 0x7ff97c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:28.449 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.446+0000 7ff9954e2700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff97c0a1470 0x7ff97c0a3930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff980000c00 tx=0x7ff980005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:08:28.450 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.449+0000 7ff986ffd700 1 -- 172.21.15.132:0/1196411904 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff98c095f60 con 0x7ff990100840 2023-10-08T09:08:28.645 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:28.644+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff99002cf80 con 0x7ff97c0a1470 2023-10-08T09:08:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:08:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:08:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] boot 2023-10-08T09:08:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: osdmap e8: 1 total, 1 up, 1 in 2023-10-08T09:08:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:29.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:29.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:29.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:29.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:28 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:08:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:08:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] boot 2023-10-08T09:08:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: osdmap e8: 1 total, 1 up, 1 in 2023-10-08T09:08:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:29.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:29.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:29.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:29.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:28 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:29.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:08:29.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:08:29.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-08T09:08:29.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:29.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: osd.0 [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] boot 2023-10-08T09:08:29.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: osdmap e8: 1 total, 1 up, 1 in 2023-10-08T09:08:29.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-08T09:08:29.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:29.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:29.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:29.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:28 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:30.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:29 smithi132 ceph-mon[103389]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T09:08:30.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:29 smithi049 ceph-mon[99485]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T09:08:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:29 smithi184 ceph-mon[103675]: osdmap e9: 1 total, 1 up, 1 in 2023-10-08T09:08:31.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:30 smithi049 ceph-mon[99485]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:31.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:30 smithi184 ceph-mon[103675]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:30 smithi132 ceph-mon[103389]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/3123534328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]: dispatch 2023-10-08T09:08:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:31 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]: dispatch 2023-10-08T09:08:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:31 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]': finished 2023-10-08T09:08:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:31 smithi049 ceph-mon[99485]: osdmap e10: 2 total, 1 up, 2 in 2023-10-08T09:08:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:31 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/1362781412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/3123534328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]: dispatch 2023-10-08T09:08:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:31 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]: dispatch 2023-10-08T09:08:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:31 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]': finished 2023-10-08T09:08:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:31 smithi184 ceph-mon[103675]: osdmap e10: 2 total, 1 up, 2 in 2023-10-08T09:08:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:31 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:32.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/1362781412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/3123534328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]: dispatch 2023-10-08T09:08:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:31 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]: dispatch 2023-10-08T09:08:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:31 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f3cdae9-b298-43b7-a0d0-703f6f7ab094"}]': finished 2023-10-08T09:08:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:31 smithi132 ceph-mon[103389]: osdmap e10: 2 total, 1 up, 2 in 2023-10-08T09:08:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/1362781412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:33 smithi049 ceph-mon[99485]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:33 smithi184 ceph-mon[103675]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:33 smithi132 ceph-mon[103389]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:35.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:35 smithi132 ceph-mon[103389]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:35.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:35 smithi049 ceph-mon[99485]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:35 smithi184 ceph-mon[103675]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:37.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:37 smithi132 ceph-mon[103389]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:37.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:37 smithi049 ceph-mon[99485]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:37 smithi184 ceph-mon[103675]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:38.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T09:08:38.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:38.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T09:08:38.703 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-08T09:08:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:39.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:39 smithi132 ceph-mon[103389]: Deploying daemon osd.1 on smithi132 2023-10-08T09:08:39.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:39 smithi132 ceph-mon[103389]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:39.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:39 smithi049 ceph-mon[99485]: Deploying daemon osd.1 on smithi132 2023-10-08T09:08:39.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:39 smithi049 ceph-mon[99485]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:39 smithi184 ceph-mon[103675]: Deploying daemon osd.1 on smithi132 2023-10-08T09:08:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:39 smithi184 ceph-mon[103675]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:41.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:41 smithi132 ceph-mon[103389]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:41 smithi049 ceph-mon[99485]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:41 smithi184 ceph-mon[103675]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:43.394 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:43 smithi132 ceph-mon[103389]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:43.394 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:43.394 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:43.395 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:43.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:43 smithi184 ceph-mon[103675]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:43.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:43.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:43.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:43.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:43 smithi049 ceph-mon[99485]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:43.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:43.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:43.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:44.908 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.907+0000 7ff986ffd700 1 -- 172.21.15.132:0/1196411904 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff99002cf80 con 0x7ff97c0a1470 2023-10-08T09:08:44.909 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 1 on host 'smithi132' 2023-10-08T09:08:44.910 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff97c0a1470 msgr2=0x7ff97c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:44.910 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff97c0a1470 0x7ff97c0a3930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff980000c00 tx=0x7ff980005fb0 comp rx=0 tx=0).stop 2023-10-08T09:08:44.910 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990100840 msgr2=0x7ff990196770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:44.910 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990100840 0x7ff990196770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d980 tx=0x7ff98c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 shutdown_connections 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff97c0a1470 0x7ff97c0a3930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9901023e0 0x7ff99019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff990100840 0x7ff990196770 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 --2- 172.21.15.132:0/1196411904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff990101a40 0x7ff990196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 >> 172.21.15.132:0/1196411904 conn(0x7ff9900fbe00 msgr2=0x7ff990104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 shutdown_connections 2023-10-08T09:08:44.911 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:08:44.910+0000 7ff997f47700 1 -- 172.21.15.132:0/1196411904 wait complete. 2023-10-08T09:08:45.430 DEBUG:teuthology.orchestra.run.smithi132:osd.1> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.1.service 2023-10-08T09:08:45.432 INFO:tasks.cephadm:Deploying osd.2 on smithi184 with /dev/vg_nvme/lv_4... 2023-10-08T09:08:45.432 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-08T09:08:45.470 INFO:journalctl@ceph.osd.1.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:08:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:45.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:45.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:45.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:08:45.999 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:08:46.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:46 smithi132 ceph-mon[103389]: from='osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:08:46.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:46 smithi132 ceph-mon[103389]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:08:46.548 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:08:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-1[108441]: 2023-10-08T09:08:46.193+0000 7f266cdc9640 -1 osd.1 0 log_to_monitors true 2023-10-08T09:08:46.653 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:46 smithi184 ceph-mon[103675]: from='osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:08:46.653 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:46 smithi184 ceph-mon[103675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:08:46.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:46 smithi049 ceph-mon[99485]: from='osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:08:46.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:46 smithi049 ceph-mon[99485]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-08T09:08:47.598 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-08T09:08:47.599 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-08T09:08:47.599 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-10-08T09:08:47.599 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-10-08T09:08:47.599 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129761 s, 808 MB/s 2023-10-08T09:08:47.599 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-10-08T09:08:47.728 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:47 smithi184 ceph-mon[103675]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:47.728 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:47 smithi184 ceph-mon[103675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T09:08:47.728 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:47 smithi184 ceph-mon[103675]: from='osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:08:47.728 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:47 smithi184 ceph-mon[103675]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T09:08:47.728 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:47.728 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:47 smithi184 ceph-mon[103675]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:08:47.797 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-1[108441]: 2023-10-08T09:08:47.415+0000 7f2662440700 -1 osd.1 0 waiting for initial osdmap 2023-10-08T09:08:47.797 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-1[108441]: 2023-10-08T09:08:47.421+0000 7f265da34700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:08:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-mon[103389]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-mon[103389]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T09:08:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-mon[103389]: from='osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:08:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-mon[103389]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T09:08:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:47 smithi132 ceph-mon[103389]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:08:47.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:47 smithi049 ceph-mon[99485]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:47.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:47 smithi049 ceph-mon[99485]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-08T09:08:47.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:47 smithi049 ceph-mon[99485]: from='osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:08:47.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:47 smithi049 ceph-mon[99485]: osdmap e11: 2 total, 1 up, 2 in 2023-10-08T09:08:47.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:47.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:47 smithi049 ceph-mon[99485]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:08:47.943 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi184:vg_nvme/lv_4 2023-10-08T09:08:48.507 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:08:48.675 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.676 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:48.677 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:48.677 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:08:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:48.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:48.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: osdmap e12: 2 total, 1 up, 2 in 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:08:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:08:48.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:08:49.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.721+0000 7fc4c67ee700 1 -- 172.21.15.184:0/4063812331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c00ff730 msgr2=0x7fc4c00ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:49.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.721+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/4063812331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c00ff730 0x7fc4c00ffb50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009990 tx=0x7fc4b0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:08:49.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 -- 172.21.15.184:0/4063812331 shutdown_connections 2023-10-08T09:08:49.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/4063812331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c00ff730 0x7fc4c00ffb50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:49.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/4063812331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c01012d0 0x7fc4c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:49.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/4063812331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c0100930 0x7fc4c0100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:49.725 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 -- 172.21.15.184:0/4063812331 >> 172.21.15.184:0/4063812331 conn(0x7fc4c00facf0 msgr2=0x7fc4c00fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 -- 172.21.15.184:0/4063812331 shutdown_connections 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 -- 172.21.15.184:0/4063812331 wait complete. 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 Processor -- start 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 -- start start 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c00ff730 0x7fc4c0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c0100930 0x7fc4c0198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.722+0000 7fc4c67ee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c01012d0 0x7fc4c019ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4c67ee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01a0540 con 0x7fc4c01012d0 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4c67ee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01a0680 con 0x7fc4c00ff730 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4c67ee700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c01a0810 con 0x7fc4c0100930 2023-10-08T09:08:49.726 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c0100930 0x7fc4c0198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c00ff730 0x7fc4c0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c0100930 0x7fc4c0198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59490/0 (socket says 172.21.15.184:59490) 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 -- 172.21.15.184:0/3469729839 learned_addr learned my addr 172.21.15.184:0/3469729839 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c00ff730 0x7fc4c0198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.184:49342/0 (socket says 172.21.15.184:49342) 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4c4d8b700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c01012d0 0x7fc4c019ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 -- 172.21.15.184:0/3469729839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c00ff730 msgr2=0x7fc4c0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c00ff730 0x7fc4c0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 -- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c01012d0 msgr2=0x7fc4c019ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c01012d0 0x7fc4c019ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 -- 172.21.15.184:0/3469729839 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b00095f0 con 0x7fc4c0100930 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bf7fe700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c0100930 0x7fc4c0198f40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400b560 tx=0x7fc4b400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:08:49.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bd7fa700 1 -- 172.21.15.184:0/3469729839 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4011830 con 0x7fc4c0100930 2023-10-08T09:08:49.728 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bd7fa700 1 -- 172.21.15.184:0/3469729839 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4b4011e50 con 0x7fc4c0100930 2023-10-08T09:08:49.728 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4bd7fa700 1 -- 172.21.15.184:0/3469729839 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4010e50 con 0x7fc4c0100930 2023-10-08T09:08:49.728 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c01a0af0 con 0x7fc4c0100930 2023-10-08T09:08:49.728 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.723+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c01a0ff0 con 0x7fc4c0100930 2023-10-08T09:08:49.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.724+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c004e4c0 con 0x7fc4c0100930 2023-10-08T09:08:49.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.725+0000 7fc4bd7fa700 1 -- 172.21.15.184:0/3469729839 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fc4b40104f0 con 0x7fc4c0100930 2023-10-08T09:08:49.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.726+0000 7fc4bd7fa700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4a80a1470 0x7fc4a80a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:08:49.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.726+0000 7fc4bd7fa700 1 -- 172.21.15.184:0/3469729839 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fc4b40cb370 con 0x7fc4c0100930 2023-10-08T09:08:49.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.726+0000 7fc4bffff700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4a80a1470 0x7fc4a80a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:08:49.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.727+0000 7fc4bffff700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4a80a1470 0x7fc4a80a3930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000c00 tx=0x7fc4b0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:08:49.731 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.730+0000 7fc4bd7fa700 1 -- 172.21.15.184:0/3469729839 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc4b4096220 con 0x7fc4c0100930 2023-10-08T09:08:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:08:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:08:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi132 2023-10-08T09:08:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi132 to 17168M 2023-10-08T09:08:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] boot 2023-10-08T09:08:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: osdmap e13: 2 total, 2 up, 2 in 2023-10-08T09:08:49.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi132 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi132 to 17168M 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] boot 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: osdmap e13: 2 total, 2 up, 2 in 2023-10-08T09:08:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:08:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:08:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-08T09:08:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi132 2023-10-08T09:08:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi132 to 17168M 2023-10-08T09:08:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: osd.1 [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] boot 2023-10-08T09:08:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: osdmap e13: 2 total, 2 up, 2 in 2023-10-08T09:08:49.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-08T09:08:49.926 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:08:49.924+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c00fba30 con 0x7fc4a80a1470 2023-10-08T09:08:50.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:50 smithi184 ceph-mon[103675]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T09:08:50.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:50.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:50.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:50 smithi132 ceph-mon[103389]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T09:08:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:50.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:50 smithi049 ceph-mon[99485]: osdmap e14: 2 total, 2 up, 2 in 2023-10-08T09:08:50.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:08:50.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:08:50.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:51 smithi184 ceph-mon[103675]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:51 smithi184 ceph-mon[103675]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:51 smithi132 ceph-mon[103389]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:51 smithi132 ceph-mon[103389]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:51 smithi049 ceph-mon[99485]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:51 smithi049 ceph-mon[99485]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:08:52.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:52 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/1696231163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]: dispatch 2023-10-08T09:08:52.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:52 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]: dispatch 2023-10-08T09:08:52.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:52 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]': finished 2023-10-08T09:08:52.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:52 smithi184 ceph-mon[103675]: osdmap e15: 3 total, 2 up, 3 in 2023-10-08T09:08:52.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:08:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:52 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/1696231163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]: dispatch 2023-10-08T09:08:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:52 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]: dispatch 2023-10-08T09:08:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:52 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]': finished 2023-10-08T09:08:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:52 smithi132 ceph-mon[103389]: osdmap e15: 3 total, 2 up, 3 in 2023-10-08T09:08:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:08:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:52 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/1696231163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]: dispatch 2023-10-08T09:08:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:52 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]: dispatch 2023-10-08T09:08:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:52 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95cbb767-d840-4487-8b70-2f8381136ef7"}]': finished 2023-10-08T09:08:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:52 smithi049 ceph-mon[99485]: osdmap e15: 3 total, 2 up, 3 in 2023-10-08T09:08:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:08:53.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:53 smithi184 ceph-mon[103675]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:53.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:53 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/2220708342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:53.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:53 smithi132 ceph-mon[103389]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:53.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:53 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/2220708342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:53 smithi049 ceph-mon[99485]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:53 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/2220708342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:08:55.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:55 smithi184 ceph-mon[103675]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:55.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:55 smithi132 ceph-mon[103389]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:55 smithi049 ceph-mon[99485]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:57.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:57 smithi184 ceph-mon[103675]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:57.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:57 smithi132 ceph-mon[103389]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:57.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:57 smithi049 ceph-mon[99485]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:59.634 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:59 smithi184 ceph-mon[103675]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:59.634 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T09:08:59.634 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:08:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:59 smithi132 ceph-mon[103389]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T09:08:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:08:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:08:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:59 smithi049 ceph-mon[99485]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:08:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-08T09:08:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:08:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:00 smithi184 ceph-mon[103675]: Deploying daemon osd.2 on smithi184 2023-10-08T09:09:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:00 smithi132 ceph-mon[103389]: Deploying daemon osd.2 on smithi184 2023-10-08T09:09:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:00 smithi049 ceph-mon[99485]: Deploying daemon osd.2 on smithi184 2023-10-08T09:09:01.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:01 smithi184 ceph-mon[103675]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:01 smithi132 ceph-mon[103389]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:01.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:01 smithi049 ceph-mon[99485]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:03 smithi184 ceph-mon[103675]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:03 smithi132 ceph-mon[103389]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:03 smithi049 ceph-mon[99485]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:05.691 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:05 smithi184 ceph-mon[103675]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:05.691 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:05 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.692 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:05 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.692 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:05 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:05.692 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:05 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:05.692 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:05 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.739 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 2 on host 'smithi184' 2023-10-08T09:09:05.740 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.739+0000 7fc4bd7fa700 1 -- 172.21.15.184:0/3469729839 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc4c00fba30 con 0x7fc4a80a1470 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4a80a1470 msgr2=0x7fc4a80a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4a80a1470 0x7fc4a80a3930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000c00 tx=0x7fc4b0005fb0 comp rx=0 tx=0).stop 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c0100930 msgr2=0x7fc4c0198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c0100930 0x7fc4c0198f40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400b560 tx=0x7fc4b400b870 comp rx=0 tx=0).stop 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 shutdown_connections 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc4a80a1470 0x7fc4a80a3930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4c01012d0 0x7fc4c019ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:05.742 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4c00ff730 0x7fc4c0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:05.743 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 --2- 172.21.15.184:0/3469729839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4c0100930 0x7fc4c0198f40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:05.743 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.741+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 >> 172.21.15.184:0/3469729839 conn(0x7fc4c00facf0 msgr2=0x7fc4c00fb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:09:05.743 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.742+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 shutdown_connections 2023-10-08T09:09:05.743 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:05.742+0000 7fc4c67ee700 1 -- 172.21.15.184:0/3469729839 wait complete. 2023-10-08T09:09:05.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:05 smithi132 ceph-mon[103389]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:05.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:05 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:05 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:05 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:05 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:05 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:05 smithi049 ceph-mon[99485]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:06.085 DEBUG:teuthology.orchestra.run.smithi184:osd.2> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.2.service 2023-10-08T09:09:06.087 INFO:tasks.cephadm:Deploying osd.3 on smithi049 with /dev/vg_nvme/lv_3... 2023-10-08T09:09:06.087 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T09:09:06.115 INFO:journalctl@ceph.osd.2.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:09:06.671 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:09:06.933 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:06 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:06.933 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:06 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:06.933 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:06 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:06.934 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:06 smithi184 ceph-mon[103675]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:07.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:06 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:07.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:06 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:07.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:06 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:07.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:06 smithi132 ceph-mon[103389]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:07.069 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:06 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:07.070 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:06 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:07.070 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:06 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:07.070 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:06 smithi049 ceph-mon[99485]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:07.247 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-2[109416]: 2023-10-08T09:09:07.023+0000 7f1185140640 -1 osd.2 0 log_to_monitors true 2023-10-08T09:09:07.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:07 smithi184 ceph-mon[103675]: from='osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:09:07.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:07 smithi184 ceph-mon[103675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:09:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:07 smithi132 ceph-mon[103389]: from='osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:09:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:07 smithi132 ceph-mon[103389]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:09:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:07 smithi049 ceph-mon[99485]: from='osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:09:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:07 smithi049 ceph-mon[99485]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-08T09:09:08.158 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T09:09:08.158 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T09:09:08.158 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2023-10-08T09:09:08.158 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2023-10-08T09:09:08.158 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00799402 s, 1.3 GB/s 2023-10-08T09:09:08.158 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2023-10-08T09:09:08.619 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi049:vg_nvme/lv_3 2023-10-08T09:09:08.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:08 smithi049 ceph-mon[99485]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T09:09:08.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:08 smithi049 ceph-mon[99485]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T09:09:08.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:08 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:08.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:08 smithi049 ceph-mon[99485]: from='osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:09:08.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:08 smithi049 ceph-mon[99485]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:09:08.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:08 smithi049 ceph-mon[99485]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:08.997 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-2[109416]: 2023-10-08T09:09:08.741+0000 7f117a7b7700 -1 osd.2 0 waiting for initial osdmap 2023-10-08T09:09:08.997 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-2[109416]: 2023-10-08T09:09:08.745+0000 7f11745a8700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:09:08.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-mon[103675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T09:09:08.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-mon[103675]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T09:09:08.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:08.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-mon[103675]: from='osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:09:08.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-mon[103675]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:09:08.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:08 smithi184 ceph-mon[103675]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:09.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:08 smithi132 ceph-mon[103389]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-08T09:09:09.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:08 smithi132 ceph-mon[103389]: osdmap e16: 3 total, 2 up, 3 in 2023-10-08T09:09:09.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:08 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:09.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:08 smithi132 ceph-mon[103389]: from='osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:09:09.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:08 smithi132 ceph-mon[103389]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:09:09.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:08 smithi132 ceph-mon[103389]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:09.184 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:09:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi184 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:09.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi184 to 17168M 2023-10-08T09:09:09.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:09.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:09.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:09.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:09:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T09:09:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi184 2023-10-08T09:09:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi184 to 17168M 2023-10-08T09:09:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:09:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: osdmap e17: 3 total, 2 up, 3 in 2023-10-08T09:09:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi184 2023-10-08T09:09:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:10.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi184 to 17168M 2023-10-08T09:09:10.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:10.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:10.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:10.521 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 -- 172.21.15.49:0/1610711835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 msgr2=0x7f60800ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 --2- 172.21.15.49:0/1610711835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f60800ffb50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f60700099c0 tx=0x7f6070009cd0 comp rx=0 tx=0).stop 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 -- 172.21.15.49:0/1610711835 shutdown_connections 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 --2- 172.21.15.49:0/1610711835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60801012d0 0x7f6080107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 --2- 172.21.15.49:0/1610711835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f60800ffb50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 --2- 172.21.15.49:0/1610711835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6080100930 0x7f6080100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 -- 172.21.15.49:0/1610711835 >> 172.21.15.49:0/1610711835 conn(0x7f60800facf0 msgr2=0x7f60800fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 -- 172.21.15.49:0/1610711835 shutdown_connections 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.518+0000 7f60848e4700 1 -- 172.21.15.49:0/1610711835 wait complete. 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 Processor -- start 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 -- start start 2023-10-08T09:09:10.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f6080194590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6080100930 0x7f6080194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60801012d0 0x7f608019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608019c0d0 con 0x7f60800ff730 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608019c240 con 0x7f6080100930 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f60848e4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608019c400 con 0x7f60801012d0 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f607e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f6080194590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f607e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f6080194590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47020/0 (socket says 172.21.15.49:47020) 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f607e59c700 1 -- 172.21.15.49:0/3253214969 learned_addr learned my addr 172.21.15.49:0/3253214969 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f607ed9d700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60801012d0 0x7f608019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.519+0000 7f607dd9b700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6080100930 0x7f6080194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:10.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f607e59c700 1 -- 172.21.15.49:0/3253214969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60801012d0 msgr2=0x7f608019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f607e59c700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60801012d0 0x7f608019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f607e59c700 1 -- 172.21.15.49:0/3253214969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6080100930 msgr2=0x7f6080194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f607e59c700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6080100930 0x7f6080194ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f607e59c700 1 -- 172.21.15.49:0/3253214969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60700095f0 con 0x7f60800ff730 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f607e59c700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f6080194590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60700058e0 tx=0x7f607000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607001d070 con 0x7f60800ff730 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607000fe70 con 0x7f60800ff730 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700049d0 con 0x7f60800ff730 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608019c680 con 0x7f60800ff730 2023-10-08T09:09:10.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.520+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608019cad0 con 0x7f60800ff730 2023-10-08T09:09:10.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.521+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608004e4c0 con 0x7f60800ff730 2023-10-08T09:09:10.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.522+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f6070004b90 con 0x7f60800ff730 2023-10-08T09:09:10.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.523+0000 7f606f7fe700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60680a13f0 0x7f60680a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:10.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.523+0000 7f607dd9b700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60680a13f0 0x7f60680a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:10.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.523+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f60700cc800 con 0x7f60800ff730 2023-10-08T09:09:10.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.523+0000 7f607dd9b700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60680a13f0 0x7f60680a38b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6074009c60 tx=0x7f6074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:09:10.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.527+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6070097520 con 0x7f60800ff730 2023-10-08T09:09:10.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:10.737+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f608019ce00 con 0x7f60680a13f0 2023-10-08T09:09:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:09:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:09:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] boot 2023-10-08T09:09:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: osdmap e18: 3 total, 3 up, 3 in 2023-10-08T09:09:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:11.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:09:11.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:11.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:11.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:10 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] boot 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: osdmap e18: 3 total, 3 up, 3 in 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:09:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:11.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:11.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:10 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:09:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:09:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: osd.2 [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] boot 2023-10-08T09:09:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: osdmap e18: 3 total, 3 up, 3 in 2023-10-08T09:09:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-08T09:09:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-08T09:09:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:09:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:10 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:11.865 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:11 smithi049 ceph-mon[99485]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:11.865 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T09:09:11.865 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:11 smithi049 ceph-mon[99485]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T09:09:11.865 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:09:12.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:11 smithi132 ceph-mon[103389]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:12.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T09:09:12.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:11 smithi132 ceph-mon[103389]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T09:09:12.051 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:09:12.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:11 smithi184 ceph-mon[103675]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:12.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-08T09:09:12.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:11 smithi184 ceph-mon[103675]: osdmap e19: 3 total, 3 up, 3 in 2023-10-08T09:09:12.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:09:12.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127705]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:09:12.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:12.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127705]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:12.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127705]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:12.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:12.774+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f60700978f0 con 0x7f60800ff730 2023-10-08T09:09:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:09:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:12 smithi132 ceph-mon[103389]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T09:09:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:12 smithi132 ceph-mon[103389]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:09:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:12 smithi049 ceph-mon[99485]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T09:09:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:12 smithi049 ceph-mon[99485]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:13.132 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127747]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:09:13.132 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127747]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:13.133 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127747]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:13.133 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:09:12 smithi049 sudo[127747]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:13.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:09:13.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:12 smithi184 ceph-mon[103675]: osdmap e20: 3 total, 3 up, 3 in 2023-10-08T09:09:13.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:12 smithi184 ceph-mon[103675]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:13.307 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113285]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:09:13.307 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113285]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:13.307 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113285]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:13.307 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113285]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:13.769 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113288]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:09:13.769 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113288]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:13.769 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113288]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:13.769 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:09:13 smithi132 sudo[113288]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:13.830 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114813]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:09:13.830 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114813]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:13.830 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114813]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:13.830 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114813]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1207617015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9fe096-1fd1-4de3-b21c-58a18d6cbec6"}]: dispatch 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1207617015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc9fe096-1fd1-4de3-b21c-58a18d6cbec6"}]': finished 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: osdmap e22: 4 total, 3 up, 4 in 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:13.831 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:13 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T09:09:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-08T09:09:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1207617015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9fe096-1fd1-4de3-b21c-58a18d6cbec6"}]: dispatch 2023-10-08T09:09:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1207617015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc9fe096-1fd1-4de3-b21c-58a18d6cbec6"}]': finished 2023-10-08T09:09:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: osdmap e22: 4 total, 3 up, 4 in 2023-10-08T09:09:14.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:14.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:13 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: osdmap e21: 3 total, 3 up, 3 in 2023-10-08T09:09:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-08T09:09:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1207617015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cc9fe096-1fd1-4de3-b21c-58a18d6cbec6"}]: dispatch 2023-10-08T09:09:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1207617015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cc9fe096-1fd1-4de3-b21c-58a18d6cbec6"}]': finished 2023-10-08T09:09:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: osdmap e22: 4 total, 3 up, 4 in 2023-10-08T09:09:14.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:14.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:14.247 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114824]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:09:14.247 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:14.247 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114824]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:14.247 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:09:13 smithi184 sudo[114824]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:14.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[127861]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:09:14.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[127861]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:14.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[127861]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:14.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[127861]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[128232]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:09:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[128232]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[128232]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:14 smithi049 sudo[128232]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:15.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:14 smithi184 sudo[114827]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:09:15.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:14 smithi184 sudo[114827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:15.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:14 smithi184 sudo[114827]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:15.244 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 sudo[114827]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:15.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 sudo[114830]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:09:15.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 sudo[114830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:15.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 sudo[114830]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:15.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 sudo[114830]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:15.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/434105444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:15.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:15.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/434105444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:15.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:15.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:15.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:15.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:15.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:15.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:15.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:15.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:15.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:15.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:15.740 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113291]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-08T09:09:15.740 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113291]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:15.740 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113291]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:15.740 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113291]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113294]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-08T09:09:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113294]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-08T09:09:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113294]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-08T09:09:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 sudo[113294]: pam_unix(sudo:session): session closed for user root 2023-10-08T09:09:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/434105444' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: pgmap v91: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:16.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:16 smithi132 ceph-mon[103389]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:16.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:16 smithi132 ceph-mon[103389]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:16.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:16 smithi132 ceph-mon[103389]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:16.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:16.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:16.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:16.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:16 smithi132 ceph-mon[103389]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:16.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:16 smithi049 ceph-mon[99485]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:16.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:16 smithi049 ceph-mon[99485]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:16.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:16 smithi049 ceph-mon[99485]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:16.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:16.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:16.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:16.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:16 smithi049 ceph-mon[99485]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:16.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:16 smithi184 ceph-mon[103675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:16.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:16 smithi184 ceph-mon[103675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:16.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:16 smithi184 ceph-mon[103675]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-08T09:09:16.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-08T09:09:16.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-08T09:09:16.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-08T09:09:16.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:16 smithi184 ceph-mon[103675]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-08T09:09:17.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:17 smithi049 ceph-mon[99485]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:17.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:17 smithi184 ceph-mon[103675]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:17 smithi132 ceph-mon[103389]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:19.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:19 smithi049 ceph-mon[99485]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:19 smithi184 ceph-mon[103675]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:19 smithi132 ceph-mon[103389]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:20.614 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T09:09:20.614 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T09:09:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-08T09:09:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:21.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:21 smithi049 ceph-mon[99485]: Deploying daemon osd.3 on smithi049 2023-10-08T09:09:21.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:21 smithi049 ceph-mon[99485]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-10-08T09:09:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:21 smithi184 ceph-mon[103675]: Deploying daemon osd.3 on smithi049 2023-10-08T09:09:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:21 smithi184 ceph-mon[103675]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-10-08T09:09:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:21 smithi132 ceph-mon[103389]: Deploying daemon osd.3 on smithi049 2023-10-08T09:09:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:21 smithi132 ceph-mon[103389]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-10-08T09:09:23.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:23 smithi049 ceph-mon[99485]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-08T09:09:23.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:23 smithi184 ceph-mon[103675]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-08T09:09:23.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:23 smithi132 ceph-mon[103389]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-10-08T09:09:25.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:25 smithi049 ceph-mon[99485]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-08T09:09:25.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:25 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:25.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:25 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:25.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:25 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:25.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:25 smithi184 ceph-mon[103675]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-08T09:09:25.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:25 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:25.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:25 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:25.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:25 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:25.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:25 smithi132 ceph-mon[103389]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-10-08T09:09:25.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:25 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:25.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:25 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:25.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:25 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:27.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.146+0000 7f606f7fe700 1 -- 172.21.15.49:0/3253214969 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f608019ce00 con 0x7f60680a13f0 2023-10-08T09:09:27.148 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 3 on host 'smithi049' 2023-10-08T09:09:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.148+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60680a13f0 msgr2=0x7f60680a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.148+0000 7f60848e4700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60680a13f0 0x7f60680a38b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6074009c60 tx=0x7f6074009260 comp rx=0 tx=0).stop 2023-10-08T09:09:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.148+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 msgr2=0x7f6080194590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:27.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.148+0000 7f60848e4700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f6080194590 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60700058e0 tx=0x7f607000fa00 comp rx=0 tx=0).stop 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 shutdown_connections 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60680a13f0 0x7f60680a38b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60801012d0 0x7f608019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60800ff730 0x7f6080194590 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 --2- 172.21.15.49:0/3253214969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6080100930 0x7f6080194ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 >> 172.21.15.49:0/3253214969 conn(0x7f60800facf0 msgr2=0x7f6080068280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 shutdown_connections 2023-10-08T09:09:27.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:09:27.149+0000 7f60848e4700 1 -- 172.21.15.49:0/3253214969 wait complete. 2023-10-08T09:09:27.583 DEBUG:teuthology.orchestra.run.smithi049:osd.3> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.3.service 2023-10-08T09:09:27.585 INFO:tasks.cephadm:Deploying osd.4 on smithi132 with /dev/vg_nvme/lv_3... 2023-10-08T09:09:27.585 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T09:09:27.616 INFO:journalctl@ceph.osd.3.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:09:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:09:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:09:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:27.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:27.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:27.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-10-08T09:09:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:28.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:28.151 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:09:28.382 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:09:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:09:28.158+0000 7fd0634bb640 -1 osd.3 0 log_to_monitors true 2023-10-08T09:09:28.645 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:28 smithi132 ceph-mon[103389]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T09:09:28.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:28 smithi049 ceph-mon[99485]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T09:09:28.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:28 smithi184 ceph-mon[103675]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-08T09:09:29.625 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T09:09:29.626 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T09:09:29.626 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-10-08T09:09:29.626 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-10-08T09:09:29.626 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115348 s, 909 MB/s 2023-10-08T09:09:29.626 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-10-08T09:09:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:29 smithi049 ceph-mon[99485]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-08T09:09:29.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:29 smithi049 ceph-mon[99485]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T09:09:29.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:29 smithi049 ceph-mon[99485]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T09:09:29.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:29 smithi049 ceph-mon[99485]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:09:29.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:29.883 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:09:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:09:29.598+0000 7fd058b32700 -1 osd.3 0 waiting for initial osdmap 2023-10-08T09:09:29.883 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:09:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:09:29.604+0000 7fd052923700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:09:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:29 smithi184 ceph-mon[103675]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-08T09:09:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:29 smithi184 ceph-mon[103675]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T09:09:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:29 smithi184 ceph-mon[103675]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T09:09:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:29 smithi184 ceph-mon[103675]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:09:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.016 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:29 smithi132 ceph-mon[103389]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-10-08T09:09:30.016 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:29 smithi132 ceph-mon[103389]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-08T09:09:30.016 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:29 smithi132 ceph-mon[103389]: osdmap e23: 4 total, 3 up, 4 in 2023-10-08T09:09:30.016 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:29 smithi132 ceph-mon[103389]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:09:30.016 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.100 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi132:vg_nvme/lv_3 2023-10-08T09:09:30.657 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:09:30.857 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:09:30.857 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T09:09:30.857 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:30.858 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:30.859 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.859 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:30.859 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:30.859 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:09:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T09:09:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:09:30.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: osdmap e24: 4 total, 3 up, 4 in 2023-10-08T09:09:30.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:30.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:30.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:30.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:30.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:30.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:30.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:09:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:09:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi049 2023-10-08T09:09:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi049 to 8584M 2023-10-08T09:09:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:31.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] boot 2023-10-08T09:09:31.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: osdmap e25: 4 total, 4 up, 4 in 2023-10-08T09:09:31.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:31.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:31 smithi049 ceph-mon[99485]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T09:09:31.898 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/3289504456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 msgr2=0x7f1b54107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:31.899 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/3289504456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b54107b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00b0d0 tx=0x7f1b4c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:09:31.900 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/3289504456 shutdown_connections 2023-10-08T09:09:31.900 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/3289504456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b540ff730 0x7f1b540ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:31.900 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/3289504456 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b54107b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:31.900 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/3289504456 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b54100930 0x7f1b54100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:31.900 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/3289504456 >> 172.21.15.132:0/3289504456 conn(0x7f1b540facf0 msgr2=0x7f1b540fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:09:31.900 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/3289504456 shutdown_connections 2023-10-08T09:09:31.900 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.897+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/3289504456 wait complete. 2023-10-08T09:09:31.903 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 Processor -- start 2023-10-08T09:09:31.903 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 -- start start 2023-10-08T09:09:31.903 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b540ff730 0x7f1b541968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:31.903 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b54100930 0x7f1b54196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b5419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419e3e0 con 0x7f1b54100930 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419e520 con 0x7f1b541012d0 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5bd0b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5419e6b0 con 0x7f1b540ff730 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b5419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b5419de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48902/0 (socket says 172.21.15.132:48902) 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 -- 172.21.15.132:0/2203788436 learned_addr learned my addr 172.21.15.132:0/2203788436 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b592a6700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b54100930 0x7f1b54196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b59aa7700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b540ff730 0x7f1b541968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 -- 172.21.15.132:0/2203788436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b540ff730 msgr2=0x7f1b541968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b540ff730 0x7f1b541968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:31.904 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 -- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b54100930 msgr2=0x7f1b54196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b54100930 0x7f1b54196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.898+0000 7f1b5a2a8700 1 -- 172.21.15.132:0/2203788436 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c009d00 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.899+0000 7f1b5a2a8700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b5419de10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007bd0 tx=0x7f1b4c0094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.899+0000 7f1b4affd700 1 -- 172.21.15.132:0/2203788436 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00e040 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.899+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5419e930 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.899+0000 7f1b4affd700 1 -- 172.21.15.132:0/2203788436 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b4c004e60 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.899+0000 7f1b4affd700 1 -- 172.21.15.132:0/2203788436 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c012a50 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.899+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5419ed40 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.900+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5404e4c0 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.901+0000 7f1b4affd700 1 -- 172.21.15.132:0/2203788436 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1b4c012bb0 con 0x7f1b541012d0 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.901+0000 7f1b4affd700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1b400a1480 0x7f1b400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:31.905 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.901+0000 7f1b4affd700 1 -- 172.21.15.132:0/2203788436 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f1b4c0cc9f0 con 0x7f1b541012d0 2023-10-08T09:09:31.906 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.902+0000 7f1b59aa7700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1b400a1480 0x7f1b400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:31.906 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.902+0000 7f1b59aa7700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1b400a1480 0x7f1b400a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b44006fd0 tx=0x7f1b44008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:09:31.907 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:31.906+0000 7f1b4affd700 1 -- 172.21.15.132:0/2203788436 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1b4c0972e0 con 0x7f1b541012d0 2023-10-08T09:09:31.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:09:31.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:09:31.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi049 2023-10-08T09:09:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi049 to 8584M 2023-10-08T09:09:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] boot 2023-10-08T09:09:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: osdmap e25: 4 total, 4 up, 4 in 2023-10-08T09:09:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:31 smithi184 ceph-mon[103675]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi049 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi049 to 8584M 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: osd.3 [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] boot 2023-10-08T09:09:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: osdmap e25: 4 total, 4 up, 4 in 2023-10-08T09:09:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-08T09:09:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:31 smithi132 ceph-mon[103389]: osdmap e26: 4 total, 4 up, 4 in 2023-10-08T09:09:32.103 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:32.102+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b54069170 con 0x7f1b400a1480 2023-10-08T09:09:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:32.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:32 smithi049 ceph-mon[99485]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T09:09:32.925 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:32.925 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:32.925 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:32.925 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:32 smithi132 ceph-mon[103389]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T09:09:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:32 smithi184 ceph-mon[103675]: osdmap e27: 4 total, 4 up, 4 in 2023-10-08T09:09:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:33 smithi049 ceph-mon[99485]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:33 smithi049 ceph-mon[99485]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:33 smithi184 ceph-mon[103675]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:33 smithi184 ceph-mon[103675]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:33 smithi132 ceph-mon[103389]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:33 smithi132 ceph-mon[103389]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:34.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:34 smithi049 ceph-mon[99485]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T09:09:34.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:34 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/2654226208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]: dispatch 2023-10-08T09:09:34.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:34 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]: dispatch 2023-10-08T09:09:34.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:34 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]': finished 2023-10-08T09:09:34.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:34 smithi049 ceph-mon[99485]: osdmap e29: 5 total, 4 up, 5 in 2023-10-08T09:09:34.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:34.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:34 smithi184 ceph-mon[103675]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T09:09:34.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:34 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/2654226208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]: dispatch 2023-10-08T09:09:34.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:34 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]: dispatch 2023-10-08T09:09:34.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:34 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]': finished 2023-10-08T09:09:34.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:34 smithi184 ceph-mon[103675]: osdmap e29: 5 total, 4 up, 5 in 2023-10-08T09:09:34.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:35.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:34 smithi132 ceph-mon[103389]: osdmap e28: 4 total, 4 up, 4 in 2023-10-08T09:09:35.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:34 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/2654226208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]: dispatch 2023-10-08T09:09:35.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:34 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]: dispatch 2023-10-08T09:09:35.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:34 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e852c5d9-cf88-4ddf-98e7-e22d644b4a30"}]': finished 2023-10-08T09:09:35.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:34 smithi132 ceph-mon[103389]: osdmap e29: 5 total, 4 up, 5 in 2023-10-08T09:09:35.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:35 smithi049 ceph-mon[99485]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:35 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/3069188124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:35.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:35 smithi184 ceph-mon[103675]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:35.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:35 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/3069188124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:36.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:35 smithi132 ceph-mon[103389]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:36.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:35 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/3069188124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:37.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:37 smithi049 ceph-mon[99485]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-08T09:09:37.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:37 smithi184 ceph-mon[103675]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-08T09:09:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:37 smithi132 ceph-mon[103389]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-08T09:09:39.710 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:39 smithi132 ceph-mon[103389]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:09:39.971 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:39 smithi049 ceph-mon[99485]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:09:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:39 smithi184 ceph-mon[103675]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:09:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:41 smithi132 ceph-mon[103389]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-10-08T09:09:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T09:09:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:41 smithi049 ceph-mon[99485]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-10-08T09:09:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T09:09:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:41 smithi184 ceph-mon[103675]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-10-08T09:09:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-08T09:09:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:42.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:42 smithi049 ceph-mon[99485]: Deploying daemon osd.4 on smithi132 2023-10-08T09:09:42.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:42 smithi184 ceph-mon[103675]: Deploying daemon osd.4 on smithi132 2023-10-08T09:09:43.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:42 smithi132 ceph-mon[103389]: Deploying daemon osd.4 on smithi132 2023-10-08T09:09:43.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:43 smithi132 ceph-mon[103389]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-10-08T09:09:43.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:43 smithi184 ceph-mon[103675]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-10-08T09:09:44.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:43 smithi049 ceph-mon[99485]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-10-08T09:09:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:45 smithi184 ceph-mon[103675]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T09:09:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:45 smithi132 ceph-mon[103389]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T09:09:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:46.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:45 smithi049 ceph-mon[99485]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T09:09:46.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:46.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:46.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:47.778 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:47 smithi132 ceph-mon[103389]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:47.778 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:47.917 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.913+0000 7f1b4affd700 1 -- 172.21.15.132:0/2203788436 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b54069170 con 0x7f1b400a1480 2023-10-08T09:09:47.918 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 4 on host 'smithi132' 2023-10-08T09:09:47.918 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1b400a1480 msgr2=0x7f1b400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1b400a1480 0x7f1b400a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b44006fd0 tx=0x7f1b44008040 comp rx=0 tx=0).stop 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 msgr2=0x7f1b5419de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b5419de10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c007bd0 tx=0x7f1b4c0094c0 comp rx=0 tx=0).stop 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 shutdown_connections 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1b400a1480 0x7f1b400a3940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b540ff730 0x7f1b541968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b541012d0 0x7f1b5419de10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 --2- 172.21.15.132:0/2203788436 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1b54100930 0x7f1b54196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 >> 172.21.15.132:0/2203788436 conn(0x7f1b540facf0 msgr2=0x7f1b54068a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:09:47.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 shutdown_connections 2023-10-08T09:09:47.920 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:09:47.916+0000 7f1b5bd0b700 1 -- 172.21.15.132:0/2203788436 wait complete. 2023-10-08T09:09:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:47 smithi184 ceph-mon[103675]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:47 smithi049 ceph-mon[99485]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:48.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.348 DEBUG:teuthology.orchestra.run.smithi132:osd.4> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.4.service 2023-10-08T09:09:48.351 INFO:tasks.cephadm:Deploying osd.5 on smithi184 with /dev/vg_nvme/lv_3... 2023-10-08T09:09:48.351 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-08T09:09:48.394 INFO:journalctl@ceph.osd.4.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:09:48.897 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.897 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:48.897 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:48.898 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.898 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.898 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:48.898 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.928 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:09:48.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:48.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:48.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:48.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:48.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:49.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:49.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:49.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:49.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:49.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:49.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:09:49.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:49.297 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:09:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:09:49.226+0000 7f3069bf4640 -1 osd.4 0 log_to_monitors true 2023-10-08T09:09:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:49 smithi184 ceph-mon[103675]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:49 smithi184 ceph-mon[103675]: from='osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:09:49.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:49 smithi184 ceph-mon[103675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:09:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:49 smithi132 ceph-mon[103389]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:49 smithi132 ceph-mon[103389]: from='osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:09:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:49 smithi132 ceph-mon[103389]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:09:50.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:49 smithi049 ceph-mon[99485]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:50.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:49 smithi049 ceph-mon[99485]: from='osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:09:50.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:49 smithi049 ceph-mon[99485]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-08T09:09:50.495 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-08T09:09:50.495 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-08T09:09:50.495 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-10-08T09:09:50.495 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-10-08T09:09:50.496 INFO:teuthology.orchestra.run.smithi184.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01111 s, 944 MB/s 2023-10-08T09:09:50.496 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-10-08T09:09:50.801 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:50 smithi184 ceph-mon[103675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T09:09:50.801 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:50 smithi184 ceph-mon[103675]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T09:09:50.801 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:50.802 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:50 smithi184 ceph-mon[103675]: from='osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:09:50.802 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:50 smithi184 ceph-mon[103675]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:09:50.869 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi184:vg_nvme/lv_3 2023-10-08T09:09:50.961 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:09:50 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:09:50.666+0000 7f305f26b700 -1 osd.4 0 waiting for initial osdmap 2023-10-08T09:09:50.961 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:09:50 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:09:50.671+0000 7f305885b700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:09:50.961 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:50 smithi132 ceph-mon[103389]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T09:09:50.961 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:50 smithi132 ceph-mon[103389]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T09:09:50.961 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:50.962 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:50 smithi132 ceph-mon[103389]: from='osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:09:50.962 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:50 smithi132 ceph-mon[103389]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:09:51.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:50 smithi049 ceph-mon[99485]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-08T09:09:51.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:50 smithi049 ceph-mon[99485]: osdmap e30: 5 total, 4 up, 5 in 2023-10-08T09:09:51.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:51.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:50 smithi049 ceph-mon[99485]: from='osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:09:51.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:50 smithi049 ceph-mon[99485]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:09:51.427 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:09:51.967 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:51.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:51.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:51.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:51.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:09:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T09:09:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:09:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: osdmap e31: 5 total, 4 up, 5 in 2023-10-08T09:09:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:52.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:09:52.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:52.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:09:52.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:09:52.672 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 -- 172.21.15.184:0/1837700458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d40ff730 msgr2=0x7fa7d40ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:52.672 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 --2- 172.21.15.184:0/1837700458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d40ff730 0x7fa7d40ffb50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009990 tx=0x7fa7bc009ca0 comp rx=0 tx=0).stop 2023-10-08T09:09:52.672 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 -- 172.21.15.184:0/1837700458 shutdown_connections 2023-10-08T09:09:52.672 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 --2- 172.21.15.184:0/1837700458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d40ff730 0x7fa7d40ffb50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:52.672 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 --2- 172.21.15.184:0/1837700458 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d41012d0 0x7fa7d4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:52.672 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 --2- 172.21.15.184:0/1837700458 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7d4100930 0x7fa7d4100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:52.673 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 -- 172.21.15.184:0/1837700458 >> 172.21.15.184:0/1837700458 conn(0x7fa7d40facf0 msgr2=0x7fa7d40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:09:52.673 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 -- 172.21.15.184:0/1837700458 shutdown_connections 2023-10-08T09:09:52.673 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.671+0000 7fa7d9806700 1 -- 172.21.15.184:0/1837700458 wait complete. 2023-10-08T09:09:52.674 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.672+0000 7fa7d9806700 1 Processor -- start 2023-10-08T09:09:52.674 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.672+0000 7fa7d9806700 1 -- start start 2023-10-08T09:09:52.674 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.672+0000 7fa7d9806700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d40ff730 0x7fa7d4195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:52.674 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d9806700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7d4100930 0x7fa7d4195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:52.674 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d9806700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d41012d0 0x7fa7d419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:52.674 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d9806700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d419d220 con 0x7fa7d40ff730 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d9806700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d419d360 con 0x7fa7d4100930 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d9806700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d419d4f0 con 0x7fa7d41012d0 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d41012d0 0x7fa7d419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d41012d0 0x7fa7d419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59574/0 (socket says 172.21.15.184:59574) 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 -- 172.21.15.184:0/2954535659 learned_addr learned my addr 172.21.15.184:0/2954535659 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d2ffd700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d40ff730 0x7fa7d4195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d27fc700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7d4100930 0x7fa7d4195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 -- 172.21.15.184:0/2954535659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7d4100930 msgr2=0x7fa7d4195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7d4100930 0x7fa7d4195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 -- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d40ff730 msgr2=0x7fa7d4195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:09:52.675 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d40ff730 0x7fa7d4195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:09:52.676 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 -- 172.21.15.184:0/2954535659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0095f0 con 0x7fa7d41012d0 2023-10-08T09:09:52.676 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.673+0000 7fa7d37fe700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d41012d0 0x7fa7d419cc50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc007f10 tx=0x7fa7cc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:09:52.676 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.675+0000 7fa7d8804700 1 -- 172.21.15.184:0/2954535659 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00dd20 con 0x7fa7d41012d0 2023-10-08T09:09:52.676 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.675+0000 7fa7d8804700 1 -- 172.21.15.184:0/2954535659 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7cc00f040 con 0x7fa7d41012d0 2023-10-08T09:09:52.677 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.675+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d40681f0 con 0x7fa7d41012d0 2023-10-08T09:09:52.677 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.675+0000 7fa7d8804700 1 -- 172.21.15.184:0/2954535659 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0048a0 con 0x7fa7d41012d0 2023-10-08T09:09:52.677 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.675+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d40686f0 con 0x7fa7d41012d0 2023-10-08T09:09:52.677 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.676+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d410b8f0 con 0x7fa7d41012d0 2023-10-08T09:09:52.677 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.677+0000 7fa7d8804700 1 -- 172.21.15.184:0/2954535659 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa7cc003a00 con 0x7fa7d41012d0 2023-10-08T09:09:52.678 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.677+0000 7fa7d8804700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa7c00a1500 0x7fa7c00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:09:52.678 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.678+0000 7fa7d8804700 1 -- 172.21.15.184:0/2954535659 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fa7cc0cbca0 con 0x7fa7d41012d0 2023-10-08T09:09:52.678 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.678+0000 7fa7d2ffd700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa7c00a1500 0x7fa7c00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:09:52.679 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.678+0000 7fa7d2ffd700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa7c00a1500 0x7fa7c00a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc003790 tx=0x7fa7bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:09:52.683 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.683+0000 7fa7d8804700 1 -- 172.21.15.184:0/2954535659 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa7cc096370 con 0x7fa7d41012d0 2023-10-08T09:09:52.879 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:09:52.878+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7d4068910 con 0x7fa7c00a1500 2023-10-08T09:09:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:52 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:09:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:52 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:09:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:52 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi132 2023-10-08T09:09:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:52 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi132 to 8584M 2023-10-08T09:09:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:52 smithi184 ceph-mon[103675]: osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] boot 2023-10-08T09:09:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:52 smithi184 ceph-mon[103675]: osdmap e32: 5 total, 5 up, 5 in 2023-10-08T09:09:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:52 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:09:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:52 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:09:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:52 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi132 2023-10-08T09:09:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:52 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi132 to 8584M 2023-10-08T09:09:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:52 smithi132 ceph-mon[103389]: osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] boot 2023-10-08T09:09:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:52 smithi132 ceph-mon[103389]: osdmap e32: 5 total, 5 up, 5 in 2023-10-08T09:09:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:52 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:09:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:52 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:09:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:52 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi132 2023-10-08T09:09:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:52 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi132 to 8584M 2023-10-08T09:09:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:52 smithi049 ceph-mon[99485]: osd.4 [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] boot 2023-10-08T09:09:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:52 smithi049 ceph-mon[99485]: osdmap e32: 5 total, 5 up, 5 in 2023-10-08T09:09:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-08T09:09:53.983 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:53 smithi184 ceph-mon[103675]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:53.983 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:53 smithi184 ceph-mon[103675]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T09:09:53.983 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:53.983 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:53.984 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:54.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:53 smithi132 ceph-mon[103389]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:54.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:53 smithi132 ceph-mon[103389]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T09:09:54.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:54.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:54.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:53 smithi049 ceph-mon[99485]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2023-10-08T09:09:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:53 smithi049 ceph-mon[99485]: osdmap e33: 5 total, 5 up, 5 in 2023-10-08T09:09:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:09:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:09:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:09:54.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:54 smithi184 ceph-mon[103675]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:55.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:54 smithi132 ceph-mon[103389]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:55.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:54 smithi049 ceph-mon[99485]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:09:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:55 smithi184 ceph-mon[103675]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:55 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/764941014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]: dispatch 2023-10-08T09:09:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:55 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]: dispatch 2023-10-08T09:09:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:55 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]': finished 2023-10-08T09:09:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:55 smithi184 ceph-mon[103675]: osdmap e34: 6 total, 5 up, 6 in 2023-10-08T09:09:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:09:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:55 smithi132 ceph-mon[103389]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:55 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/764941014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]: dispatch 2023-10-08T09:09:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:55 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]: dispatch 2023-10-08T09:09:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:55 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]': finished 2023-10-08T09:09:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:55 smithi132 ceph-mon[103389]: osdmap e34: 6 total, 5 up, 6 in 2023-10-08T09:09:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:09:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:55 smithi049 ceph-mon[99485]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:55 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/764941014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]: dispatch 2023-10-08T09:09:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:55 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]: dispatch 2023-10-08T09:09:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:55 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95803835-c3f8-469c-bee8-95249eccbe51"}]': finished 2023-10-08T09:09:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:55 smithi049 ceph-mon[99485]: osdmap e34: 6 total, 5 up, 6 in 2023-10-08T09:09:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:09:56.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:56 smithi184 ceph-mon[103675]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:56.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:56 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/175078762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:57.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:56 smithi132 ceph-mon[103389]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:57.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:56 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/175078762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:57.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:56 smithi049 ceph-mon[99485]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:57.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:56 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/175078762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:09:59.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:09:58 smithi049 ceph-mon[99485]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:59.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:09:58 smithi184 ceph-mon[103675]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:09:59.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:09:58 smithi132 ceph-mon[103389]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:01.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:00 smithi049 ceph-mon[99485]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:01.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:00 smithi049 ceph-mon[99485]: overall HEALTH_OK 2023-10-08T09:10:01.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:00 smithi184 ceph-mon[103675]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:01.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:00 smithi184 ceph-mon[103675]: overall HEALTH_OK 2023-10-08T09:10:01.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:00 smithi132 ceph-mon[103389]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:01.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:00 smithi132 ceph-mon[103389]: overall HEALTH_OK 2023-10-08T09:10:02.632 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:02 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T09:10:02.632 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:02 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:02 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T09:10:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:02 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:02 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-08T09:10:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:02 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:03.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:03 smithi184 ceph-mon[103675]: Deploying daemon osd.5 on smithi184 2023-10-08T09:10:03.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:03 smithi184 ceph-mon[103675]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:03 smithi049 ceph-mon[99485]: Deploying daemon osd.5 on smithi184 2023-10-08T09:10:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:03 smithi049 ceph-mon[99485]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:03 smithi132 ceph-mon[103389]: Deploying daemon osd.5 on smithi184 2023-10-08T09:10:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:03 smithi132 ceph-mon[103389]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:05.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:05 smithi049 ceph-mon[99485]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:05.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:05 smithi184 ceph-mon[103675]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:05 smithi132 ceph-mon[103389]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:07.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:07 smithi184 ceph-mon[103675]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:07.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:07 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:07.206 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:07 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:07.207 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:07 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:07.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:07 smithi049 ceph-mon[99485]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:07.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:07 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:07.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:07 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:07.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:07 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:07.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:07 smithi132 ceph-mon[103389]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:07.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:07 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:07.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:07 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:07.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:07 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:08.791 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.790+0000 7fa7d8804700 1 -- 172.21.15.184:0/2954535659 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa7d4068910 con 0x7fa7c00a1500 2023-10-08T09:10:08.791 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 5 on host 'smithi184' 2023-10-08T09:10:08.793 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa7c00a1500 msgr2=0x7fa7c00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:08.794 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa7c00a1500 0x7fa7c00a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc003790 tx=0x7fa7bc005fd0 comp rx=0 tx=0).stop 2023-10-08T09:10:08.794 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d41012d0 msgr2=0x7fa7d419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:08.794 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d41012d0 0x7fa7d419cc50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc007f10 tx=0x7fa7cc00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:10:08.794 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 shutdown_connections 2023-10-08T09:10:08.794 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa7c00a1500 0x7fa7c00a39c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:08.795 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d41012d0 0x7fa7d419cc50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:08.795 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7d40ff730 0x7fa7d4195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:08.795 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 --2- 172.21.15.184:0/2954535659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7d4100930 0x7fa7d4195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:08.795 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 >> 172.21.15.184:0/2954535659 conn(0x7fa7d40facf0 msgr2=0x7fa7d4103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:10:08.795 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 shutdown_connections 2023-10-08T09:10:08.796 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:08.793+0000 7fa7d9806700 1 -- 172.21.15.184:0/2954535659 wait complete. 2023-10-08T09:10:09.205 DEBUG:teuthology.orchestra.run.smithi184:osd.5> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.5.service 2023-10-08T09:10:09.207 INFO:tasks.cephadm:Deploying osd.6 on smithi049 with /dev/vg_nvme/lv_2... 2023-10-08T09:10:09.208 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T09:10:09.238 INFO:journalctl@ceph.osd.5.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:10:09.806 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:09.898 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:10.247 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:10:10 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:10:10.059+0000 7f28a9f2c640 -1 osd.5 0 log_to_monitors true 2023-10-08T09:10:10.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:10 smithi049 ceph-mon[99485]: from='osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:10:10.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:10 smithi049 ceph-mon[99485]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:10:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:10 smithi184 ceph-mon[103675]: from='osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:10:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:10 smithi184 ceph-mon[103675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:10:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:10 smithi132 ceph-mon[103389]: from='osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:10:11.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:10 smithi132 ceph-mon[103389]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-08T09:10:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T09:10:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T09:10:11.324 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2023-10-08T09:10:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2023-10-08T09:10:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00840378 s, 1.2 GB/s 2023-10-08T09:10:11.324 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2023-10-08T09:10:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:11 smithi049 ceph-mon[99485]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:11 smithi049 ceph-mon[99485]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T09:10:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:11 smithi049 ceph-mon[99485]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T09:10:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:11 smithi049 ceph-mon[99485]: from='osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:10:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:11 smithi049 ceph-mon[99485]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:10:11.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:10:11.578+0000 7f289f5a3700 -1 osd.5 0 waiting for initial osdmap 2023-10-08T09:10:11.636 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:10:11.584+0000 7f2899394700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:10:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-mon[103675]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-mon[103675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T09:10:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-mon[103675]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T09:10:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-mon[103675]: from='osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:10:11.636 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:11 smithi184 ceph-mon[103675]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:10:11.771 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi049:vg_nvme/lv_2 2023-10-08T09:10:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:11 smithi132 ceph-mon[103389]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:11 smithi132 ceph-mon[103389]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-08T09:10:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:11 smithi132 ceph-mon[103389]: osdmap e35: 6 total, 5 up, 6 in 2023-10-08T09:10:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:11 smithi132 ceph-mon[103389]: from='osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:10:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:11 smithi132 ceph-mon[103389]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:10:12.336 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:10:12.863 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:13.003 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:13.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:13.004 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.004 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:10:13.004 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:13.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:10:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: osdmap e36: 6 total, 5 up, 6 in 2023-10-08T09:10:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:13.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:13.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:13.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:13.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:13.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 -- 172.21.15.49:0/51501359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108103c90 msgr2=0x7f0108106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 --2- 172.21.15.49:0/51501359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108103c90 0x7f0108106080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009990 tx=0x7f00f0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:10:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 -- 172.21.15.49:0/51501359 shutdown_connections 2023-10-08T09:10:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 --2- 172.21.15.49:0/51501359 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01081065c0 0x7f01081089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 --2- 172.21.15.49:0/51501359 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0108101360 0x7f0108103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 --2- 172.21.15.49:0/51501359 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108103c90 0x7f0108106080 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 -- 172.21.15.49:0/51501359 >> 172.21.15.49:0/51501359 conn(0x7f01080facf0 msgr2=0x7f01080fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:10:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 -- 172.21.15.49:0/51501359 shutdown_connections 2023-10-08T09:10:13.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 -- 172.21.15.49:0/51501359 wait complete. 2023-10-08T09:10:13.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 Processor -- start 2023-10-08T09:10:13.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.766+0000 7f010dae4700 1 -- start start 2023-10-08T09:10:13.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f010dae4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108101360 0x7f0108100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f010dae4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0108103c90 0x7f01080ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f010dae4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01081065c0 0x7f01080ff850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f010dae4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01080ffff0 con 0x7f0108101360 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f010dae4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108100160 con 0x7f01081065c0 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f010dae4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01081a07d0 con 0x7f0108103c90 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108101360 0x7f0108100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108101360 0x7f0108100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47152/0 (socket says 172.21.15.49:47152) 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 -- 172.21.15.49:0/2491573182 learned_addr learned my addr 172.21.15.49:0/2491573182 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:10:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f0106ffd700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0108103c90 0x7f01080ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f0107fff700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01081065c0 0x7f01080ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 -- 172.21.15.49:0/2491573182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0108103c90 msgr2=0x7f01080ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0108103c90 0x7f01080ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 -- 172.21.15.49:0/2491573182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01081065c0 msgr2=0x7f01080ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01081065c0 0x7f01080ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 -- 172.21.15.49:0/2491573182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00095f0 con 0x7f0108101360 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f01077fe700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108101360 0x7f0108100c70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00f800d950 tx=0x7f00f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.767+0000 7f0104ff9700 1 -- 172.21.15.49:0/2491573182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8009940 con 0x7f0108101360 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.768+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01081a0a60 con 0x7f0108101360 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.768+0000 7f0104ff9700 1 -- 172.21.15.49:0/2491573182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f8010460 con 0x7f0108101360 2023-10-08T09:10:13.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.768+0000 7f0104ff9700 1 -- 172.21.15.49:0/2491573182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800f620 con 0x7f0108101360 2023-10-08T09:10:13.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.768+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01081a0f60 con 0x7f0108101360 2023-10-08T09:10:13.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.768+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01080fc870 con 0x7f0108101360 2023-10-08T09:10:13.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.771+0000 7f0104ff9700 1 -- 172.21.15.49:0/2491573182 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f00f80108d0 con 0x7f0108101360 2023-10-08T09:10:13.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.771+0000 7f0104ff9700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00f40a1480 0x7f00f40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.771+0000 7f0106ffd700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00f40a1480 0x7f00f40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.771+0000 7f0104ff9700 1 -- 172.21.15.49:0/2491573182 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f00f80cbcf0 con 0x7f0108101360 2023-10-08T09:10:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.772+0000 7f0106ffd700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00f40a1480 0x7f00f40a3940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b5c0 tx=0x7f00f0003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:10:13.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.775+0000 7f0104ff9700 1 -- 172.21.15.49:0/2491573182 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f00f80961f0 con 0x7f0108101360 2023-10-08T09:10:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:10:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:10:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi184 2023-10-08T09:10:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi184 to 8584M 2023-10-08T09:10:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] boot 2023-10-08T09:10:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: osdmap e37: 6 total, 6 up, 6 in 2023-10-08T09:10:13.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:13.985 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:13.985+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f010802cfb0 con 0x7f00f40a1480 2023-10-08T09:10:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:10:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:10:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi184 2023-10-08T09:10:13.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi184 to 8584M 2023-10-08T09:10:13.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:13.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] boot 2023-10-08T09:10:13.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: osdmap e37: 6 total, 6 up, 6 in 2023-10-08T09:10:13.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:13 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:10:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:10:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi184 2023-10-08T09:10:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi184 to 8584M 2023-10-08T09:10:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2023-10-08T09:10:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: osd.5 [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] boot 2023-10-08T09:10:14.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: osdmap e37: 6 total, 6 up, 6 in 2023-10-08T09:10:14.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:13 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-08T09:10:14.845 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:14 smithi049 ceph-mon[99485]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T09:10:14.845 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:14 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:14.845 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:14 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:14.845 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:14 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:14.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:14 smithi184 ceph-mon[103675]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T09:10:14.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:14 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:14.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:14 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:14.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:14 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:15.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:14 smithi132 ceph-mon[103389]: osdmap e38: 6 total, 6 up, 6 in 2023-10-08T09:10:15.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:14 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:15.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:14 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:15.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:14 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:15 smithi049 ceph-mon[99485]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:15 smithi049 ceph-mon[99485]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:15 smithi049 ceph-mon[99485]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T09:10:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:15 smithi184 ceph-mon[103675]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:15 smithi184 ceph-mon[103675]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:15 smithi184 ceph-mon[103675]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T09:10:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:15 smithi132 ceph-mon[103389]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:15 smithi132 ceph-mon[103389]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:15 smithi132 ceph-mon[103389]: osdmap e39: 6 total, 6 up, 6 in 2023-10-08T09:10:16.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:16 smithi049 ceph-mon[99485]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T09:10:16.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:16 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4107723982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811a7eaa-3df2-45a3-9cf3-b6a559db1546"}]: dispatch 2023-10-08T09:10:16.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:16 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4107723982' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "811a7eaa-3df2-45a3-9cf3-b6a559db1546"}]': finished 2023-10-08T09:10:16.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:16 smithi049 ceph-mon[99485]: osdmap e41: 7 total, 6 up, 7 in 2023-10-08T09:10:16.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:16.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:16 smithi184 ceph-mon[103675]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T09:10:16.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:16 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4107723982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811a7eaa-3df2-45a3-9cf3-b6a559db1546"}]: dispatch 2023-10-08T09:10:16.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:16 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4107723982' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "811a7eaa-3df2-45a3-9cf3-b6a559db1546"}]': finished 2023-10-08T09:10:16.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:16 smithi184 ceph-mon[103675]: osdmap e41: 7 total, 6 up, 7 in 2023-10-08T09:10:16.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:17.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:16 smithi132 ceph-mon[103389]: osdmap e40: 6 total, 6 up, 6 in 2023-10-08T09:10:17.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:16 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4107723982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "811a7eaa-3df2-45a3-9cf3-b6a559db1546"}]: dispatch 2023-10-08T09:10:17.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:16 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4107723982' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "811a7eaa-3df2-45a3-9cf3-b6a559db1546"}]': finished 2023-10-08T09:10:17.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:16 smithi132 ceph-mon[103389]: osdmap e41: 7 total, 6 up, 7 in 2023-10-08T09:10:17.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:17.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:17 smithi049 ceph-mon[99485]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:17.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:17 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/514703131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:10:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:17 smithi184 ceph-mon[103675]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:17 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/514703131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:10:18.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:17 smithi132 ceph-mon[103389]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:18.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:17 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/514703131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:10:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:19 smithi049 ceph-mon[99485]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:19.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:19 smithi184 ceph-mon[103675]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:19 smithi132 ceph-mon[103389]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:21.678 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:21 smithi049 ceph-mon[99485]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:10:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:21 smithi184 ceph-mon[103675]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:10:22.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:21 smithi132 ceph-mon[103389]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:10:23.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:23 smithi049 ceph-mon[99485]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-10-08T09:10:23.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:23 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T09:10:23.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:23 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:23 smithi184 ceph-mon[103675]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-10-08T09:10:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:23 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T09:10:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:23 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:24.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:23 smithi132 ceph-mon[103389]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-10-08T09:10:24.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:23 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-08T09:10:24.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:23 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:24.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:24 smithi049 ceph-mon[99485]: Deploying daemon osd.6 on smithi049 2023-10-08T09:10:24.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:24 smithi184 ceph-mon[103675]: Deploying daemon osd.6 on smithi049 2023-10-08T09:10:25.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:24 smithi132 ceph-mon[103389]: Deploying daemon osd.6 on smithi049 2023-10-08T09:10:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:25 smithi049 ceph-mon[99485]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-10-08T09:10:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:25 smithi184 ceph-mon[103675]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-10-08T09:10:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:25 smithi132 ceph-mon[103389]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-10-08T09:10:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:27 smithi049 ceph-mon[99485]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T09:10:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:27 smithi184 ceph-mon[103675]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T09:10:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:27 smithi132 ceph-mon[103389]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-08T09:10:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:29 smithi049 ceph-mon[99485]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-08T09:10:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:29 smithi184 ceph-mon[103675]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-08T09:10:30.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:29 smithi132 ceph-mon[103389]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-08T09:10:30.239 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 6 on host 'smithi049' 2023-10-08T09:10:30.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.238+0000 7f0104ff9700 1 -- 172.21.15.49:0/2491573182 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f010802cfb0 con 0x7f00f40a1480 2023-10-08T09:10:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00f40a1480 msgr2=0x7f00f40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00f40a1480 0x7f00f40a3940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b5c0 tx=0x7f00f0003790 comp rx=0 tx=0).stop 2023-10-08T09:10:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108101360 msgr2=0x7f0108100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108101360 0x7f0108100c70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f00f800d950 tx=0x7f00f800dc60 comp rx=0 tx=0).stop 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 shutdown_connections 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00f40a1480 0x7f00f40a3940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f01081065c0 0x7f01080ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0108101360 0x7f0108100c70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 --2- 172.21.15.49:0/2491573182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0108103c90 0x7f01080ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 >> 172.21.15.49:0/2491573182 conn(0x7f01080facf0 msgr2=0x7f0108101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 shutdown_connections 2023-10-08T09:10:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:10:30.240+0000 7f010dae4700 1 -- 172.21.15.49:0/2491573182 wait complete. 2023-10-08T09:10:30.636 DEBUG:teuthology.orchestra.run.smithi049:osd.6> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.6.service 2023-10-08T09:10:30.638 INFO:tasks.cephadm:Deploying osd.7 on smithi132 with /dev/vg_nvme/lv_2... 2023-10-08T09:10:30.638 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T09:10:30.668 INFO:journalctl@ceph.osd.6.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:10:31.207 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:10:31.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:10:31.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:31.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:31.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:31.267 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:10:31.267 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.267 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.267 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:31.268 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:31.268 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.268 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.268 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.268 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:10:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:31.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:31.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:10:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:10:31.512+0000 7fbc36984640 -1 osd.6 0 log_to_monitors true 2023-10-08T09:10:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:32 smithi132 ceph-mon[103389]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T09:10:32.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:32 smithi049 ceph-mon[99485]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T09:10:32.742 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T09:10:32.742 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T09:10:32.742 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-10-08T09:10:32.742 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-10-08T09:10:32.742 INFO:teuthology.orchestra.run.smithi132.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110905 s, 945 MB/s 2023-10-08T09:10:32.743 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-10-08T09:10:32.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:32 smithi184 ceph-mon[103675]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-08T09:10:33.162 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi132:vg_nvme/lv_2 2023-10-08T09:10:33.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:10:33 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:10:33.368+0000 7fbc2bffb700 -1 osd.6 0 waiting for initial osdmap 2023-10-08T09:10:33.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:10:33 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:10:33.379+0000 7fbc24dea700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:10:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:33 smithi049 ceph-mon[99485]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:33 smithi049 ceph-mon[99485]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T09:10:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:33 smithi049 ceph-mon[99485]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T09:10:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:33 smithi049 ceph-mon[99485]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:10:33.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:33 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:33.717 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:10:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:33 smithi184 ceph-mon[103675]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:33 smithi184 ceph-mon[103675]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T09:10:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:33 smithi184 ceph-mon[103675]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T09:10:33.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:33 smithi184 ceph-mon[103675]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:10:33.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:33 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:33 smithi132 ceph-mon[103389]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:33 smithi132 ceph-mon[103389]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-08T09:10:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:33 smithi132 ceph-mon[103389]: osdmap e42: 7 total, 6 up, 7 in 2023-10-08T09:10:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:33 smithi132 ceph-mon[103389]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:10:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:33 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:34.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:10:34.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T09:10:34.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:34.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi049 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi049 to 5722M 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:34.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:34.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:10:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T09:10:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi049 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi049 to 5722M 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:34.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:10:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: osdmap e43: 7 total, 6 up, 7 in 2023-10-08T09:10:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi049 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi049 to 5722M 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:34.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:34.942 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 -- 172.21.15.132:0/2588471290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc40100ef0 msgr2=0x7fbc40101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:34.942 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 --2- 172.21.15.132:0/2588471290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc40100ef0 0x7fbc40101310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbc30009990 tx=0x7fbc30009ca0 comp rx=0 tx=0).stop 2023-10-08T09:10:34.942 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 -- 172.21.15.132:0/2588471290 shutdown_connections 2023-10-08T09:10:34.942 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 --2- 172.21.15.132:0/2588471290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc401003d0 0x7fbc40107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:34.942 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 --2- 172.21.15.132:0/2588471290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc40100ef0 0x7fbc40101310 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 --2- 172.21.15.132:0/2588471290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc400ff9c0 0x7fbc400ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 -- 172.21.15.132:0/2588471290 >> 172.21.15.132:0/2588471290 conn(0x7fbc400fa490 msgr2=0x7fbc400fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 -- 172.21.15.132:0/2588471290 shutdown_connections 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 -- 172.21.15.132:0/2588471290 wait complete. 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 Processor -- start 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.940+0000 7fbc45467700 1 -- start start 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc400ff9c0 0x7fbc400ff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc401003d0 0x7fbc4019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc40100ef0 0x7fbc4019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4019e2f0 con 0x7fbc40100ef0 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4019e460 con 0x7fbc400ff9c0 2023-10-08T09:10:34.943 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4019e620 con 0x7fbc401003d0 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc400ff9c0 0x7fbc400ff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc400ff9c0 0x7fbc400ff3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48998/0 (socket says 172.21.15.132:48998) 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 -- 172.21.15.132:0/3830694302 learned_addr learned my addr 172.21.15.132:0/3830694302 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3e7fc700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc401003d0 0x7fbc4019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3f7fe700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc40100ef0 0x7fbc4019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 -- 172.21.15.132:0/3830694302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc401003d0 msgr2=0x7fbc4019b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc401003d0 0x7fbc4019b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 -- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc40100ef0 msgr2=0x7fbc4019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc40100ef0 0x7fbc4019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 -- 172.21.15.132:0/3830694302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300095f0 con 0x7fbc400ff9c0 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc3effd700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc400ff9c0 0x7fbc400ff3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbc300095c0 tx=0x7fbc3000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:10:34.944 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc27fff700 1 -- 172.21.15.132:0/3830694302 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3001d070 con 0x7fbc400ff9c0 2023-10-08T09:10:34.945 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4019e8a0 con 0x7fbc400ff9c0 2023-10-08T09:10:34.945 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc27fff700 1 -- 172.21.15.132:0/3830694302 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc30022470 con 0x7fbc400ff9c0 2023-10-08T09:10:34.945 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc27fff700 1 -- 172.21.15.132:0/3830694302 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc30017650 con 0x7fbc400ff9c0 2023-10-08T09:10:34.945 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.941+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4019ed10 con 0x7fbc400ff9c0 2023-10-08T09:10:34.946 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.943+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc40106c30 con 0x7fbc400ff9c0 2023-10-08T09:10:34.946 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.944+0000 7fbc27fff700 1 -- 172.21.15.132:0/3830694302 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbc30022850 con 0x7fbc400ff9c0 2023-10-08T09:10:34.946 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.944+0000 7fbc27fff700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc280a1480 0x7fbc280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:34.946 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.944+0000 7fbc27fff700 1 -- 172.21.15.132:0/3830694302 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fbc300cd380 con 0x7fbc400ff9c0 2023-10-08T09:10:34.946 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.944+0000 7fbc3e7fc700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc280a1480 0x7fbc280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:34.946 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.945+0000 7fbc3e7fc700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc280a1480 0x7fbc280a3940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbc34005950 tx=0x7fbc3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:10:34.948 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:34.949+0000 7fbc27fff700 1 -- 172.21.15.132:0/3830694302 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc300978a0 con 0x7fbc400ff9c0 2023-10-08T09:10:35.145 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:35.145+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc40060c30 con 0x7fbc280a1480 2023-10-08T09:10:35.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:10:35.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:10:35.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:35.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] boot 2023-10-08T09:10:35.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: osdmap e44: 7 total, 7 up, 7 in 2023-10-08T09:10:35.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:35.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:35.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:35.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:10:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:10:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] boot 2023-10-08T09:10:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: osdmap e44: 7 total, 7 up, 7 in 2023-10-08T09:10:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:10:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:10:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-10-08T09:10:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: osd.6 [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] boot 2023-10-08T09:10:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: osdmap e44: 7 total, 7 up, 7 in 2023-10-08T09:10:35.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-08T09:10:35.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:35.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:35.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:36.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:36 smithi132 ceph-mon[103389]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:36.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:36 smithi132 ceph-mon[103389]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T09:10:36.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:36 smithi184 ceph-mon[103675]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:36.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:36 smithi184 ceph-mon[103675]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T09:10:36.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:36 smithi049 ceph-mon[99485]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:36.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:36 smithi049 ceph-mon[99485]: osdmap e45: 7 total, 7 up, 7 in 2023-10-08T09:10:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:37 smithi184 ceph-mon[103675]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:37 smithi132 ceph-mon[103389]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:37.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:37 smithi049 ceph-mon[99485]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/3339249123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]: dispatch 2023-10-08T09:10:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:38 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]: dispatch 2023-10-08T09:10:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:38 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]': finished 2023-10-08T09:10:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:38 smithi184 ceph-mon[103675]: osdmap e46: 8 total, 7 up, 8 in 2023-10-08T09:10:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:38.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/1691405163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:10:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/3339249123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]: dispatch 2023-10-08T09:10:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:38 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]: dispatch 2023-10-08T09:10:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:38 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]': finished 2023-10-08T09:10:38.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:38 smithi132 ceph-mon[103389]: osdmap e46: 8 total, 7 up, 8 in 2023-10-08T09:10:38.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:38.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/1691405163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:10:38.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/3339249123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]: dispatch 2023-10-08T09:10:38.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:38 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]: dispatch 2023-10-08T09:10:38.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:38 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d84fc995-e0a5-4655-831b-cacfceee4afa"}]': finished 2023-10-08T09:10:38.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:38 smithi049 ceph-mon[99485]: osdmap e46: 8 total, 7 up, 8 in 2023-10-08T09:10:38.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:38.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/1691405163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:10:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:39 smithi049 ceph-mon[99485]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:39 smithi184 ceph-mon[103675]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:39 smithi132 ceph-mon[103389]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:41 smithi049 ceph-mon[99485]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:41 smithi184 ceph-mon[103675]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:41 smithi132 ceph-mon[103389]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:43.727 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:43 smithi132 ceph-mon[103389]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:43 smithi049 ceph-mon[99485]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:43.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:43 smithi184 ceph-mon[103675]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:44.874 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T09:10:44.874 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:44.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T09:10:44.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:44.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-08T09:10:44.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:45.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:45 smithi132 ceph-mon[103389]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:46.045 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:45 smithi132 ceph-mon[103389]: Deploying daemon osd.7 on smithi132 2023-10-08T09:10:46.045 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:45 smithi184 ceph-mon[103675]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:46.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:45 smithi184 ceph-mon[103675]: Deploying daemon osd.7 on smithi132 2023-10-08T09:10:46.046 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:45 smithi049 ceph-mon[99485]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:46.046 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:45 smithi049 ceph-mon[99485]: Deploying daemon osd.7 on smithi132 2023-10-08T09:10:47.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:47 smithi049 ceph-mon[99485]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:47 smithi184 ceph-mon[103675]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:48.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:47 smithi132 ceph-mon[103389]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:49.791 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:49 smithi132 ceph-mon[103389]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:49.791 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:49.792 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:49.792 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:49 smithi049 ceph-mon[99485]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:49 smithi184 ceph-mon[103675]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:51.313 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.312+0000 7fbc27fff700 1 -- 172.21.15.132:0/3830694302 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbc40060c30 con 0x7fbc280a1480 2023-10-08T09:10:51.314 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 7 on host 'smithi132' 2023-10-08T09:10:51.316 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc280a1480 msgr2=0x7fbc280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:51.316 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc280a1480 0x7fbc280a3940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbc34005950 tx=0x7fbc3400b410 comp rx=0 tx=0).stop 2023-10-08T09:10:51.316 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc400ff9c0 msgr2=0x7fbc400ff3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:51.316 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc400ff9c0 0x7fbc400ff3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbc300095c0 tx=0x7fbc3000f690 comp rx=0 tx=0).stop 2023-10-08T09:10:51.316 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 shutdown_connections 2023-10-08T09:10:51.317 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc280a1480 0x7fbc280a3940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:51.317 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc40100ef0 0x7fbc4019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:51.317 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc400ff9c0 0x7fbc400ff3c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:51.317 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 --2- 172.21.15.132:0/3830694302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc401003d0 0x7fbc4019b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:51.317 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 >> 172.21.15.132:0/3830694302 conn(0x7fbc400fa490 msgr2=0x7fbc40103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:10:51.318 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 shutdown_connections 2023-10-08T09:10:51.318 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:10:51.314+0000 7fbc45467700 1 -- 172.21.15.132:0/3830694302 wait complete. 2023-10-08T09:10:51.715 DEBUG:teuthology.orchestra.run.smithi132:osd.7> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.7.service 2023-10-08T09:10:51.717 INFO:tasks.cephadm:Deploying osd.8 on smithi184 with /dev/vg_nvme/lv_2... 2023-10-08T09:10:51.717 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-08T09:10:51.749 INFO:journalctl@ceph.osd.7.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:10:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:51 smithi049 ceph-mon[99485]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:51 smithi184 ceph-mon[103675]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:51.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:51 smithi132 ceph-mon[103389]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:52.294 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:10:52.547 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:10:52 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:10:52.498+0000 7ff8bf070640 -1 osd.7 0 log_to_monitors true 2023-10-08T09:10:52.806 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:52 smithi184 ceph-mon[103675]: from='osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:10:52.806 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:52 smithi184 ceph-mon[103675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:10:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:52 smithi049 ceph-mon[99485]: from='osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:10:52.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:52 smithi049 ceph-mon[99485]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:10:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:52 smithi132 ceph-mon[103389]: from='osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:10:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:52 smithi132 ceph-mon[103389]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-08T09:10:53.750 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-08T09:10:53.750 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-08T09:10:53.750 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-10-08T09:10:53.751 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-10-08T09:10:53.751 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110493 s, 949 MB/s 2023-10-08T09:10:53.751 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-10-08T09:10:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:53 smithi049 ceph-mon[99485]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:53 smithi049 ceph-mon[99485]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T09:10:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:53 smithi049 ceph-mon[99485]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T09:10:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:53.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:53 smithi049 ceph-mon[99485]: from='osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:10:53.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:53 smithi049 ceph-mon[99485]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:10:53.906 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:53 smithi184 ceph-mon[103675]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:53.906 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:53 smithi184 ceph-mon[103675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T09:10:53.907 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:53 smithi184 ceph-mon[103675]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T09:10:53.907 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:53.908 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:53 smithi184 ceph-mon[103675]: from='osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:10:53.908 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:53 smithi184 ceph-mon[103675]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:10:53.999 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:10:53.599+0000 7ff8b46e7700 -1 osd.7 0 waiting for initial osdmap 2023-10-08T09:10:53.999 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:10:53.612+0000 7ff8ae4d8700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:10:53.999 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-mon[103389]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:53.999 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-mon[103389]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-08T09:10:54.000 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-mon[103389]: osdmap e47: 8 total, 7 up, 8 in 2023-10-08T09:10:54.000 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:54.000 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-mon[103389]: from='osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:10:54.000 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:53 smithi132 ceph-mon[103389]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:10:54.141 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi184:vg_nvme/lv_2 2023-10-08T09:10:54.699 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:54.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:54.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:10:54.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T09:10:54.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:54.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:54.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:54.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: osdmap e48: 8 total, 7 up, 8 in 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.937 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:10:54.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:54.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:54.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:54.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:55.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:10:55.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:10:55.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi132 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi132 to 5722M 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] boot 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: osdmap e49: 8 total, 8 up, 8 in 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:55.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi132 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi132 to 5722M 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] boot 2023-10-08T09:10:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: osdmap e49: 8 total, 8 up, 8 in 2023-10-08T09:10:55.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:55.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:55.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:55.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:55.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:55.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 -- 172.21.15.184:0/2941206205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 msgr2=0x7f54c00ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:55.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/2941206205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c00ffb50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009990 tx=0x7f54b4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:10:55.885 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 -- 172.21.15.184:0/2941206205 shutdown_connections 2023-10-08T09:10:55.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/2941206205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c00ffb50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:55.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/2941206205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54c01012d0 0x7f54c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:55.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/2941206205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c0100930 0x7f54c0100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:55.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 -- 172.21.15.184:0/2941206205 >> 172.21.15.184:0/2941206205 conn(0x7f54c00facf0 msgr2=0x7f54c00fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:10:55.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 -- 172.21.15.184:0/2941206205 shutdown_connections 2023-10-08T09:10:55.886 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.885+0000 7f54c4cd7700 1 -- 172.21.15.184:0/2941206205 wait complete. 2023-10-08T09:10:55.888 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 Processor -- start 2023-10-08T09:10:55.888 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 -- start start 2023-10-08T09:10:55.888 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:55.888 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54c0100930 0x7f54c0198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:55.888 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c01012d0 0x7f54c019ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:55.888 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c01a0520 con 0x7f54c0100930 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c01a0660 con 0x7f54c01012d0 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54c4cd7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c01a07f0 con 0x7f54c00ff730 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c01989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59690/0 (socket says 172.21.15.184:59690) 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 -- 172.21.15.184:0/1272558959 learned_addr learned my addr 172.21.15.184:0/1272558959 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54bdd9b700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54c0100930 0x7f54c0198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 -- 172.21.15.184:0/1272558959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c01012d0 msgr2=0x7f54c019ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54bed9d700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c01012d0 0x7f54c019ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c01012d0 0x7f54c019ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 -- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54c0100930 msgr2=0x7f54c0198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:10:55.889 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54c0100930 0x7f54c0198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54be59c700 1 -- 172.21.15.184:0/1272558959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40095f0 con 0x7f54c00ff730 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.886+0000 7f54bdd9b700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54c0100930 0x7f54c0198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.887+0000 7f54be59c700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c01989e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54b4000c00 tx=0x7f54b400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.887+0000 7f54af7fe700 1 -- 172.21.15.184:0/1272558959 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b401d070 con 0x7f54c00ff730 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.887+0000 7f54af7fe700 1 -- 172.21.15.184:0/1272558959 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b4003e20 con 0x7f54c00ff730 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.887+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c01a0a70 con 0x7f54c00ff730 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.887+0000 7f54af7fe700 1 -- 172.21.15.184:0/1272558959 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b40056c0 con 0x7f54c00ff730 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.887+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c01a0ee0 con 0x7f54c00ff730 2023-10-08T09:10:55.890 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.888+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c004e4c0 con 0x7f54c00ff730 2023-10-08T09:10:55.891 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.889+0000 7f54af7fe700 1 -- 172.21.15.184:0/1272558959 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f54b400fc20 con 0x7f54c00ff730 2023-10-08T09:10:55.891 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.890+0000 7f54af7fe700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54a80a1500 0x7f54a80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:10:55.891 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.890+0000 7f54af7fe700 1 -- 172.21.15.184:0/1272558959 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f54b40cd3c0 con 0x7f54c00ff730 2023-10-08T09:10:55.891 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.890+0000 7f54bdd9b700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54a80a1500 0x7f54a80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:10:55.891 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.890+0000 7f54bdd9b700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54a80a1500 0x7f54a80a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54b00099d0 tx=0x7f54b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:10:55.893 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:55.893+0000 7f54af7fe700 1 -- 172.21.15.184:0/1272558959 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f54b40975f0 con 0x7f54c00ff730 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi132 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi132 to 5722M 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: osd.7 [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] boot 2023-10-08T09:10:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: osdmap e49: 8 total, 8 up, 8 in 2023-10-08T09:10:56.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-08T09:10:56.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:10:56.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:56.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:10:56.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:10:56.089 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:10:56.089+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f54c00688d0 con 0x7f54a80a1500 2023-10-08T09:10:56.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:56 smithi049 ceph-mon[99485]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T09:10:56.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:56 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:56.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:56 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:56.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:56 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:56.928 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:56 smithi184 ceph-mon[103675]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T09:10:56.928 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:56 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:56.928 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:56 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:56.928 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:56 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:57.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:56 smithi132 ceph-mon[103389]: osdmap e50: 8 total, 8 up, 8 in 2023-10-08T09:10:57.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:56 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:10:57.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:56 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:10:57.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:56 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:10:57.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:57 smithi049 ceph-mon[99485]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:10:57.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:57 smithi049 ceph-mon[99485]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:57.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:57 smithi049 ceph-mon[99485]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T09:10:57.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:57 smithi184 ceph-mon[103675]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:10:57.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:57 smithi184 ceph-mon[103675]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:57.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:57 smithi184 ceph-mon[103675]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T09:10:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:57 smithi132 ceph-mon[103389]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:10:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:57 smithi132 ceph-mon[103389]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:10:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:57 smithi132 ceph-mon[103389]: osdmap e51: 8 total, 8 up, 8 in 2023-10-08T09:10:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:58 smithi049 ceph-mon[99485]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T09:10:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:58 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/2525861184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]: dispatch 2023-10-08T09:10:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:58 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]: dispatch 2023-10-08T09:10:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:58 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]': finished 2023-10-08T09:10:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:58 smithi049 ceph-mon[99485]: osdmap e53: 9 total, 8 up, 9 in 2023-10-08T09:10:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:58 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:10:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:58 smithi184 ceph-mon[103675]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T09:10:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:58 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/2525861184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]: dispatch 2023-10-08T09:10:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:58 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]: dispatch 2023-10-08T09:10:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:58 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]': finished 2023-10-08T09:10:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:58 smithi184 ceph-mon[103675]: osdmap e53: 9 total, 8 up, 9 in 2023-10-08T09:10:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:58 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:10:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:58 smithi132 ceph-mon[103389]: osdmap e52: 8 total, 8 up, 8 in 2023-10-08T09:10:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:58 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/2525861184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]: dispatch 2023-10-08T09:10:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:58 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]: dispatch 2023-10-08T09:10:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:58 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42"}]': finished 2023-10-08T09:10:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:58 smithi132 ceph-mon[103389]: osdmap e53: 9 total, 8 up, 9 in 2023-10-08T09:10:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:58 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:10:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:59 smithi049 ceph-mon[99485]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:10:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:10:59 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/1962896172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:10:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:59 smithi184 ceph-mon[103675]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:10:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:10:59 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/1962896172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:00.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:59 smithi132 ceph-mon[103389]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:00.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:10:59 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/1962896172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:01.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:01 smithi184 ceph-mon[103675]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:02.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:01 smithi132 ceph-mon[103389]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:02.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:01 smithi049 ceph-mon[99485]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:03.904 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:03 smithi184 ceph-mon[103675]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:04.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:03 smithi132 ceph-mon[103389]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:04.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:03 smithi049 ceph-mon[99485]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:05.837 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:05 smithi184 ceph-mon[103675]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:05.838 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:05 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T09:11:05.838 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:05 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:05 smithi132 ceph-mon[103389]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:05 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T09:11:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:05 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:05 smithi049 ceph-mon[99485]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-08T09:11:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:05 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:06 smithi184 ceph-mon[103675]: Deploying daemon osd.8 on smithi184 2023-10-08T09:11:07.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:06 smithi132 ceph-mon[103389]: Deploying daemon osd.8 on smithi184 2023-10-08T09:11:07.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:06 smithi049 ceph-mon[99485]: Deploying daemon osd.8 on smithi184 2023-10-08T09:11:07.915 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:07 smithi184 ceph-mon[103675]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:07 smithi132 ceph-mon[103389]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:07 smithi049 ceph-mon[99485]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:09 smithi184 ceph-mon[103675]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:09 smithi132 ceph-mon[103389]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:09 smithi049 ceph-mon[99485]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:11.833 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:11 smithi184 ceph-mon[103675]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:11.834 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:11.834 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:11.834 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:11.834 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:11.834 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:11 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:11.901 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 8 on host 'smithi184' 2023-10-08T09:11:11.902 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.900+0000 7f54af7fe700 1 -- 172.21.15.184:0/1272558959 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f54c00688d0 con 0x7f54a80a1500 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54a80a1500 msgr2=0x7f54a80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54a80a1500 0x7f54a80a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54b00099d0 tx=0x7f54b0008040 comp rx=0 tx=0).stop 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 msgr2=0x7f54c01989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c01989e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f54b4000c00 tx=0x7f54b400f740 comp rx=0 tx=0).stop 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 shutdown_connections 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54a80a1500 0x7f54a80a39c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54c00ff730 0x7f54c01989e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c01012d0 0x7f54c019ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 --2- 172.21.15.184:0/1272558959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54c0100930 0x7f54c0198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:11.904 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 >> 172.21.15.184:0/1272558959 conn(0x7f54c00facf0 msgr2=0x7f54c00681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:11:11.905 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 shutdown_connections 2023-10-08T09:11:11.905 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:11.902+0000 7f54c4cd7700 1 -- 172.21.15.184:0/1272558959 wait complete. 2023-10-08T09:11:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:11 smithi132 ceph-mon[103389]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:11 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:11 smithi049 ceph-mon[99485]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:11 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:12.274 DEBUG:teuthology.orchestra.run.smithi184:osd.8> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.8.service 2023-10-08T09:11:12.276 INFO:tasks.cephadm:Deploying osd.9 on smithi049 with /dev/vg_nvme/lv_1... 2023-10-08T09:11:12.276 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T09:11:12.305 INFO:journalctl@ceph.osd.8.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:11:12.857 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:11:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:12 smithi049 ceph-mon[99485]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:12 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:13.145 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:12 smithi184 ceph-mon[103675]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:13.145 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:13.145 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:13.145 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:12 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:13.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:12 smithi132 ceph-mon[103389]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:13.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:13.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:13.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:12 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:13.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:11:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:11:13.208+0000 7f3c40701640 -1 osd.8 0 log_to_monitors true 2023-10-08T09:11:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:13 smithi184 ceph-mon[103675]: from='osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:11:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:13 smithi184 ceph-mon[103675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:11:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:13 smithi049 ceph-mon[99485]: from='osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:11:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:13 smithi049 ceph-mon[99485]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:11:14.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:13 smithi132 ceph-mon[103389]: from='osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:11:14.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:13 smithi132 ceph-mon[103389]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-08T09:11:14.445 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T09:11:14.445 INFO:teuthology.orchestra.run.smithi049.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T09:11:14.445 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 10+0 records in 2023-10-08T09:11:14.445 INFO:teuthology.orchestra.run.smithi049.stderr:10+0 records out 2023-10-08T09:11:14.446 INFO:teuthology.orchestra.run.smithi049.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00808673 s, 1.3 GB/s 2023-10-08T09:11:14.446 INFO:teuthology.orchestra.run.smithi049.stderr: stderr: 2023-10-08T09:11:14.446 INFO:teuthology.orchestra.run.smithi049.stderr:--> Zapping successful for: 2023-10-08T09:11:14.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi049:vg_nvme/lv_1 2023-10-08T09:11:14.882 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-mon[103675]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:14.882 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-mon[103675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T09:11:14.882 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-mon[103675]: osdmap e54: 9 total, 8 up, 9 in 2023-10-08T09:11:14.883 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:14.883 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-mon[103675]: from='osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:11:14.883 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-mon[103675]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:11:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:14 smithi049 ceph-mon[99485]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:14 smithi049 ceph-mon[99485]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T09:11:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:14 smithi049 ceph-mon[99485]: osdmap e54: 9 total, 8 up, 9 in 2023-10-08T09:11:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:14 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:14 smithi049 ceph-mon[99485]: from='osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:11:15.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:14 smithi049 ceph-mon[99485]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:11:15.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:11:14.884+0000 7f3c35d78700 -1 osd.8 0 waiting for initial osdmap 2023-10-08T09:11:15.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:11:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:11:14.893+0000 7f3c2fb69700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:11:15.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:14 smithi132 ceph-mon[103389]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:15.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:14 smithi132 ceph-mon[103389]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-08T09:11:15.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:14 smithi132 ceph-mon[103389]: osdmap e54: 9 total, 8 up, 9 in 2023-10-08T09:11:15.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:14 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:15.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:14 smithi132 ceph-mon[103389]: from='osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:11:15.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:14 smithi132 ceph-mon[103389]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:11:15.394 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:11:16.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:11:16.151 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi184 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi184 to 5722M 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.152 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:16.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:16.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.153 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:11:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T09:11:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi184 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi184 to 5722M 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:11:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: osdmap e55: 9 total, 8 up, 9 in 2023-10-08T09:11:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi184 2023-10-08T09:11:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi184 to 5722M 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 -- 172.21.15.49:0/795427506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a4100930 msgr2=0x7f39a4100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 --2- 172.21.15.49:0/795427506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a4100930 0x7f39a4100d90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f39980099c0 tx=0x7f3998009cd0 comp rx=0 tx=0).stop 2023-10-08T09:11:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 -- 172.21.15.49:0/795427506 shutdown_connections 2023-10-08T09:11:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 --2- 172.21.15.49:0/795427506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39a41012d0 0x7f39a4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 --2- 172.21.15.49:0/795427506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a40ff730 0x7f39a40ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 --2- 172.21.15.49:0/795427506 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a4100930 0x7f39a4100d90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 -- 172.21.15.49:0/795427506 >> 172.21.15.49:0/795427506 conn(0x7f39a40facf0 msgr2=0x7f39a40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 -- 172.21.15.49:0/795427506 shutdown_connections 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.656+0000 7f39ab275700 1 -- 172.21.15.49:0/795427506 wait complete. 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 Processor -- start 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 -- start start 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a40ff730 0x7f39a41956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39a4100930 0x7f39a4195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:16.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a41012d0 0x7f39a419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a419d220 con 0x7f39a40ff730 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a419d360 con 0x7f39a41012d0 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39ab275700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a4068a20 con 0x7f39a4100930 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39a9011700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a40ff730 0x7f39a41956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39a9011700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a40ff730 0x7f39a41956e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47294/0 (socket says 172.21.15.49:47294) 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39a9011700 1 -- 172.21.15.49:0/15671419 learned_addr learned my addr 172.21.15.49:0/15671419 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39a8810700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39a4100930 0x7f39a4195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.657+0000 7f39a9812700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a41012d0 0x7f39a419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39a9011700 1 -- 172.21.15.49:0/15671419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39a4100930 msgr2=0x7f39a4195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39a9011700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39a4100930 0x7f39a4195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39a9011700 1 -- 172.21.15.49:0/15671419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a41012d0 msgr2=0x7f39a419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39a9011700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a41012d0 0x7f39a419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39a9011700 1 -- 172.21.15.49:0/15671419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980095f0 con 0x7f39a40ff730 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39a9812700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a41012d0 0x7f39a419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39a9011700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a40ff730 0x7f39a41956e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39a000b530 tx=0x7f39a000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39967fc700 1 -- 172.21.15.49:0/15671419 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0011830 con 0x7f39a40ff730 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39967fc700 1 -- 172.21.15.49:0/15671419 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a0011e50 con 0x7f39a40ff730 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a4068d00 con 0x7f39a40ff730 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39967fc700 1 -- 172.21.15.49:0/15671419 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a000f2e0 con 0x7f39a40ff730 2023-10-08T09:11:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.658+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a4069250 con 0x7f39a40ff730 2023-10-08T09:11:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.659+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a4106c10 con 0x7f39a40ff730 2023-10-08T09:11:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.660+0000 7f39967fc700 1 -- 172.21.15.49:0/15671419 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f39a0010580 con 0x7f39a40ff730 2023-10-08T09:11:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.661+0000 7f39967fc700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f39900a1500 0x7f39900a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.661+0000 7f39a8810700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f39900a1500 0x7f39900a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.661+0000 7f39967fc700 1 -- 172.21.15.49:0/15671419 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f39a00cc370 con 0x7f39a40ff730 2023-10-08T09:11:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.661+0000 7f39a8810700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f39900a1500 0x7f39900a39c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3998009990 tx=0x7f3998005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:11:16.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.665+0000 7f39967fc700 1 -- 172.21.15.49:0/15671419 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f39a00962c0 con 0x7f39a40ff730 2023-10-08T09:11:16.878 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:16.876+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f39a40694d0 con 0x7f39900a1500 2023-10-08T09:11:17.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:11:17.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:11:17.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:17.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] boot 2023-10-08T09:11:17.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: osdmap e56: 9 total, 9 up, 9 in 2023-10-08T09:11:17.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:17.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: osdmap e57: 9 total, 9 up, 9 in 2023-10-08T09:11:17.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:11:17.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:11:17.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] boot 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: osdmap e56: 9 total, 9 up, 9 in 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: osdmap e57: 9 total, 9 up, 9 in 2023-10-08T09:11:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:11:17.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:11:17.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:11:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:11:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-10-08T09:11:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: osd.8 [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] boot 2023-10-08T09:11:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: osdmap e56: 9 total, 9 up, 9 in 2023-10-08T09:11:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-08T09:11:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: osdmap e57: 9 total, 9 up, 9 in 2023-10-08T09:11:17.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:11:17.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:11:17.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:18.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:18 smithi049 ceph-mon[99485]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:11:18.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:18 smithi049 ceph-mon[99485]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T09:11:18.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:18 smithi184 ceph-mon[103675]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:11:18.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:18 smithi184 ceph-mon[103675]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T09:11:18.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:18 smithi132 ceph-mon[103389]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi049:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:11:18.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:18 smithi132 ceph-mon[103389]: osdmap e58: 9 total, 9 up, 9 in 2023-10-08T09:11:19.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:19 smithi049 ceph-mon[99485]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:19.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:19 smithi049 ceph-mon[99485]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T09:11:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:19 smithi184 ceph-mon[103675]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:19 smithi184 ceph-mon[103675]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T09:11:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:19 smithi132 ceph-mon[103389]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:19 smithi132 ceph-mon[103389]: osdmap e59: 9 total, 9 up, 9 in 2023-10-08T09:11:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:20 smithi184 ceph-mon[103675]: osdmap e60: 9 total, 9 up, 9 in 2023-10-08T09:11:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:20 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/879088364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1a0dc66-f039-4112-93d3-73f6f3a6869e"}]: dispatch 2023-10-08T09:11:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:20 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/879088364' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1a0dc66-f039-4112-93d3-73f6f3a6869e"}]': finished 2023-10-08T09:11:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:20 smithi184 ceph-mon[103675]: osdmap e61: 10 total, 9 up, 10 in 2023-10-08T09:11:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:20 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3167412531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:20 smithi132 ceph-mon[103389]: osdmap e60: 9 total, 9 up, 9 in 2023-10-08T09:11:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:20 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/879088364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1a0dc66-f039-4112-93d3-73f6f3a6869e"}]: dispatch 2023-10-08T09:11:20.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:20 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/879088364' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1a0dc66-f039-4112-93d3-73f6f3a6869e"}]': finished 2023-10-08T09:11:20.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:20 smithi132 ceph-mon[103389]: osdmap e61: 10 total, 9 up, 10 in 2023-10-08T09:11:20.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:20.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:20 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3167412531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:20.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:20 smithi049 ceph-mon[99485]: osdmap e60: 9 total, 9 up, 9 in 2023-10-08T09:11:20.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:20 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/879088364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1a0dc66-f039-4112-93d3-73f6f3a6869e"}]: dispatch 2023-10-08T09:11:20.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:20 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/879088364' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1a0dc66-f039-4112-93d3-73f6f3a6869e"}]': finished 2023-10-08T09:11:20.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:20 smithi049 ceph-mon[99485]: osdmap e61: 10 total, 9 up, 10 in 2023-10-08T09:11:20.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:20.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:20 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3167412531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:21 smithi132 ceph-mon[103389]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:21 smithi049 ceph-mon[99485]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:21 smithi184 ceph-mon[103675]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:23.596 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:23 smithi049 ceph-mon[99485]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-08T09:11:23.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:23 smithi132 ceph-mon[103389]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-08T09:11:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:23 smithi184 ceph-mon[103675]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-10-08T09:11:25.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:25 smithi132 ceph-mon[103389]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:11:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:25 smithi049 ceph-mon[99485]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:11:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:25 smithi184 ceph-mon[103675]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-08T09:11:27.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:27 smithi049 ceph-mon[99485]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-10-08T09:11:27.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T09:11:27.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:27.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:27 smithi049 ceph-mon[99485]: Deploying daemon osd.9 on smithi049 2023-10-08T09:11:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:27 smithi184 ceph-mon[103675]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-10-08T09:11:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T09:11:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:27 smithi184 ceph-mon[103675]: Deploying daemon osd.9 on smithi049 2023-10-08T09:11:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:27 smithi132 ceph-mon[103389]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-10-08T09:11:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-08T09:11:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:27 smithi132 ceph-mon[103389]: Deploying daemon osd.9 on smithi049 2023-10-08T09:11:29.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:29 smithi049 ceph-mon[99485]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-08T09:11:29.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:29 smithi184 ceph-mon[103675]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-08T09:11:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:29 smithi132 ceph-mon[103389]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-10-08T09:11:31.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:31 smithi049 ceph-mon[99485]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-10-08T09:11:31.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:31 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:31.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:31 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:31.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:31 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:31 smithi184 ceph-mon[103675]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-10-08T09:11:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:31 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:31 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:31 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:31 smithi132 ceph-mon[103389]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2023-10-08T09:11:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:31 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:33 smithi049 ceph-mon[99485]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:11:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:33 smithi184 ceph-mon[103675]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:11:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:33 smithi132 ceph-mon[103389]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:11:34.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.086+0000 7f39967fc700 1 -- 172.21.15.49:0/15671419 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f39a40694d0 con 0x7f39900a1500 2023-10-08T09:11:34.086 INFO:teuthology.orchestra.run.smithi049.stdout:Created osd(s) 9 on host 'smithi049' 2023-10-08T09:11:34.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f39900a1500 msgr2=0x7f39900a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:34.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f39900a1500 0x7f39900a39c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3998009990 tx=0x7f3998005bc0 comp rx=0 tx=0).stop 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a40ff730 msgr2=0x7f39a41956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a40ff730 0x7f39a41956e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39a000b530 tx=0x7f39a000b840 comp rx=0 tx=0).stop 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 shutdown_connections 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f39900a1500 0x7f39900a39c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39a41012d0 0x7f39a419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f39a40ff730 0x7f39a41956e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 --2- 172.21.15.49:0/15671419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39a4100930 0x7f39a4195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 >> 172.21.15.49:0/15671419 conn(0x7f39a40facf0 msgr2=0x7f39a4103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 shutdown_connections 2023-10-08T09:11:34.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:11:34.088+0000 7f39ab275700 1 -- 172.21.15.49:0/15671419 wait complete. 2023-10-08T09:11:34.510 DEBUG:teuthology.orchestra.run.smithi049:osd.9> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.9.service 2023-10-08T09:11:34.512 INFO:tasks.cephadm:Deploying osd.10 on smithi132 with /dev/vg_nvme/lv_1... 2023-10-08T09:11:34.512 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T09:11:34.542 INFO:journalctl@ceph.osd.9.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:11:35.081 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:11:35.316 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:35 smithi132 ceph-mon[103389]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:35.316 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.316 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.316 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.316 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.325 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:35 smithi049 ceph-mon[99485]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:35.325 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.325 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.325 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.325 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.326 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:11:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:11:35.187+0000 7fa34e9fa640 -1 osd.9 0 log_to_monitors true 2023-10-08T09:11:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:35 smithi184 ceph-mon[103675]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:36.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:36 smithi049 ceph-mon[99485]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T09:11:36.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:36 smithi184 ceph-mon[103675]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T09:11:36.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:36 smithi132 ceph-mon[103389]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-08T09:11:36.590 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T09:11:36.590 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T09:11:36.590 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2023-10-08T09:11:36.590 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2023-10-08T09:11:36.590 INFO:teuthology.orchestra.run.smithi132.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011107 s, 944 MB/s 2023-10-08T09:11:36.590 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2023-10-08T09:11:37.037 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi132:vg_nvme/lv_1 2023-10-08T09:11:37.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:37 smithi132 ceph-mon[103389]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:37.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:37 smithi132 ceph-mon[103389]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T09:11:37.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:37 smithi132 ceph-mon[103389]: osdmap e62: 10 total, 9 up, 10 in 2023-10-08T09:11:37.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:37 smithi132 ceph-mon[103389]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:11:37.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:37.353 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:11:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:11:37.077+0000 7fa344071700 -1 osd.9 0 waiting for initial osdmap 2023-10-08T09:11:37.354 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:11:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:11:37.090+0000 7fa33de62700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:11:37.354 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:37 smithi049 ceph-mon[99485]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:37.354 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:37 smithi049 ceph-mon[99485]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T09:11:37.354 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:37 smithi049 ceph-mon[99485]: osdmap e62: 10 total, 9 up, 10 in 2023-10-08T09:11:37.354 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:37 smithi049 ceph-mon[99485]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:11:37.354 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:37 smithi184 ceph-mon[103675]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:37 smithi184 ceph-mon[103675]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-08T09:11:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:37 smithi184 ceph-mon[103675]: osdmap e62: 10 total, 9 up, 10 in 2023-10-08T09:11:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:37 smithi184 ceph-mon[103675]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]: dispatch 2023-10-08T09:11:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:37.498 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:11:38.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:11:38.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: osdmap e63: 10 total, 9 up, 10 in 2023-10-08T09:11:38.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi049 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi049 to 4292M 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:38.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:38.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:38.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:38.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:11:38.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: osdmap e63: 10 total, 9 up, 10 in 2023-10-08T09:11:38.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi049 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi049 to 4292M 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:38.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:38.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:38.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi049", "root=default"]}]': finished 2023-10-08T09:11:38.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: osdmap e63: 10 total, 9 up, 10 in 2023-10-08T09:11:38.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi049 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi049 to 4292M 2023-10-08T09:11:38.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:38.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:38.716 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.712+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/1819082226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 msgr2=0x7fa9b4108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:38.716 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.712+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/1819082226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4108df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b0a0 tx=0x7fa9ac00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:11:38.716 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/1819082226 shutdown_connections 2023-10-08T09:11:38.716 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/1819082226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9b40feb20 0x7fa9b40fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/1819082226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4108df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/1819082226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9b4101ba0 0x7fa9b4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/1819082226 >> 172.21.15.132:0/1819082226 conn(0x7fa9b40fa480 msgr2=0x7fa9b40fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/1819082226 shutdown_connections 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/1819082226 wait complete. 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 Processor -- start 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- start start 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9b40feb20 0x7fa9b4101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9b4101ba0 0x7fa9b40ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4100090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:38.717 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4100770 con 0x7fa9b4101ba0 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41008e0 con 0x7fa9b4102540 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9ba8a2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41a09c0 con 0x7fa9b40feb20 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.713+0000 7fa9b8e3f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4100090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4100090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:49146/0 (socket says 172.21.15.132:49146) 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 -- 172.21.15.132:0/4219327791 learned_addr learned my addr 172.21.15.132:0/4219327791 (peer_addr_for_me v2:172.21.15.132:0/0) 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b3fff700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9b40feb20 0x7fa9b4101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9abfff700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9b4101ba0 0x7fa9b40ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 -- 172.21.15.132:0/4219327791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9b40feb20 msgr2=0x7fa9b4101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9b40feb20 0x7fa9b4101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 -- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9b4101ba0 msgr2=0x7fa9b40ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9b4101ba0 0x7fa9b40ffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:38.718 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 -- 172.21.15.132:0/4219327791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac009d00 con 0x7fa9b4102540 2023-10-08T09:11:38.719 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b8e3f700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4100090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac004d00 tx=0x7fa9ac004de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:11:38.719 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b1ffb700 1 -- 172.21.15.132:0/4219327791 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00e040 con 0x7fa9b4102540 2023-10-08T09:11:38.719 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b1ffb700 1 -- 172.21.15.132:0/4219327791 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9ac009440 con 0x7fa9b4102540 2023-10-08T09:11:38.719 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b41a0bc0 con 0x7fa9b4102540 2023-10-08T09:11:38.719 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9b1ffb700 1 -- 172.21.15.132:0/4219327791 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac01ba80 con 0x7fa9b4102540 2023-10-08T09:11:38.719 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.714+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b41a1030 con 0x7fa9b4102540 2023-10-08T09:11:38.719 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.715+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b4107d60 con 0x7fa9b4102540 2023-10-08T09:11:38.720 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.716+0000 7fa9b1ffb700 1 -- 172.21.15.132:0/4219327791 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa9ac01bbe0 con 0x7fa9b4102540 2023-10-08T09:11:38.720 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.717+0000 7fa9b1ffb700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa9940a1500 0x7fa9940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:38.720 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.717+0000 7fa9b1ffb700 1 -- 172.21.15.132:0/4219327791 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fa9ac0ce310 con 0x7fa9b4102540 2023-10-08T09:11:38.720 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.717+0000 7fa9b3fff700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa9940a1500 0x7fa9940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:38.720 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.717+0000 7fa9b3fff700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa9940a1500 0x7fa9940a39c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:11:38.722 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.721+0000 7fa9b1ffb700 1 -- 172.21.15.132:0/4219327791 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa9ac098200 con 0x7fa9b4102540 2023-10-08T09:11:38.919 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:38.918+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9b402d070 con 0x7fa9940a1500 2023-10-08T09:11:39.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:11:39.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:11:39.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:39.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] boot 2023-10-08T09:11:39.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: osdmap e64: 10 total, 10 up, 10 in 2023-10-08T09:11:39.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:39.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:11:39.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:11:39.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:39.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:11:39.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:11:39.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:39.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] boot 2023-10-08T09:11:39.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: osdmap e64: 10 total, 10 up, 10 in 2023-10-08T09:11:39.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:39.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:11:39.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:11:39.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:39.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:11:39.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:11:39.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-10-08T09:11:39.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: osd.9 [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] boot 2023-10-08T09:11:39.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: osdmap e64: 10 total, 10 up, 10 in 2023-10-08T09:11:39.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-08T09:11:39.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:11:39.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:11:39.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:40.296 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:40 smithi132 ceph-mon[103389]: from='client.24464 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:11:40.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:40 smithi132 ceph-mon[103389]: osdmap e65: 10 total, 10 up, 10 in 2023-10-08T09:11:40.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:40 smithi049 ceph-mon[99485]: from='client.24464 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:11:40.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:40 smithi049 ceph-mon[99485]: osdmap e65: 10 total, 10 up, 10 in 2023-10-08T09:11:40.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:40 smithi184 ceph-mon[103675]: from='client.24464 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:11:40.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:40 smithi184 ceph-mon[103675]: osdmap e65: 10 total, 10 up, 10 in 2023-10-08T09:11:41.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:41 smithi049 ceph-mon[99485]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:41.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:41 smithi184 ceph-mon[103675]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:41.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:41 smithi132 ceph-mon[103389]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:42 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/807783428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]: dispatch 2023-10-08T09:11:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:42 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]: dispatch 2023-10-08T09:11:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:42 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]': finished 2023-10-08T09:11:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:42 smithi049 ceph-mon[99485]: osdmap e66: 11 total, 10 up, 11 in 2023-10-08T09:11:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:42 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:42 smithi049 ceph-mon[99485]: from='client.? 172.21.15.132:0/1423636372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:42 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/807783428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]: dispatch 2023-10-08T09:11:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:42 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]: dispatch 2023-10-08T09:11:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:42 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]': finished 2023-10-08T09:11:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:42 smithi184 ceph-mon[103675]: osdmap e66: 11 total, 10 up, 11 in 2023-10-08T09:11:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:42 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:42 smithi184 ceph-mon[103675]: from='client.? 172.21.15.132:0/1423636372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:42 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/807783428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]: dispatch 2023-10-08T09:11:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:42 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]: dispatch 2023-10-08T09:11:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:42 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f90e2ab-e7e8-42b3-966b-7c0d40a89925"}]': finished 2023-10-08T09:11:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:42 smithi132 ceph-mon[103389]: osdmap e66: 11 total, 10 up, 11 in 2023-10-08T09:11:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:42 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:42 smithi132 ceph-mon[103389]: from='client.? 172.21.15.132:0/1423636372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:11:43.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:43 smithi049 ceph-mon[99485]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:43.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:43 smithi184 ceph-mon[103675]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:43.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:43 smithi132 ceph-mon[103389]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:45.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:45 smithi049 ceph-mon[99485]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:45 smithi184 ceph-mon[103675]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:45 smithi132 ceph-mon[103389]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:47.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:47 smithi049 ceph-mon[99485]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:47.640 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:47 smithi132 ceph-mon[103389]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:47 smithi184 ceph-mon[103675]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:48.729 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T09:11:48.729 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T09:11:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-08T09:11:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:49.730 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:49 smithi132 ceph-mon[103389]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:49.731 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:49 smithi132 ceph-mon[103389]: Deploying daemon osd.10 on smithi132 2023-10-08T09:11:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:49 smithi184 ceph-mon[103675]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:49.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:49 smithi184 ceph-mon[103675]: Deploying daemon osd.10 on smithi132 2023-10-08T09:11:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:49 smithi049 ceph-mon[99485]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:49 smithi049 ceph-mon[99485]: Deploying daemon osd.10 on smithi132 2023-10-08T09:11:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:51 smithi184 ceph-mon[103675]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:51 smithi132 ceph-mon[103389]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:51 smithi049 ceph-mon[99485]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:53.647 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:53 smithi132 ceph-mon[103389]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:53.648 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:53.648 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:53.648 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:53 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:53 smithi049 ceph-mon[99485]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:53 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:53.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:53 smithi184 ceph-mon[103675]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:53.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:53.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:53.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:53 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:55.339 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.338+0000 7fa9b1ffb700 1 -- 172.21.15.132:0/4219327791 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fa9b402d070 con 0x7fa9940a1500 2023-10-08T09:11:55.339 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 10 on host 'smithi132' 2023-10-08T09:11:55.341 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa9940a1500 msgr2=0x7fa9940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:55.341 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa9940a1500 0x7fa9940a39c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c008040 comp rx=0 tx=0).stop 2023-10-08T09:11:55.341 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 msgr2=0x7fa9b4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:55.341 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4100090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac004d00 tx=0x7fa9ac004de0 comp rx=0 tx=0).stop 2023-10-08T09:11:55.342 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 shutdown_connections 2023-10-08T09:11:55.342 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa9940a1500 0x7fa9940a39c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:55.342 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa9b40feb20 0x7fa9b4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:55.342 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9b4102540 0x7fa9b4100090 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:55.342 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 --2- 172.21.15.132:0/4219327791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9b4101ba0 0x7fa9b40ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:55.343 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 >> 172.21.15.132:0/4219327791 conn(0x7fa9b40fa480 msgr2=0x7fa9b4104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:11:55.343 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 shutdown_connections 2023-10-08T09:11:55.343 INFO:teuthology.orchestra.run.smithi132.stderr:2023-10-08T09:11:55.340+0000 7fa9ba8a2700 1 -- 172.21.15.132:0/4219327791 wait complete. 2023-10-08T09:11:55.773 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:55.773 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.773 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:55.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:55.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:55.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.774 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:55 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.789 DEBUG:teuthology.orchestra.run.smithi132:osd.10> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.10.service 2023-10-08T09:11:55.792 INFO:tasks.cephadm:Deploying osd.11 on smithi184 with /dev/vg_nvme/lv_1... 2023-10-08T09:11:55.792 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-08T09:11:55.821 INFO:journalctl@ceph.osd.10.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.825 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:55 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:11:55.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:55.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:55 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:56.366 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:11:56.747 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:11:56 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:11:56.443+0000 7f7305a14640 -1 osd.10 0 log_to_monitors true 2023-10-08T09:11:57.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:57 smithi132 ceph-mon[103389]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:57.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:57 smithi132 ceph-mon[103389]: from='osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:11:57.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:57 smithi132 ceph-mon[103389]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:11:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:57 smithi049 ceph-mon[99485]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:57 smithi049 ceph-mon[99485]: from='osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:11:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:57 smithi049 ceph-mon[99485]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:11:57.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:57 smithi184 ceph-mon[103675]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:57 smithi184 ceph-mon[103675]: from='osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:11:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:57 smithi184 ceph-mon[103675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-08T09:11:57.846 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-08T09:11:57.847 INFO:teuthology.orchestra.run.smithi184.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-08T09:11:57.847 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10+0 records in 2023-10-08T09:11:57.847 INFO:teuthology.orchestra.run.smithi184.stderr:10+0 records out 2023-10-08T09:11:57.847 INFO:teuthology.orchestra.run.smithi184.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111779 s, 938 MB/s 2023-10-08T09:11:57.847 INFO:teuthology.orchestra.run.smithi184.stderr:--> Zapping successful for: 2023-10-08T09:11:58.207 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch daemon add osd smithi184:vg_nvme/lv_1 2023-10-08T09:11:58.403 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:11:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:11:58.390+0000 7f72fb08b700 -1 osd.10 0 waiting for initial osdmap 2023-10-08T09:11:58.403 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:58 smithi132 ceph-mon[103389]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T09:11:58.403 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:58 smithi132 ceph-mon[103389]: from='osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:11:58.403 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:58 smithi132 ceph-mon[103389]: osdmap e67: 11 total, 10 up, 11 in 2023-10-08T09:11:58.403 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:58 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:58.403 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:58 smithi132 ceph-mon[103389]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:11:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:58 smithi184 ceph-mon[103675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T09:11:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:58 smithi184 ceph-mon[103675]: from='osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:11:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:58 smithi184 ceph-mon[103675]: osdmap e67: 11 total, 10 up, 11 in 2023-10-08T09:11:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:58 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:58 smithi184 ceph-mon[103675]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:11:58.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:11:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:11:58.402+0000 7f72f3e7a700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:11:58.806 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:11:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:58 smithi049 ceph-mon[99485]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-08T09:11:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:58 smithi049 ceph-mon[99485]: from='osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:11:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:58 smithi049 ceph-mon[99485]: osdmap e67: 11 total, 10 up, 11 in 2023-10-08T09:11:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:58 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:58.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:58 smithi049 ceph-mon[99485]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2023-10-08T09:11:59.574 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:59.574 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: osdmap e68: 11 total, 10 up, 11 in 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi132 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.575 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi132 to 4292M 2023-10-08T09:11:59.576 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.576 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:59.576 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:59.576 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:11:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:11:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: osdmap e68: 11 total, 10 up, 11 in 2023-10-08T09:11:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi132 2023-10-08T09:11:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi132 to 4292M 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:59.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:11:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: osdmap e68: 11 total, 10 up, 11 in 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi132 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:11:59.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi132 to 4292M 2023-10-08T09:11:59.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:11:59.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:11:59.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:11:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:11:59.966 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.964+0000 7f001eda9700 1 -- 172.21.15.184:0/3507020122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0018100840 msgr2=0x7f0018100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:59.966 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.964+0000 7f001eda9700 1 --2- 172.21.15.184:0/3507020122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0018100840 0x7f0018100c60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f000c009990 tx=0x7f000c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:11:59.966 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.964+0000 7f001eda9700 1 -- 172.21.15.184:0/3507020122 shutdown_connections 2023-10-08T09:11:59.966 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.964+0000 7f001eda9700 1 --2- 172.21.15.184:0/3507020122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0018100840 0x7f0018100c60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:59.966 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.964+0000 7f001eda9700 1 --2- 172.21.15.184:0/3507020122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f00181023e0 0x7f0018108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:59.966 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.964+0000 7f001eda9700 1 --2- 172.21.15.184:0/3507020122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018101a40 0x7f0018101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:59.967 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.964+0000 7f001eda9700 1 -- 172.21.15.184:0/3507020122 >> 172.21.15.184:0/3507020122 conn(0x7f00180fbe00 msgr2=0x7f00180fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:11:59.967 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 -- 172.21.15.184:0/3507020122 shutdown_connections 2023-10-08T09:11:59.967 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 -- 172.21.15.184:0/3507020122 wait complete. 2023-10-08T09:11:59.968 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 Processor -- start 2023-10-08T09:11:59.968 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 -- start start 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100840 0x7f0018198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0018101a40 0x7f0018198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00181023e0 0x7f001819ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00181a0580 con 0x7f0018100840 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00181a06c0 con 0x7f0018101a40 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001eda9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00181a0850 con 0x7f00181023e0 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001d346700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00181023e0 0x7f001819ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001d346700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00181023e0 0x7f001819ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59852/0 (socket says 172.21.15.184:59852) 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.965+0000 7f001d346700 1 -- 172.21.15.184:0/2333002740 learned_addr learned my addr 172.21.15.184:0/2333002740 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f0017fff700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0018101a40 0x7f0018198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:59.969 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001cb45700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100840 0x7f0018198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001d346700 1 -- 172.21.15.184:0/2333002740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0018101a40 msgr2=0x7f0018198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001d346700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0018101a40 0x7f0018198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001d346700 1 -- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100840 msgr2=0x7f0018198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001d346700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100840 0x7f0018198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001d346700 1 -- 172.21.15.184:0/2333002740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c0095f0 con 0x7f00181023e0 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001d346700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00181023e0 0x7f001819ffb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f001000b9a0 tx=0x7f001000bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f0015ffb700 1 -- 172.21.15.184:0/2333002740 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001000c930 con 0x7f00181023e0 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00181a0b30 con 0x7f00181023e0 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f0015ffb700 1 -- 172.21.15.184:0/2333002740 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0010014440 con 0x7f00181023e0 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f0015ffb700 1 -- 172.21.15.184:0/2333002740 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100075d0 con 0x7f00181023e0 2023-10-08T09:11:59.970 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.966+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00181a1030 con 0x7f00181023e0 2023-10-08T09:11:59.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.967+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001804e4c0 con 0x7f00181023e0 2023-10-08T09:11:59.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.968+0000 7f0015ffb700 1 -- 172.21.15.184:0/2333002740 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f00100145b0 con 0x7f00181023e0 2023-10-08T09:11:59.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.969+0000 7f0015ffb700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00080a1480 0x7f00080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:11:59.971 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.969+0000 7f0015ffb700 1 -- 172.21.15.184:0/2333002740 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f0010006e40 con 0x7f00181023e0 2023-10-08T09:11:59.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.969+0000 7f001cb45700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00080a1480 0x7f00080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:11:59.972 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.969+0000 7f001cb45700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00080a1480 0x7f00080a3940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b5c0 tx=0x7f000c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:11:59.974 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:11:59.973+0000 7f0015ffb700 1 -- 172.21.15.184:0/2333002740 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f001004f100 con 0x7f00181023e0 2023-10-08T09:12:00.171 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:00.170+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0018074140 con 0x7f00080a1480 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] boot 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: osdmap e69: 11 total, 11 up, 11 in 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:12:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:12:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:12:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] boot 2023-10-08T09:12:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: osdmap e69: 11 total, 11 up, 11 in 2023-10-08T09:12:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:12:00.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:12:00.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:12:00.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:12:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:12:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: osd.10 [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] boot 2023-10-08T09:12:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: osdmap e69: 11 total, 11 up, 11 in 2023-10-08T09:12:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-08T09:12:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-08T09:12:00.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-08T09:12:00.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:01.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:01 smithi184 ceph-mon[103675]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:01.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:01 smithi184 ceph-mon[103675]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:12:01.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:01 smithi184 ceph-mon[103675]: osdmap e70: 11 total, 11 up, 11 in 2023-10-08T09:12:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:01 smithi132 ceph-mon[103389]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:01 smithi132 ceph-mon[103389]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:12:01.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:01 smithi132 ceph-mon[103389]: osdmap e70: 11 total, 11 up, 11 in 2023-10-08T09:12:01.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:01 smithi049 ceph-mon[99485]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:01.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:01 smithi049 ceph-mon[99485]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi184:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:12:01.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:01 smithi049 ceph-mon[99485]: osdmap e70: 11 total, 11 up, 11 in 2023-10-08T09:12:02.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:02 smithi184 ceph-mon[103675]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T09:12:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:02 smithi132 ceph-mon[103389]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T09:12:02.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:02 smithi049 ceph-mon[99485]: osdmap e71: 11 total, 11 up, 11 in 2023-10-08T09:12:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T09:12:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/2619315569' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]: dispatch 2023-10-08T09:12:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]: dispatch 2023-10-08T09:12:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]': finished 2023-10-08T09:12:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: osdmap e73: 12 total, 11 up, 12 in 2023-10-08T09:12:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:03 smithi184 ceph-mon[103675]: from='client.? 172.21.15.184:0/3038111592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:12:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T09:12:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/2619315569' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]: dispatch 2023-10-08T09:12:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]: dispatch 2023-10-08T09:12:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]': finished 2023-10-08T09:12:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: osdmap e73: 12 total, 11 up, 12 in 2023-10-08T09:12:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:03 smithi132 ceph-mon[103389]: from='client.? 172.21.15.184:0/3038111592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:12:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: osdmap e72: 11 total, 11 up, 11 in 2023-10-08T09:12:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/2619315569' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]: dispatch 2023-10-08T09:12:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]: dispatch 2023-10-08T09:12:03.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4139ccc6-3973-496b-828c-069a9a0735fb"}]': finished 2023-10-08T09:12:03.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: osdmap e73: 12 total, 11 up, 12 in 2023-10-08T09:12:03.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:03.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:03 smithi049 ceph-mon[99485]: from='client.? 172.21.15.184:0/3038111592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-08T09:12:05.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:05 smithi184 ceph-mon[103675]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:05.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:05 smithi132 ceph-mon[103389]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:05 smithi049 ceph-mon[99485]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:07.688 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:07 smithi184 ceph-mon[103675]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-10-08T09:12:07.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:07 smithi132 ceph-mon[103389]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-10-08T09:12:07.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:07 smithi049 ceph-mon[99485]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-10-08T09:12:09.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:09 smithi184 ceph-mon[103675]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-08T09:12:09.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T09:12:09.724 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:09 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:09.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:09 smithi132 ceph-mon[103389]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-08T09:12:09.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T09:12:09.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:09 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:09 smithi049 ceph-mon[99485]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2023-10-08T09:12:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-08T09:12:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:09 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:10.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:10 smithi184 ceph-mon[103675]: Deploying daemon osd.11 on smithi184 2023-10-08T09:12:10.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:10 smithi132 ceph-mon[103389]: Deploying daemon osd.11 on smithi184 2023-10-08T09:12:10.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:10 smithi049 ceph-mon[99485]: Deploying daemon osd.11 on smithi184 2023-10-08T09:12:11.484 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:11 smithi184 ceph-mon[103675]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:12:11.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:11 smithi132 ceph-mon[103389]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:12:11.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:11 smithi049 ceph-mon[99485]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-08T09:12:13.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:13 smithi184 ceph-mon[103675]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-08T09:12:13.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:13 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:13.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:13 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:13.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:13 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:13 smithi132 ceph-mon[103389]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-08T09:12:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:13 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:13 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:13 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:13 smithi049 ceph-mon[99485]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-10-08T09:12:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:13 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:15.433 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:15 smithi184 ceph-mon[103675]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-08T09:12:15.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:15 smithi132 ceph-mon[103389]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-08T09:12:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:15 smithi049 ceph-mon[99485]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-10-08T09:12:15.933 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.932+0000 7f0015ffb700 1 -- 172.21.15.184:0/2333002740 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f0018074140 con 0x7f00080a1480 2023-10-08T09:12:15.933 INFO:teuthology.orchestra.run.smithi184.stdout:Created osd(s) 11 on host 'smithi184' 2023-10-08T09:12:15.935 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00080a1480 msgr2=0x7f00080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:15.935 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00080a1480 0x7f00080a3940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b5c0 tx=0x7f000c005fd0 comp rx=0 tx=0).stop 2023-10-08T09:12:15.935 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00181023e0 msgr2=0x7f001819ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:15.935 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00181023e0 0x7f001819ffb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f001000b9a0 tx=0x7f001000bcb0 comp rx=0 tx=0).stop 2023-10-08T09:12:15.935 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 shutdown_connections 2023-10-08T09:12:15.936 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00080a1480 0x7f00080a3940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:15.936 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00181023e0 0x7f001819ffb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:15.936 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0018100840 0x7f0018198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:15.936 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 --2- 172.21.15.184:0/2333002740 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0018101a40 0x7f0018198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:15.936 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 >> 172.21.15.184:0/2333002740 conn(0x7f00180fbe00 msgr2=0x7f0018072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:15.936 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 shutdown_connections 2023-10-08T09:12:15.936 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:15.934+0000 7f001eda9700 1 -- 172.21.15.184:0/2333002740 wait complete. 2023-10-08T09:12:16.401 DEBUG:teuthology.orchestra.run.smithi184:osd.11> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.11.service 2023-10-08T09:12:16.403 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-08T09:12:16.404 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-08T09:12:16.433 INFO:journalctl@ceph.osd.11.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:12:16.986 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:12:17.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:16 smithi049 ceph-mon[99485]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:12:17.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.224 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.225 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.242 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:16 smithi184 ceph-mon[103675]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:12:17.242 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.242 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.243 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.243 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:16 smithi132 ceph-mon[103389]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-08T09:12:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:17.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:12:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:12:17.308+0000 7fa95fca5640 -1 osd.11 0 log_to_monitors true 2023-10-08T09:12:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:17 smithi184 ceph-mon[103675]: from='osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:12:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:17 smithi184 ceph-mon[103675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:12:18.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:17 smithi132 ceph-mon[103389]: from='osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:12:18.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:17 smithi132 ceph-mon[103389]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:12:18.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.299+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/2131983846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c38100930 msgr2=0x7f0c38100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:18.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.299+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/2131983846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c38100930 0x7f0c38100d90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0099c0 tx=0x7f0c2c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.299+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/2131983846 shutdown_connections 2023-10-08T09:12:18.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.299+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/2131983846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c381012d0 0x7f0c38107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.299+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/2131983846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c380ff730 0x7f0c380ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.299+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/2131983846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c38100930 0x7f0c38100d90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.299+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/2131983846 >> 172.21.15.49:0/2131983846 conn(0x7f0c380facf0 msgr2=0x7f0c380fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:18.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/2131983846 shutdown_connections 2023-10-08T09:12:18.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/2131983846 wait complete. 2023-10-08T09:12:18.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 Processor -- start 2023-10-08T09:12:18.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 -- start start 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380ff730 0x7f0c381967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c38100930 0x7f0c38196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c381012d0 0x7f0c3819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3819e2e0 con 0x7f0c381012d0 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.300+0000 7f0c3f7c0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3819e420 con 0x7f0c38100930 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3f7c0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3819e5b0 con 0x7f0c380ff730 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3dd5d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c381012d0 0x7f0c3819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380ff730 0x7f0c381967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3dd5d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c381012d0 0x7f0c3819dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47456/0 (socket says 172.21.15.49:47456) 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380ff730 0x7f0c381967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:34486/0 (socket says 172.21.15.49:34486) 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3dd5d700 1 -- 172.21.15.49:0/1298411862 learned_addr learned my addr 172.21.15.49:0/1298411862 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3cd5b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c38100930 0x7f0c38196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 -- 172.21.15.49:0/1298411862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c38100930 msgr2=0x7f0c38196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c38100930 0x7f0c38196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 -- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c381012d0 msgr2=0x7f0c3819dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c381012d0 0x7f0c3819dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 -- 172.21.15.49:0/1298411862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0095f0 con 0x7f0c380ff730 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3dd5d700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c381012d0 0x7f0c3819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3d55c700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380ff730 0x7f0c381967a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400d770 tx=0x7f0c3400da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c2a7fc700 1 -- 172.21.15.49:0/1298411862 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34004220 con 0x7f0c380ff730 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3819e890 con 0x7f0c380ff730 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c2a7fc700 1 -- 172.21.15.49:0/1298411862 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c34004380 con 0x7f0c380ff730 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c2a7fc700 1 -- 172.21.15.49:0/1298411862 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34003cf0 con 0x7f0c380ff730 2023-10-08T09:12:18.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.301+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3819ede0 con 0x7f0c380ff730 2023-10-08T09:12:18.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.302+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3804e4c0 con 0x7f0c380ff730 2023-10-08T09:12:18.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.304+0000 7f0c2a7fc700 1 -- 172.21.15.49:0/1298411862 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0c3400f690 con 0x7f0c380ff730 2023-10-08T09:12:18.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.304+0000 7f0c2a7fc700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c240a1480 0x7f0c240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:18.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.304+0000 7f0c3cd5b700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c240a1480 0x7f0c240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:18.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.304+0000 7f0c2a7fc700 1 -- 172.21.15.49:0/1298411862 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f0c340cc5c0 con 0x7f0c380ff730 2023-10-08T09:12:18.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.305+0000 7f0c3cd5b700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c240a1480 0x7f0c240a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c003730 tx=0x7f0c2c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:18.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.309+0000 7f0c2a7fc700 1 -- 172.21.15.49:0/1298411862 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0c34096150 con 0x7f0c380ff730 2023-10-08T09:12:18.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:17 smithi049 ceph-mon[99485]: from='osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:12:18.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:17 smithi049 ceph-mon[99485]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-08T09:12:18.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.511+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0c3810b0f0 con 0x7f0c380ff730 2023-10-08T09:12:18.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.511+0000 7f0c2a7fc700 1 -- 172.21.15.49:0/1298411862 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0c34099770 con 0x7f0c380ff730 2023-10-08T09:12:18.513 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c240a1480 msgr2=0x7f0c240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c240a1480 0x7f0c240a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c003730 tx=0x7f0c2c00b540 comp rx=0 tx=0).stop 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380ff730 msgr2=0x7f0c381967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380ff730 0x7f0c381967a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400d770 tx=0x7f0c3400da80 comp rx=0 tx=0).stop 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 shutdown_connections 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0c240a1480 0x7f0c240a3940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c380ff730 0x7f0c381967a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c381012d0 0x7f0c3819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 --2- 172.21.15.49:0/1298411862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c38100930 0x7f0c38196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 >> 172.21.15.49:0/1298411862 conn(0x7f0c380facf0 msgr2=0x7f0c38068a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:18.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 shutdown_connections 2023-10-08T09:12:18.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:18.513+0000 7f0c3f7c0700 1 -- 172.21.15.49:0/1298411862 wait complete. 2023-10-08T09:12:18.950 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1696756319,"num_in_osds":12,"osd_in_since":1696756322,"num_remapped_pgs":0} 2023-10-08T09:12:19.248 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:12:18.936+0000 7fa95531c700 -1 osd.11 0 waiting for initial osdmap 2023-10-08T09:12:19.248 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:12:18.948+0000 7fa94e90c700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T09:12:19.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-mon[103675]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:19.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-mon[103675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T09:12:19.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-mon[103675]: osdmap e74: 12 total, 11 up, 12 in 2023-10-08T09:12:19.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:19.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-mon[103675]: from='osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:12:19.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-mon[103675]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:12:19.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:18 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1298411862' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:12:19.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:18 smithi132 ceph-mon[103389]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:19.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:18 smithi132 ceph-mon[103389]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T09:12:19.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:18 smithi132 ceph-mon[103389]: osdmap e74: 12 total, 11 up, 12 in 2023-10-08T09:12:19.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:18 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:19.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:18 smithi132 ceph-mon[103389]: from='osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:12:19.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:18 smithi132 ceph-mon[103389]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:12:19.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:18 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1298411862' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:12:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:18 smithi049 ceph-mon[99485]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:18 smithi049 ceph-mon[99485]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-08T09:12:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:18 smithi049 ceph-mon[99485]: osdmap e74: 12 total, 11 up, 12 in 2023-10-08T09:12:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:18 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:18 smithi049 ceph-mon[99485]: from='osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:12:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:18 smithi049 ceph-mon[99485]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]: dispatch 2023-10-08T09:12:19.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:18 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1298411862' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:12:19.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd stat -f json 2023-10-08T09:12:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:12:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: osdmap e75: 12 total, 11 up, 12 in 2023-10-08T09:12:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: Detected new or changed devices on smithi184 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi184 to 4292M 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:20.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:20.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:20.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:20.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:20.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:12:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: osdmap e75: 12 total, 11 up, 12 in 2023-10-08T09:12:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: Detected new or changed devices on smithi184 2023-10-08T09:12:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi184 to 4292M 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:20.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:20.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:20.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi184", "root=default"]}]': finished 2023-10-08T09:12:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: osdmap e75: 12 total, 11 up, 12 in 2023-10-08T09:12:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: Detected new or changed devices on smithi184 2023-10-08T09:12:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi184 to 4292M 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:20.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:20.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:20.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:20.518 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:12:21.025 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:20 smithi049 ceph-mon[99485]: purged_snaps scrub starts 2023-10-08T09:12:21.025 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:20 smithi049 ceph-mon[99485]: purged_snaps scrub ok 2023-10-08T09:12:21.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:20 smithi049 ceph-mon[99485]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:21.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:21.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:20 smithi049 ceph-mon[99485]: osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] boot 2023-10-08T09:12:21.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:20 smithi049 ceph-mon[99485]: osdmap e76: 12 total, 12 up, 12 in 2023-10-08T09:12:21.026 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:21.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:20 smithi184 ceph-mon[103675]: purged_snaps scrub starts 2023-10-08T09:12:21.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:20 smithi184 ceph-mon[103675]: purged_snaps scrub ok 2023-10-08T09:12:21.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:20 smithi184 ceph-mon[103675]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:21.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:21.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:20 smithi184 ceph-mon[103675]: osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] boot 2023-10-08T09:12:21.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:20 smithi184 ceph-mon[103675]: osdmap e76: 12 total, 12 up, 12 in 2023-10-08T09:12:21.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:21.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:20 smithi132 ceph-mon[103389]: purged_snaps scrub starts 2023-10-08T09:12:21.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:20 smithi132 ceph-mon[103389]: purged_snaps scrub ok 2023-10-08T09:12:21.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:20 smithi132 ceph-mon[103389]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-08T09:12:21.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:21.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:20 smithi132 ceph-mon[103389]: osd.11 [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] boot 2023-10-08T09:12:21.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:20 smithi132 ceph-mon[103389]: osdmap e76: 12 total, 12 up, 12 in 2023-10-08T09:12:21.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-08T09:12:21.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 -- 172.21.15.49:0/3073269649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55381023e0 msgr2=0x7f5538108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:21.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 --2- 172.21.15.49:0/3073269649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55381023e0 0x7f5538108c90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f553000b0d0 tx=0x7f553000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 -- 172.21.15.49:0/3073269649 shutdown_connections 2023-10-08T09:12:21.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 --2- 172.21.15.49:0/3073269649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55381023e0 0x7f5538108c90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 --2- 172.21.15.49:0/3073269649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5538100840 0x7f5538100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 --2- 172.21.15.49:0/3073269649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5538101a40 0x7f5538101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 -- 172.21.15.49:0/3073269649 >> 172.21.15.49:0/3073269649 conn(0x7f55380fbde0 msgr2=0x7f55380fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 -- 172.21.15.49:0/3073269649 shutdown_connections 2023-10-08T09:12:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.720+0000 7f554054b700 1 -- 172.21.15.49:0/3073269649 wait complete. 2023-10-08T09:12:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 Processor -- start 2023-10-08T09:12:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 -- start start 2023-10-08T09:12:21.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5538100840 0x7f5538196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5538101a40 0x7f5538196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381023e0 0x7f553819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553819e2d0 con 0x7f5538100840 2023-10-08T09:12:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553819e410 con 0x7f55381023e0 2023-10-08T09:12:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f554054b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553819e5a0 con 0x7f5538101a40 2023-10-08T09:12:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f553e2e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5538100840 0x7f5538196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:21.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f553e2e7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5538100840 0x7f5538196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47478/0 (socket says 172.21.15.49:47478) 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.721+0000 7f553e2e7700 1 -- 172.21.15.49:0/4275171467 learned_addr learned my addr 172.21.15.49:0/4275171467 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553dae6700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5538101a40 0x7f5538196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553eae8700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381023e0 0x7f553819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553e2e7700 1 -- 172.21.15.49:0/4275171467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5538101a40 msgr2=0x7f5538196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553e2e7700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5538101a40 0x7f5538196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553e2e7700 1 -- 172.21.15.49:0/4275171467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381023e0 msgr2=0x7f553819dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553e2e7700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381023e0 0x7f553819dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553e2e7700 1 -- 172.21.15.49:0/4275171467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009d00 con 0x7f5538100840 2023-10-08T09:12:21.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553dae6700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5538101a40 0x7f5538196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:21.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f553e2e7700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5538100840 0x7f5538196790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f552c00c9c0 tx=0x7f552c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:21.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f552b7fe700 1 -- 172.21.15.49:0/4275171467 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c007ac0 con 0x7f5538100840 2023-10-08T09:12:21.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553819e880 con 0x7f5538100840 2023-10-08T09:12:21.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f552b7fe700 1 -- 172.21.15.49:0/4275171467 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f552c007c20 con 0x7f5538100840 2023-10-08T09:12:21.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f552b7fe700 1 -- 172.21.15.49:0/4275171467 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c018660 con 0x7f5538100840 2023-10-08T09:12:21.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.722+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553819ed80 con 0x7f5538100840 2023-10-08T09:12:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.723+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5538107c00 con 0x7f5538100840 2023-10-08T09:12:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.724+0000 7f552b7fe700 1 -- 172.21.15.49:0/4275171467 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f552c004020 con 0x7f5538100840 2023-10-08T09:12:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.725+0000 7f552b7fe700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55240a1480 0x7f55240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.725+0000 7f553dae6700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55240a1480 0x7f55240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.725+0000 7f552b7fe700 1 -- 172.21.15.49:0/4275171467 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f552c0cca40 con 0x7f5538100840 2023-10-08T09:12:21.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.725+0000 7f553dae6700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55240a1480 0x7f55240a3940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5534005950 tx=0x7f55340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:21.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.728+0000 7f552b7fe700 1 -- 172.21.15.49:0/4275171467 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f552c096530 con 0x7f5538100840 2023-10-08T09:12:21.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.931+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f553804e4c0 con 0x7f5538100840 2023-10-08T09:12:21.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.931+0000 7f552b7fe700 1 -- 172.21.15.49:0/4275171467 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f552c099b50 con 0x7f5538100840 2023-10-08T09:12:21.934 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:21.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.933+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55240a1480 msgr2=0x7f55240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:21.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.933+0000 7f554054b700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55240a1480 0x7f55240a3940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5534005950 tx=0x7f55340058e0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.933+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5538100840 msgr2=0x7f5538196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:21.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.933+0000 7f554054b700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5538100840 0x7f5538196790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f552c00c9c0 tx=0x7f552c00cd80 comp rx=0 tx=0).stop 2023-10-08T09:12:21.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 shutdown_connections 2023-10-08T09:12:21.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55240a1480 0x7f55240a3940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55381023e0 0x7f553819dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5538100840 0x7f5538196790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 --2- 172.21.15.49:0/4275171467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5538101a40 0x7f5538196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:21.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 >> 172.21.15.49:0/4275171467 conn(0x7f55380fbde0 msgr2=0x7f5538104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:21.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 shutdown_connections 2023-10-08T09:12:21.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:21.934+0000 7f554054b700 1 -- 172.21.15.49:0/4275171467 wait complete. 2023-10-08T09:12:22.284 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1696756339,"num_in_osds":12,"osd_in_since":1696756322,"num_remapped_pgs":0} 2023-10-08T09:12:22.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T09:12:22.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:22 smithi049 ceph-mon[99485]: osdmap e77: 12 total, 12 up, 12 in 2023-10-08T09:12:22.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:22 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4275171467' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:12:22.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:22 smithi184 ceph-mon[103675]: osdmap e77: 12 total, 12 up, 12 in 2023-10-08T09:12:22.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:22 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4275171467' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:12:22.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:22 smithi132 ceph-mon[103389]: osdmap e77: 12 total, 12 up, 12 in 2023-10-08T09:12:22.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:22 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4275171467' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-08T09:12:22.859 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:12:23.679 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:23 smithi049 ceph-mon[99485]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:23.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:23 smithi184 ceph-mon[103675]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:23.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:23 smithi132 ceph-mon[103389]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 -- 172.21.15.49:0/1633268452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 msgr2=0x7f66fc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 --2- 172.21.15.49:0/1633268452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc105a90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009990 tx=0x7f66f8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 -- 172.21.15.49:0/1633268452 shutdown_connections 2023-10-08T09:12:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 --2- 172.21.15.49:0/1633268452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc100110 0x7f66fc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 --2- 172.21.15.49:0/1633268452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc105a90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 --2- 172.21.15.49:0/1633268452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66fc0ff750 0x7f66fc0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 -- 172.21.15.49:0/1633268452 >> 172.21.15.49:0/1633268452 conn(0x7f66fc075790 msgr2=0x7f66fc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 -- 172.21.15.49:0/1633268452 shutdown_connections 2023-10-08T09:12:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.185+0000 7f67036d6700 1 -- 172.21.15.49:0/1633268452 wait complete. 2023-10-08T09:12:24.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.186+0000 7f67036d6700 1 Processor -- start 2023-10-08T09:12:24.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.186+0000 7f67036d6700 1 -- start start 2023-10-08T09:12:24.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f67036d6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc0ff750 0x7f66fc19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:24.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f67036d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66fc100110 0x7f66fc19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:24.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f67036d6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc1948f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:24.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f67036d6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc194f50 con 0x7f66fc1056b0 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f67036d6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1950c0 con 0x7f66fc100110 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f67036d6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc195280 con 0x7f66fc0ff750 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc1948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc1948f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47496/0 (socket says 172.21.15.49:47496) 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 -- 172.21.15.49:0/748161706 learned_addr learned my addr 172.21.15.49:0/748161706 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6700c71700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66fc100110 0x7f66fc19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701472700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc0ff750 0x7f66fc19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 -- 172.21.15.49:0/748161706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc0ff750 msgr2=0x7f66fc19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc0ff750 0x7f66fc19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 -- 172.21.15.49:0/748161706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66fc100110 msgr2=0x7f66fc19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66fc100110 0x7f66fc19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 -- 172.21.15.49:0/748161706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80095f0 con 0x7f66fc1056b0 2023-10-08T09:12:24.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f6701c73700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc1948f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f66f400b970 tx=0x7f66f400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:24.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f66ee7fc700 1 -- 172.21.15.49:0/748161706 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f400c9b0 con 0x7f66fc1056b0 2023-10-08T09:12:24.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f66ee7fc700 1 -- 172.21.15.49:0/748161706 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66f4014440 con 0x7f66fc1056b0 2023-10-08T09:12:24.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.187+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc195560 con 0x7f66fc1056b0 2023-10-08T09:12:24.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.188+0000 7f66ee7fc700 1 -- 172.21.15.49:0/748161706 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40075d0 con 0x7f66fc1056b0 2023-10-08T09:12:24.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.188+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc1a17a0 con 0x7f66fc1056b0 2023-10-08T09:12:24.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.188+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc107cb0 con 0x7f66fc1056b0 2023-10-08T09:12:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.190+0000 7f66ee7fc700 1 -- 172.21.15.49:0/748161706 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f66f40145b0 con 0x7f66fc1056b0 2023-10-08T09:12:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.190+0000 7f66ee7fc700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66e80a3710 0x7f66e80a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.190+0000 7f66ee7fc700 1 -- 172.21.15.49:0/748161706 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66f4006e40 con 0x7f66fc1056b0 2023-10-08T09:12:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.190+0000 7f6701472700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66e80a3710 0x7f66e80a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:24.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.191+0000 7f6701472700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66e80a3710 0x7f66e80a5bd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f66f8006010 tx=0x7f66f800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:24.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.194+0000 7f66ee7fc700 1 -- 172.21.15.49:0/748161706 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f66f4095f10 con 0x7f66fc1056b0 2023-10-08T09:12:24.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.394+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66fc100df0 con 0x7f66fc1056b0 2023-10-08T09:12:24.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.395+0000 7f66ee7fc700 1 -- 172.21.15.49:0/748161706 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18431 (secure 0 0 0) 0x7f66f4099530 con 0x7f66fc1056b0 2023-10-08T09:12:24.397 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:24.397 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":77,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:12:21.320137+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:12:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66e80a3710 msgr2=0x7f66e80a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66e80a3710 0x7f66e80a5bd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f66f8006010 tx=0x7f66f800b540 comp rx=0 tx=0).stop 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 msgr2=0x7f66fc1948f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc1948f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f66f400b970 tx=0x7f66f400bd30 comp rx=0 tx=0).stop 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 shutdown_connections 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66e80a3710 0x7f66e80a5bd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66fc0ff750 0x7f66fc19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66fc1056b0 0x7f66fc1948f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 --2- 172.21.15.49:0/748161706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66fc100110 0x7f66fc19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 >> 172.21.15.49:0/748161706 conn(0x7f66fc075790 msgr2=0x7f66fc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 shutdown_connections 2023-10-08T09:12:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:24.397+0000 7f67036d6700 1 -- 172.21.15.49:0/748161706 wait complete. 2023-10-08T09:12:24.807 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T09:12:24.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-08T09:12:25.372 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:12:25.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:25 smithi049 ceph-mon[99485]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:25.650 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:25 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/748161706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:12:25.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:25 smithi184 ceph-mon[103675]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:25.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:25 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/748161706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:12:25.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:25 smithi132 ceph-mon[103389]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:25.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:25 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/748161706' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:12:26.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 -- 172.21.15.49:0/2311434521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62441000a0 msgr2=0x7f624410c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:26.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 --2- 172.21.15.49:0/2311434521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62441000a0 0x7f624410c560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f623c00b0a0 tx=0x7f623c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:12:26.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 -- 172.21.15.49:0/2311434521 shutdown_connections 2023-10-08T09:12:26.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 --2- 172.21.15.49:0/2311434521 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6244105690 0x7f6244105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:26.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 --2- 172.21.15.49:0/2311434521 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f62441000a0 0x7f624410c560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:26.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 --2- 172.21.15.49:0/2311434521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62440ff6e0 0x7f62440ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:26.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 -- 172.21.15.49:0/2311434521 >> 172.21.15.49:0/2311434521 conn(0x7f62440766c0 msgr2=0x7f6244076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:26.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 -- 172.21.15.49:0/2311434521 shutdown_connections 2023-10-08T09:12:26.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.907+0000 7f6249883700 1 -- 172.21.15.49:0/2311434521 wait complete. 2023-10-08T09:12:26.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 Processor -- start 2023-10-08T09:12:26.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 -- start start 2023-10-08T09:12:26.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62440ff6e0 0x7f6244101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62441000a0 0x7f62441017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6244105690 0x7f6244101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441a5090 con 0x7f6244105690 2023-10-08T09:12:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441a5200 con 0x7f62441000a0 2023-10-08T09:12:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6249883700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441a53c0 con 0x7f62440ff6e0 2023-10-08T09:12:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f62437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6244105690 0x7f6244101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:26.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f62437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6244105690 0x7f6244101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47508/0 (socket says 172.21.15.49:47508) 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f62437fe700 1 -- 172.21.15.49:0/3567547481 learned_addr learned my addr 172.21.15.49:0/3567547481 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f6242ffd700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62440ff6e0 0x7f6244101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f62427fc700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62441000a0 0x7f62441017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.908+0000 7f62437fe700 1 -- 172.21.15.49:0/3567547481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62440ff6e0 msgr2=0x7f6244101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f62437fe700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62440ff6e0 0x7f6244101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f62437fe700 1 -- 172.21.15.49:0/3567547481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62441000a0 msgr2=0x7f62441017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f62437fe700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62441000a0 0x7f62441017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f62437fe700 1 -- 172.21.15.49:0/3567547481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009d00 con 0x7f6244105690 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f6242ffd700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62440ff6e0 0x7f6244101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:12:26.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f62427fc700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62441000a0 0x7f62441017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f62437fe700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6244105690 0x7f6244101ce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f623c00b070 tx=0x7f623c007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f6248881700 1 -- 172.21.15.49:0/3567547481 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00e040 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62441a5640 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f6248881700 1 -- 172.21.15.49:0/3567547481 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f623c009440 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f6248881700 1 -- 172.21.15.49:0/3567547481 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c01b6b0 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.909+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62441a5a60 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.910+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6244100dd0 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.911+0000 7f6248881700 1 -- 172.21.15.49:0/3567547481 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f623c01d480 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.912+0000 7f6248881700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f622c0a1480 0x7f622c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.912+0000 7f6242ffd700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f622c0a1480 0x7f622c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.912+0000 7f6248881700 1 -- 172.21.15.49:0/3567547481 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f623c0ce230 con 0x7f6244105690 2023-10-08T09:12:26.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.912+0000 7f6242ffd700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f622c0a1480 0x7f622c0a3940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6234009b50 tx=0x7f6234009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:26.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:26.916+0000 7f6248881700 1 -- 172.21.15.49:0/3567547481 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f623c097d20 con 0x7f6244105690 2023-10-08T09:12:27.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.116+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f624410f680 con 0x7f6244105690 2023-10-08T09:12:27.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.117+0000 7f6248881700 1 -- 172.21.15.49:0/3567547481 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f623c01de60 con 0x7f6244105690 2023-10-08T09:12:27.119 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f622c0a1480 msgr2=0x7f622c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f622c0a1480 0x7f622c0a3940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6234009b50 tx=0x7f6234009550 comp rx=0 tx=0).stop 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6244105690 msgr2=0x7f6244101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6244105690 0x7f6244101ce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f623c00b070 tx=0x7f623c007920 comp rx=0 tx=0).stop 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 shutdown_connections 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f622c0a1480 0x7f622c0a3940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62440ff6e0 0x7f6244101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6244105690 0x7f6244101ce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 --2- 172.21.15.49:0/3567547481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62441000a0 0x7f62441017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:27.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 >> 172.21.15.49:0/3567547481 conn(0x7f62440766c0 msgr2=0x7f62441034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:27.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 shutdown_connections 2023-10-08T09:12:27.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:27.119+0000 7f6249883700 1 -- 172.21.15.49:0/3567547481 wait complete. 2023-10-08T09:12:27.518 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:27 smithi049 ceph-mon[99485]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:27.518 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:27 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3567547481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:12:27.520 INFO:tasks.cephadm:Adding mds.a on smithi049 2023-10-08T09:12:27.520 INFO:tasks.cephadm:Adding mds.d on smithi049 2023-10-08T09:12:27.520 INFO:tasks.cephadm:Adding mds.g on smithi049 2023-10-08T09:12:27.520 INFO:tasks.cephadm:Adding mds.j on smithi049 2023-10-08T09:12:27.520 INFO:tasks.cephadm:Adding mds.b on smithi132 2023-10-08T09:12:27.520 INFO:tasks.cephadm:Adding mds.e on smithi132 2023-10-08T09:12:27.521 INFO:tasks.cephadm:Adding mds.h on smithi132 2023-10-08T09:12:27.521 INFO:tasks.cephadm:Adding mds.k on smithi132 2023-10-08T09:12:27.521 INFO:tasks.cephadm:Adding mds.c on smithi184 2023-10-08T09:12:27.521 INFO:tasks.cephadm:Adding mds.f on smithi184 2023-10-08T09:12:27.521 INFO:tasks.cephadm:Adding mds.i on smithi184 2023-10-08T09:12:27.521 INFO:tasks.cephadm:Adding mds.l on smithi184 2023-10-08T09:12:27.522 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph orch apply mds all '12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l' 2023-10-08T09:12:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:27 smithi184 ceph-mon[103675]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:27 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3567547481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:12:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:27 smithi132 ceph-mon[103389]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:27 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3567547481' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:12:28.101 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:12:29.328 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 -- 172.21.15.184:0/304099024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 msgr2=0x7fd9b0100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:29.328 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 --2- 172.21.15.184:0/304099024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0100d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0009990 tx=0x7fd9a0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 -- 172.21.15.184:0/304099024 shutdown_connections 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 --2- 172.21.15.184:0/304099024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9b01012d0 0x7fd9b0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 --2- 172.21.15.184:0/304099024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b00ff730 0x7fd9b00ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 --2- 172.21.15.184:0/304099024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0100d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 -- 172.21.15.184:0/304099024 >> 172.21.15.184:0/304099024 conn(0x7fd9b00facf0 msgr2=0x7fd9b00fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 -- 172.21.15.184:0/304099024 shutdown_connections 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.327+0000 7fd9b659a700 1 -- 172.21.15.184:0/304099024 wait complete. 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 Processor -- start 2023-10-08T09:12:29.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 -- start start 2023-10-08T09:12:29.331 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b00ff730 0x7fd9b01955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:29.331 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:29.331 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9b01012d0 0x7fd9b019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:29.331 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b019d0f0 con 0x7fd9b00ff730 2023-10-08T09:12:29.331 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b019d230 con 0x7fd9b01012d0 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.328+0000 7fd9b659a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b019d3c0 con 0x7fd9b0100930 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0195af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:59972/0 (socket says 172.21.15.184:59972) 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 -- 172.21.15.184:0/399101152 learned_addr learned my addr 172.21.15.184:0/399101152 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9b4b37700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9b01012d0 0x7fd9b019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9affff700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b00ff730 0x7fd9b01955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 -- 172.21.15.184:0/399101152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9b01012d0 msgr2=0x7fd9b019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9b01012d0 0x7fd9b019cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 -- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b00ff730 msgr2=0x7fd9b01955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b00ff730 0x7fd9b01955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 -- 172.21.15.184:0/399101152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a00095f0 con 0x7fd9b0100930 2023-10-08T09:12:29.332 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9af7fe700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0195af0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00036f0 tx=0x7fd9a0003b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:29.333 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9ad7fa700 1 -- 172.21.15.184:0/399101152 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a001d070 con 0x7fd9b0100930 2023-10-08T09:12:29.333 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b019d640 con 0x7fd9b0100930 2023-10-08T09:12:29.333 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9ad7fa700 1 -- 172.21.15.184:0/399101152 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9a0003e10 con 0x7fd9b0100930 2023-10-08T09:12:29.333 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9ad7fa700 1 -- 172.21.15.184:0/399101152 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000fe00 con 0x7fd9b0100930 2023-10-08T09:12:29.333 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.329+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b019dab0 con 0x7fd9b0100930 2023-10-08T09:12:29.333 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.330+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b0106b10 con 0x7fd9b0100930 2023-10-08T09:12:29.333 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.331+0000 7fd9ad7fa700 1 -- 172.21.15.184:0/399101152 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd9a000f4f0 con 0x7fd9b0100930 2023-10-08T09:12:29.334 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.332+0000 7fd9ad7fa700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd99c0a1480 0x7fd99c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:29.334 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.332+0000 7fd9ad7fa700 1 -- 172.21.15.184:0/399101152 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd9a00cdb00 con 0x7fd9b0100930 2023-10-08T09:12:29.334 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.332+0000 7fd9affff700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd99c0a1480 0x7fd99c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:29.334 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.332+0000 7fd9affff700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd99c0a1480 0x7fd99c0a3940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd998005950 tx=0x7fd9980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:29.336 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.336+0000 7fd9ad7fa700 1 -- 172.21.15.184:0/399101152 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9a0097690 con 0x7fd9b0100930 2023-10-08T09:12:29.533 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.533+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9b0060d30 con 0x7fd99c0a1480 2023-10-08T09:12:29.542 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.541+0000 7fd9ad7fa700 1 -- 172.21.15.184:0/399101152 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fd9b0060d30 con 0x7fd99c0a1480 2023-10-08T09:12:29.542 INFO:teuthology.orchestra.run.smithi184.stdout:Scheduled mds.all update... 2023-10-08T09:12:29.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.543+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd99c0a1480 msgr2=0x7fd99c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:29.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.543+0000 7fd9b659a700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd99c0a1480 0x7fd99c0a3940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd998005950 tx=0x7fd9980058e0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 msgr2=0x7fd9b0195af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:29.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0195af0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00036f0 tx=0x7fd9a0003b70 comp rx=0 tx=0).stop 2023-10-08T09:12:29.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 shutdown_connections 2023-10-08T09:12:29.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd99c0a1480 0x7fd99c0a3940 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9b01012d0 0x7fd9b019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9b00ff730 0x7fd9b01955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 --2- 172.21.15.184:0/399101152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b0100930 0x7fd9b0195af0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:29.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 >> 172.21.15.184:0/399101152 conn(0x7fd9b00facf0 msgr2=0x7fd9b0103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:29.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 shutdown_connections 2023-10-08T09:12:29.545 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T09:12:29.544+0000 7fd9b659a700 1 -- 172.21.15.184:0/399101152 wait complete. 2023-10-08T09:12:29.698 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:29 smithi184 ceph-mon[103675]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:29 smithi132 ceph-mon[103389]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:29 smithi049 ceph-mon[99485]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:29.960 DEBUG:teuthology.orchestra.run.smithi049:mds.a> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.a.service 2023-10-08T09:12:29.964 DEBUG:teuthology.orchestra.run.smithi049:mds.d> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.d.service 2023-10-08T09:12:30.002 INFO:journalctl@ceph.mds.a.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:12:30.003 DEBUG:teuthology.orchestra.run.smithi049:mds.g> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.g.service 2023-10-08T09:12:30.034 INFO:journalctl@ceph.mds.d.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:12:30.035 DEBUG:teuthology.orchestra.run.smithi049:mds.j> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.j.service 2023-10-08T09:12:30.063 INFO:journalctl@ceph.mds.g.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:12:30.065 DEBUG:teuthology.orchestra.run.smithi132:mds.b> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.b.service 2023-10-08T09:12:30.067 DEBUG:teuthology.orchestra.run.smithi132:mds.e> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.e.service 2023-10-08T09:12:30.092 INFO:journalctl@ceph.mds.j.smithi049.stdout:-- Logs begin at Sun 2023-10-08 08:50:45 UTC. -- 2023-10-08T09:12:30.106 INFO:journalctl@ceph.mds.b.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:12:30.107 DEBUG:teuthology.orchestra.run.smithi132:mds.h> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.h.service 2023-10-08T09:12:30.136 INFO:journalctl@ceph.mds.e.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:12:30.137 DEBUG:teuthology.orchestra.run.smithi132:mds.k> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.k.service 2023-10-08T09:12:30.164 INFO:journalctl@ceph.mds.h.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:12:30.165 DEBUG:teuthology.orchestra.run.smithi184:mds.c> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.c.service 2023-10-08T09:12:30.167 DEBUG:teuthology.orchestra.run.smithi184:mds.f> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.f.service 2023-10-08T09:12:30.191 INFO:journalctl@ceph.mds.k.smithi132.stdout:-- Logs begin at Sun 2023-10-08 08:48:52 UTC. -- 2023-10-08T09:12:30.198 INFO:journalctl@ceph.mds.c.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:12:30.199 DEBUG:teuthology.orchestra.run.smithi184:mds.i> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.i.service 2023-10-08T09:12:30.226 INFO:journalctl@ceph.mds.f.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:12:30.227 DEBUG:teuthology.orchestra.run.smithi184:mds.l> sudo journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.l.service 2023-10-08T09:12:30.253 INFO:journalctl@ceph.mds.i.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:12:30.254 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-08T09:12:30.255 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:12:30.296 INFO:journalctl@ceph.mds.l.smithi184.stdout:-- Logs begin at Sun 2023-10-08 08:49:06 UTC. -- 2023-10-08T09:12:30.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 -- 172.21.15.49:0/2007137136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0101c90 msgr2=0x7fefa01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 --2- 172.21.15.49:0/2007137136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0101c90 0x7fefa01020b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef940099c0 tx=0x7fef94009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 -- 172.21.15.49:0/2007137136 shutdown_connections 2023-10-08T09:12:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 --2- 172.21.15.49:0/2007137136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa00ff0a0 0x7fefa0108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 --2- 172.21.15.49:0/2007137136 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa0101c90 0x7fefa01020b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 --2- 172.21.15.49:0/2007137136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa00fe700 0x7fefa00feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 -- 172.21.15.49:0/2007137136 >> 172.21.15.49:0/2007137136 conn(0x7fefa00fa2c0 msgr2=0x7fefa00fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:30.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 -- 172.21.15.49:0/2007137136 shutdown_connections 2023-10-08T09:12:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 -- 172.21.15.49:0/2007137136 wait complete. 2023-10-08T09:12:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.404+0000 7fefa48e5700 1 Processor -- start 2023-10-08T09:12:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fefa48e5700 1 -- start start 2023-10-08T09:12:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fefa48e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa00fe700 0x7fefa0101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fefa48e5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa00ff0a0 0x7fefa00ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fefa48e5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa0101c90 0x7fefa00fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fefa48e5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa01006b0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fefa48e5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa0100820 con 0x7fefa00fe700 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fefa48e5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa01a06d0 con 0x7fefa0101c90 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa00ff0a0 0x7fefa00ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa00ff0a0 0x7fefa00ff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47520/0 (socket says 172.21.15.49:47520) 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 -- 172.21.15.49:0/3425992466 learned_addr learned my addr 172.21.15.49:0/3425992466 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9e59c700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa00fe700 0x7fefa0101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9ed9d700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa0101c90 0x7fefa00fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 -- 172.21.15.49:0/3425992466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa0101c90 msgr2=0x7fefa00fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa0101c90 0x7fefa00fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 -- 172.21.15.49:0/3425992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa00fe700 msgr2=0x7fefa0101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa00fe700 0x7fefa0101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 -- 172.21.15.49:0/3425992466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940095f0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef9dd9b700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa00ff0a0 0x7fefa00ff9d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef9000d950 tx=0x7fef9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.405+0000 7fef8f7fe700 1 -- 172.21.15.49:0/3425992466 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9000f890 con 0x7fefa00ff0a0 2023-10-08T09:12:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.406+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa01a0960 con 0x7fefa00ff0a0 2023-10-08T09:12:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.406+0000 7fef8f7fe700 1 -- 172.21.15.49:0/3425992466 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef90010460 con 0x7fefa00ff0a0 2023-10-08T09:12:30.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.406+0000 7fef8f7fe700 1 -- 172.21.15.49:0/3425992466 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900195c0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.406+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa01a0ec0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.406+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa0107d00 con 0x7fefa00ff0a0 2023-10-08T09:12:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.408+0000 7fef8f7fe700 1 -- 172.21.15.49:0/3425992466 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fef900108d0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.408+0000 7fef8f7fe700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fef880a1480 0x7fef880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.408+0000 7fef9e59c700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fef880a1480 0x7fef880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.408+0000 7fef8f7fe700 1 -- 172.21.15.49:0/3425992466 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fef900cc5c0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.409+0000 7fef9e59c700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fef880a1480 0x7fef880a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef94003820 tx=0x7fef94005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:30.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.412+0000 7fef8f7fe700 1 -- 172.21.15.49:0/3425992466 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef900960c0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.619+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa010bec0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.620+0000 7fef8f7fe700 1 -- 172.21.15.49:0/3425992466 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18431 (secure 0 0 0) 0x7fef900996e0 con 0x7fefa00ff0a0 2023-10-08T09:12:30.623 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:30.623 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":77,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:12:21.320137+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:12:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fef880a1480 msgr2=0x7fef880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fef880a1480 0x7fef880a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef94003820 tx=0x7fef94005920 comp rx=0 tx=0).stop 2023-10-08T09:12:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa00ff0a0 msgr2=0x7fefa00ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa00ff0a0 0x7fefa00ff9d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fef9000d950 tx=0x7fef9000dc60 comp rx=0 tx=0).stop 2023-10-08T09:12:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 shutdown_connections 2023-10-08T09:12:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fef880a1480 0x7fef880a3940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefa0101c90 0x7fefa00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefa00fe700 0x7fefa0101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 --2- 172.21.15.49:0/3425992466 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefa00ff0a0 0x7fefa00ff9d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 >> 172.21.15.49:0/3425992466 conn(0x7fefa00fa2c0 msgr2=0x7fefa0104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 shutdown_connections 2023-10-08T09:12:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.622+0000 7fefa48e5700 1 -- 172.21.15.49:0/3425992466 wait complete. 2023-10-08T09:12:30.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T09:12:30.638 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:12:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 -- 172.21.15.49:0/1983226272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0101002a0 msgr2=0x7fd010107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 --2- 172.21.15.49:0/1983226272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0101002a0 0x7fd010107b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd00800b0a0 tx=0x7fd00800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 -- 172.21.15.49:0/1983226272 shutdown_connections 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 --2- 172.21.15.49:0/1983226272 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0101002a0 0x7fd010107b40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 --2- 172.21.15.49:0/1983226272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd010100d20 0x7fd010101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 --2- 172.21.15.49:0/1983226272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0100ff8e0 0x7fd0100ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 -- 172.21.15.49:0/1983226272 >> 172.21.15.49:0/1983226272 conn(0x7fd0100fa390 msgr2=0x7fd0100fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 -- 172.21.15.49:0/1983226272 shutdown_connections 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 -- 172.21.15.49:0/1983226272 wait complete. 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.765+0000 7fd0182fa700 1 Processor -- start 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 -- start start 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0100ff8e0 0x7fd010195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0101002a0 0x7fd010195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd010100d20 0x7fd01019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01019d290 con 0x7fd0100ff8e0 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01019d3d0 con 0x7fd010100d20 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01019d560 con 0x7fd0101002a0 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0100ff8e0 0x7fd010195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0100ff8e0 0x7fd010195760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47532/0 (socket says 172.21.15.49:47532) 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 -- 172.21.15.49:0/2654795913 learned_addr learned my addr 172.21.15.49:0/2654795913 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd015895700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0101002a0 0x7fd010195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 -- 172.21.15.49:0/2654795913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0101002a0 msgr2=0x7fd010195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0101002a0 0x7fd010195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 -- 172.21.15.49:0/2654795913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd010100d20 msgr2=0x7fd01019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd010100d20 0x7fd01019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 -- 172.21.15.49:0/2654795913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd008009d00 con 0x7fd0100ff8e0 2023-10-08T09:12:30.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd016096700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0100ff8e0 0x7fd010195760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd000007910 tx=0x7fd000007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:30.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0077fe700 1 -- 172.21.15.49:0/2654795913 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000012070 con 0x7fd0100ff8e0 2023-10-08T09:12:30.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0077fe700 1 -- 172.21.15.49:0/2654795913 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd00000d810 con 0x7fd0100ff8e0 2023-10-08T09:12:30.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0100fe830 con 0x7fd0100ff8e0 2023-10-08T09:12:30.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.766+0000 7fd0077fe700 1 -- 172.21.15.49:0/2654795913 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000005280 con 0x7fd0100ff8e0 2023-10-08T09:12:30.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.767+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0100fed40 con 0x7fd0100ff8e0 2023-10-08T09:12:30.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.768+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01010b830 con 0x7fd0100ff8e0 2023-10-08T09:12:30.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.769+0000 7fd0077fe700 1 -- 172.21.15.49:0/2654795913 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd000016070 con 0x7fd0100ff8e0 2023-10-08T09:12:30.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.769+0000 7fd0077fe700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcffc0a1480 0x7fcffc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:30.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.770+0000 7fd015895700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcffc0a1480 0x7fcffc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:30.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.770+0000 7fd0077fe700 1 -- 172.21.15.49:0/2654795913 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd0000ce210 con 0x7fd0100ff8e0 2023-10-08T09:12:30.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.770+0000 7fd015895700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcffc0a1480 0x7fcffc0a3940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009c60 tx=0x7fd00c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:30.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.773+0000 7fd0077fe700 1 -- 172.21.15.49:0/2654795913 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd000097d10 con 0x7fd0100ff8e0 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:30.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:30.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:30.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:30 smithi132 ceph-mon[103389]: Deploying daemon mds.l on smithi184 2023-10-08T09:12:30.802 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:12:30.802 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2023-10-08T09:12:30.802 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:30.802 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:30.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:30.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:30.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:30.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:30.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:30.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:30.803 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:30 smithi184 ceph-mon[103675]: Deploying daemon mds.l on smithi184 2023-10-08T09:12:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='client.24529 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:12:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: Saving service mds.all spec with placement smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2023-10-08T09:12:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:12:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:30.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:30 smithi049 ceph-mon[99485]: Deploying daemon mds.l on smithi184 2023-10-08T09:12:30.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.977+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd010106510 con 0x7fd0100ff8e0 2023-10-08T09:12:30.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.977+0000 7fd0077fe700 1 -- 172.21.15.49:0/2654795913 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd000007e60 con 0x7fd0100ff8e0 2023-10-08T09:12:30.979 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:12:30.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.979+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcffc0a1480 msgr2=0x7fcffc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.979+0000 7fd0182fa700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcffc0a1480 0x7fcffc0a3940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009c60 tx=0x7fd00c009260 comp rx=0 tx=0).stop 2023-10-08T09:12:30.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.979+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0100ff8e0 msgr2=0x7fd010195760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:30.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.979+0000 7fd0182fa700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0100ff8e0 0x7fd010195760 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd000007910 tx=0x7fd000007c20 comp rx=0 tx=0).stop 2023-10-08T09:12:30.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 shutdown_connections 2023-10-08T09:12:30.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcffc0a1480 0x7fcffc0a3940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd010100d20 0x7fd01019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd0100ff8e0 0x7fd010195760 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 --2- 172.21.15.49:0/2654795913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0101002a0 0x7fd010195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:30.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 >> 172.21.15.49:0/2654795913 conn(0x7fd0100fa390 msgr2=0x7fd010103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:30.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 shutdown_connections 2023-10-08T09:12:30.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:30.980+0000 7fd0182fa700 1 -- 172.21.15.49:0/2654795913 wait complete. 2023-10-08T09:12:30.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.117+0000 7f9308833700 1 -- 172.21.15.49:0/1534599033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 msgr2=0x7f93040fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.117+0000 7f9308833700 1 --2- 172.21.15.49:0/1534599033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f93040fedb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009990 tx=0x7f92f4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 -- 172.21.15.49:0/1534599033 shutdown_connections 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 --2- 172.21.15.49:0/1534599033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93041023a0 0x7f9304108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 --2- 172.21.15.49:0/1534599033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f93040fedb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 --2- 172.21.15.49:0/1534599033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9304101a00 0x7f9304101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 -- 172.21.15.49:0/1534599033 >> 172.21.15.49:0/1534599033 conn(0x7f93040fa2c0 msgr2=0x7f93040fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 -- 172.21.15.49:0/1534599033 shutdown_connections 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 -- 172.21.15.49:0/1534599033 wait complete. 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 Processor -- start 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.118+0000 7f9308833700 1 -- start start 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9308833700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f9304196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:31.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9308833700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9304101a00 0x7f9304196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9308833700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93041023a0 0x7f930419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9308833700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930419e1a0 con 0x7f93040fe990 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9308833700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930419e2e0 con 0x7f93041023a0 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9308833700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930419e470 con 0x7f9304101a00 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f9304196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f9304196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47546/0 (socket says 172.21.15.49:47546) 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 -- 172.21.15.49:0/4063000203 learned_addr learned my addr 172.21.15.49:0/4063000203 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9301d9b700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9304101a00 0x7f9304196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f9302d9d700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93041023a0 0x7f930419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 -- 172.21.15.49:0/4063000203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9304101a00 msgr2=0x7f9304196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9304101a00 0x7f9304196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 -- 172.21.15.49:0/4063000203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93041023a0 msgr2=0x7f930419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93041023a0 0x7f930419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 -- 172.21.15.49:0/4063000203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40095f0 con 0x7f93040fe990 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.119+0000 7f930259c700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f9304196670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b680 tx=0x7f92f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.120+0000 7f92fb7fe700 1 -- 172.21.15.49:0/4063000203 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f401d070 con 0x7f93040fe990 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.120+0000 7f92fb7fe700 1 -- 172.21.15.49:0/4063000203 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f400fbd0 con 0x7f93040fe990 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.120+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930419e6f0 con 0x7f93040fe990 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.120+0000 7f92fb7fe700 1 -- 172.21.15.49:0/4063000203 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4017650 con 0x7f93040fe990 2023-10-08T09:12:31.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.120+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930419eb10 con 0x7f93040fe990 2023-10-08T09:12:31.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.121+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304107c30 con 0x7f93040fe990 2023-10-08T09:12:31.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.122+0000 7f92fb7fe700 1 -- 172.21.15.49:0/4063000203 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f92f4021410 con 0x7f93040fe990 2023-10-08T09:12:31.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.122+0000 7f92fb7fe700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f92f00a1480 0x7f92f00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:31.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.122+0000 7f92fb7fe700 1 -- 172.21.15.49:0/4063000203 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f92f40ce070 con 0x7f93040fe990 2023-10-08T09:12:31.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.122+0000 7f9301d9b700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f92f00a1480 0x7f92f00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:31.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.123+0000 7f9301d9b700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f92f00a1480 0x7f92f00a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009c60 tx=0x7f92ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:31.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.126+0000 7f92fb7fe700 1 -- 172.21.15.49:0/4063000203 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f92f4097c00 con 0x7f93040fe990 2023-10-08T09:12:31.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.373+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9304066bc0 con 0x7f93040fe990 2023-10-08T09:12:31.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.374+0000 7f92fb7fe700 1 -- 172.21.15.49:0/4063000203 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f92f409b220 con 0x7f93040fe990 2023-10-08T09:12:31.375 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:31.375 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-08T09:12:31.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f92f00a1480 msgr2=0x7f92f00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:31.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f92f00a1480 0x7f92f00a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009c60 tx=0x7f92ec009260 comp rx=0 tx=0).stop 2023-10-08T09:12:31.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 msgr2=0x7f9304196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:31.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f9304196670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b680 tx=0x7f92f4004730 comp rx=0 tx=0).stop 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 shutdown_connections 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f92f00a1480 0x7f92f00a3940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93041023a0 0x7f930419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93040fe990 0x7f9304196670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 --2- 172.21.15.49:0/4063000203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9304101a00 0x7f9304196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 >> 172.21.15.49:0/4063000203 conn(0x7f93040fa2c0 msgr2=0x7f9304104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 shutdown_connections 2023-10-08T09:12:31.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:31.376+0000 7f9308833700 1 -- 172.21.15.49:0/4063000203 wait complete. 2023-10-08T09:12:31.379 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 1 2023-10-08T09:12:31.444 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:12:31 smithi184 systemd[1]: Starting Ceph mds.l for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:31.701 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:12:31 smithi184 podman[140910]: 2023-10-08T09:12:31.701 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:31 smithi184 ceph-mon[103675]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:31.701 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3425992466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:12:31.701 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2654795913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:12:31.701 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4063000203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:31 smithi049 ceph-mon[99485]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3425992466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:12:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2654795913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:12:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4063000203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:31.997 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:12:31 smithi184 ceph-mds[140951]: -- 172.21.15.184:0/2962393136 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564a621e91e0 con 0x564a621bf000 2023-10-08T09:12:31.997 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:12:31 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-l[140926]: starting mds.l at 2023-10-08T09:12:31.997 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:12:31 smithi184 bash[140910]: 08b8cfe48cc19da52e1ee2f63aeae5e25143b54099cd4e0617e97fe59ce129f4 2023-10-08T09:12:31.997 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:12:31 smithi184 systemd[1]: Started Ceph mds.l for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:31 smithi132 ceph-mon[103389]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3425992466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:12:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2654795913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:12:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4063000203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:32.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] up:boot 2023-10-08T09:12:32.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: fsmap 1 up:standby 2023-10-08T09:12:32.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:12:32.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:32.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:32.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:32.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:32.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:32.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:32.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:32.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:32 smithi132 ceph-mon[103389]: Deploying daemon mds.k on smithi132 2023-10-08T09:12:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] up:boot 2023-10-08T09:12:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: fsmap 1 up:standby 2023-10-08T09:12:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:12:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:32.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:32 smithi184 ceph-mon[103675]: Deploying daemon mds.k on smithi132 2023-10-08T09:12:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] up:boot 2023-10-08T09:12:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: fsmap 1 up:standby 2023-10-08T09:12:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:12:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:33.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:33.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:33.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:33.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:32 smithi049 ceph-mon[99485]: Deploying daemon mds.k on smithi132 2023-10-08T09:12:33.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:33.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/3804194726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 msgr2=0x7fbc180ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/3804194726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc180ff9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0099c0 tx=0x7fbc0c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/3804194726 shutdown_connections 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/3804194726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc18101130 0x7fbc181079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/3804194726 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc180ff9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/3804194726 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc18100790 0x7fbc18100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/3804194726 >> 172.21.15.49:0/3804194726 conn(0x7fbc180fab10 msgr2=0x7fbc180fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/3804194726 shutdown_connections 2023-10-08T09:12:33.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.531+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/3804194726 wait complete. 2023-10-08T09:12:33.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 Processor -- start 2023-10-08T09:12:33.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 -- start start 2023-10-08T09:12:33.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc18068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:33.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc18100790 0x7fbc1819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc18101130 0x7fbc1819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1819e0e0 con 0x7fbc180ff5a0 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1819e250 con 0x7fbc18100790 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1c9c9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1819e410 con 0x7fbc18101130 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc18068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc18068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47560/0 (socket says 172.21.15.49:47560) 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1659c700 1 -- 172.21.15.49:0/2152068935 learned_addr learned my addr 172.21.15.49:0/2152068935 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc15d9b700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc18100790 0x7fbc1819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc16d9d700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc18101130 0x7fbc1819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1659c700 1 -- 172.21.15.49:0/2152068935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc18101130 msgr2=0x7fbc1819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1659c700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc18101130 0x7fbc1819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1659c700 1 -- 172.21.15.49:0/2152068935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc18100790 msgr2=0x7fbc1819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.532+0000 7fbc1659c700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc18100790 0x7fbc1819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc1659c700 1 -- 172.21.15.49:0/2152068935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0095f0 con 0x7fbc180ff5a0 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc15d9b700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc18100790 0x7fbc1819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc1659c700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc18068370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0058e0 tx=0x7fbc0c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc077fe700 1 -- 172.21.15.49:0/2152068935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c01d070 con 0x7fbc180ff5a0 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1819e630 con 0x7fbc180ff5a0 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc077fe700 1 -- 172.21.15.49:0/2152068935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc0c00fe70 con 0x7fbc180ff5a0 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc077fe700 1 -- 172.21.15.49:0/2152068935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0049d0 con 0x7fbc180ff5a0 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.533+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1819eb30 con 0x7fbc180ff5a0 2023-10-08T09:12:33.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.534+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc181069d0 con 0x7fbc180ff5a0 2023-10-08T09:12:33.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.535+0000 7fbc077fe700 1 -- 172.21.15.49:0/2152068935 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbc0c004b90 con 0x7fbc180ff5a0 2023-10-08T09:12:33.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.536+0000 7fbc077fe700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc000a1480 0x7fbc000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:33.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.536+0000 7fbc077fe700 1 -- 172.21.15.49:0/2152068935 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc0c0cd9c0 con 0x7fbc180ff5a0 2023-10-08T09:12:33.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.536+0000 7fbc15d9b700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc000a1480 0x7fbc000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:33.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.536+0000 7fbc15d9b700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc000a1480 0x7fbc000a3940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbc18068910 tx=0x7fbc08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:33.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.539+0000 7fbc077fe700 1 -- 172.21.15.49:0/2152068935 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc0c097550 con 0x7fbc180ff5a0 2023-10-08T09:12:33.547 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:12:33 smithi132 systemd[1]: Starting Ceph mds.k for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.784+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc18067450 con 0x7fbc180ff5a0 2023-10-08T09:12:33.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.785+0000 7fbc077fe700 1 -- 172.21.15.49:0/2152068935 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fbc0c09ab70 con 0x7fbc180ff5a0 2023-10-08T09:12:33.785 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:33.786 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:12:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc000a1480 msgr2=0x7fbc000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc000a1480 0x7fbc000a3940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbc18068910 tx=0x7fbc08009260 comp rx=0 tx=0).stop 2023-10-08T09:12:33.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 msgr2=0x7fbc18068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc18068370 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0058e0 tx=0x7fbc0c00fa00 comp rx=0 tx=0).stop 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 shutdown_connections 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbc000a1480 0x7fbc000a3940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc18101130 0x7fbc1819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbc180ff5a0 0x7fbc18068370 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 --2- 172.21.15.49:0/2152068935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc18100790 0x7fbc1819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 >> 172.21.15.49:0/2152068935 conn(0x7fbc180fab10 msgr2=0x7fbc18103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 shutdown_connections 2023-10-08T09:12:33.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:33.787+0000 7fbc1c9c9700 1 -- 172.21.15.49:0/2152068935 wait complete. 2023-10-08T09:12:33.789 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2023-10-08T09:12:33.981 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:33 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2152068935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:33.982 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:12:33 smithi132 podman[138393]: 2023-10-08T09:12:33.982 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:12:33 smithi132 ceph-mds[138433]: -- 172.21.15.132:0/3166869819 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55845bac91e0 con 0x55845ba9f000 2023-10-08T09:12:33.982 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:12:33 smithi132 bash[138393]: 8ac49e162211d4afb7d8fb0978406639bc9448546743614d4a9a05d2085ce191 2023-10-08T09:12:33.982 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:12:33 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-k[138409]: starting mds.k at 2023-10-08T09:12:34.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:33 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2152068935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:34.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:33 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2152068935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:34.297 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:12:33 smithi132 systemd[1]: Started Ceph mds.k for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: Deploying daemon mds.j on smithi049 2023-10-08T09:12:35.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] up:boot 2023-10-08T09:12:35.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: fsmap 2 up:standby 2023-10-08T09:12:35.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:12:35.327 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:35.327 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.327 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.327 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.327 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:35.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:35.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:35.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: Deploying daemon mds.j on smithi049 2023-10-08T09:12:35.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] up:boot 2023-10-08T09:12:35.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: fsmap 2 up:standby 2023-10-08T09:12:35.328 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:12:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: Deploying daemon mds.j on smithi049 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] up:boot 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: fsmap 2 up:standby 2023-10-08T09:12:35.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:12:35.632 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:12:35 smithi049 systemd[1]: Starting Ceph mds.j for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:35.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.936+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3582434274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb41006b0 msgr2=0x7fcdb4100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.936+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3582434274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb41006b0 0x7fcdb4100ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcda40099c0 tx=0x7fcda4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3582434274 shutdown_connections 2023-10-08T09:12:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3582434274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb4102240 0x7fcdb4108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3582434274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb41006b0 0x7fcdb4100ad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3582434274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb41018a0 0x7fcdb4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3582434274 >> 172.21.15.49:0/3582434274 conn(0x7fcdb40fbc20 msgr2=0x7fcdb40fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3582434274 shutdown_connections 2023-10-08T09:12:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3582434274 wait complete. 2023-10-08T09:12:35.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 Processor -- start 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.937+0000 7fcdba5f4700 1 -- start start 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdba5f4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb41006b0 0x7fcdb41944e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdba5f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb41018a0 0x7fcdb41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdba5f4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb4102240 0x7fcdb4194b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdba5f4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb41acfa0 con 0x7fcdb4102240 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdba5f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb41ad110 con 0x7fcdb41018a0 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdba5f4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb41ad2d0 con 0x7fcdb41006b0 2023-10-08T09:12:35.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb4102240 0x7fcdb4194b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb4102240 0x7fcdb4194b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47574/0 (socket says 172.21.15.49:47574) 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 -- 172.21.15.49:0/3776596508 learned_addr learned my addr 172.21.15.49:0/3776596508 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb3fff700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb41006b0 0x7fcdb41944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb37fe700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb41018a0 0x7fcdb41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 -- 172.21.15.49:0/3776596508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb41006b0 msgr2=0x7fcdb41944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb41006b0 0x7fcdb41944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 -- 172.21.15.49:0/3776596508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb41018a0 msgr2=0x7fcdb41999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:35.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb41018a0 0x7fcdb41999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb8b91700 1 -- 172.21.15.49:0/3776596508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40095f0 con 0x7fcdb4102240 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb3fff700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb41006b0 0x7fcdb41944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.938+0000 7fcdb37fe700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb41018a0 0x7fcdb41999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.939+0000 7fcdb8b91700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb4102240 0x7fcdb4194b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcdac007ee0 tx=0x7fcdac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.939+0000 7fcdb17fa700 1 -- 172.21.15.49:0/3776596508 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ddd0 con 0x7fcdb4102240 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.939+0000 7fcdb17fa700 1 -- 172.21.15.49:0/3776596508 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdac00f040 con 0x7fcdb4102240 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.939+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb41ad5b0 con 0x7fcdb4102240 2023-10-08T09:12:35.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.939+0000 7fcdb17fa700 1 -- 172.21.15.49:0/3776596508 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac015400 con 0x7fcdb4102240 2023-10-08T09:12:35.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.939+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb41ada70 con 0x7fcdb4102240 2023-10-08T09:12:35.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.940+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb40678a0 con 0x7fcdb4102240 2023-10-08T09:12:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.941+0000 7fcdb17fa700 1 -- 172.21.15.49:0/3776596508 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcdac004000 con 0x7fcdb4102240 2023-10-08T09:12:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.942+0000 7fcdb17fa700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd9c0a1480 0x7fcd9c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.942+0000 7fcdb3fff700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd9c0a1480 0x7fcd9c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.942+0000 7fcdb17fa700 1 -- 172.21.15.49:0/3776596508 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcdac0cca30 con 0x7fcdb4102240 2023-10-08T09:12:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.942+0000 7fcdb3fff700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd9c0a1480 0x7fcd9c0a3940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009990 tx=0x7fcda401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:35.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:35.946+0000 7fcdb17fa700 1 -- 172.21.15.49:0/3776596508 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcdac096530 con 0x7fcdb4102240 2023-10-08T09:12:36.132 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:12:35 smithi049 podman[153807]: 2023-10-08T09:12:36.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.230+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdb410c160 con 0x7fcdb4102240 2023-10-08T09:12:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.230+0000 7fcdb17fa700 1 -- 172.21.15.49:0/3776596508 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fcdac099b50 con 0x7fcdb4102240 2023-10-08T09:12:36.231 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:36.231 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:12:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.232+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd9c0a1480 msgr2=0x7fcd9c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.232+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd9c0a1480 0x7fcd9c0a3940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009990 tx=0x7fcda401a040 comp rx=0 tx=0).stop 2023-10-08T09:12:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.232+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb4102240 msgr2=0x7fcdb4194b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.232+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb4102240 0x7fcdb4194b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcdac007ee0 tx=0x7fcdac00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:12:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 shutdown_connections 2023-10-08T09:12:36.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd9c0a1480 0x7fcd9c0a3940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdb41006b0 0x7fcdb41944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdb4102240 0x7fcdb4194b10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 --2- 172.21.15.49:0/3776596508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb41018a0 0x7fcdb41999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 >> 172.21.15.49:0/3776596508 conn(0x7fcdb40fbc20 msgr2=0x7fcdb4072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 shutdown_connections 2023-10-08T09:12:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:36.233+0000 7fcdba5f4700 1 -- 172.21.15.49:0/3776596508 wait complete. 2023-10-08T09:12:36.234 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2023-10-08T09:12:36.632 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:12:36 smithi049 bash[153807]: 71c18cc96bf948fcf86f44e355a0f427dd314f94d8b8eac7909e243a55f62e3c 2023-10-08T09:12:36.632 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:12:36 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-j[153846]: starting mds.j at 2023-10-08T09:12:36.632 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:12:36 smithi049 systemd[1]: Started Ceph mds.j for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:37.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:37.479 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3776596508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:37.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:37.480 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:37.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:37.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: Deploying daemon mds.d on smithi049 2023-10-08T09:12:37.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.49:6834/3458420640,v1:172.21.15.49:6835/3458420640] up:boot 2023-10-08T09:12:37.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: fsmap 3 up:standby 2023-10-08T09:12:37.481 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T09:12:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3776596508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: Deploying daemon mds.d on smithi049 2023-10-08T09:12:37.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.49:6834/3458420640,v1:172.21.15.49:6835/3458420640] up:boot 2023-10-08T09:12:37.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: fsmap 3 up:standby 2023-10-08T09:12:37.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T09:12:37.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:37.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3776596508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:37.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:37.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:37.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:37.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:37.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: Deploying daemon mds.d on smithi049 2023-10-08T09:12:37.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.49:6834/3458420640,v1:172.21.15.49:6835/3458420640] up:boot 2023-10-08T09:12:37.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: fsmap 3 up:standby 2023-10-08T09:12:37.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-08T09:12:38.132 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:12:37 smithi049 systemd[1]: Starting Ceph mds.d for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:38.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:38.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 -- 172.21.15.49:0/1568639406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac1011b0 msgr2=0x7fcdac1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:38.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 --2- 172.21.15.49:0/1568639406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac1011b0 0x7fcdac1035a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009990 tx=0x7fcda0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 -- 172.21.15.49:0/1568639406 shutdown_connections 2023-10-08T09:12:38.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 --2- 172.21.15.49:0/1568639406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdac106410 0x7fcdac108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 --2- 172.21.15.49:0/1568639406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac1011b0 0x7fcdac1035a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 --2- 172.21.15.49:0/1568639406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdac103ae0 0x7fcdac105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 -- 172.21.15.49:0/1568639406 >> 172.21.15.49:0/1568639406 conn(0x7fcdac0fab40 msgr2=0x7fcdac0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:38.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 -- 172.21.15.49:0/1568639406 shutdown_connections 2023-10-08T09:12:38.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.383+0000 7fcdb0994700 1 -- 172.21.15.49:0/1568639406 wait complete. 2023-10-08T09:12:38.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.384+0000 7fcdb0994700 1 Processor -- start 2023-10-08T09:12:38.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.384+0000 7fcdb0994700 1 -- start start 2023-10-08T09:12:38.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdac1011b0 0x7fcdac196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:38.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdac103ae0 0x7fcdac196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:38.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac106410 0x7fcdac19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac19e140 con 0x7fcdac106410 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac19e280 con 0x7fcdac103ae0 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac19e410 con 0x7fcdac1011b0 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac106410 0x7fcdac19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac106410 0x7fcdac19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47604/0 (socket says 172.21.15.49:47604) 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 -- 172.21.15.49:0/3149922560 learned_addr learned my addr 172.21.15.49:0/3149922560 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcda9d9b700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdac103ae0 0x7fcdac196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaa59c700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdac1011b0 0x7fcdac196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 -- 172.21.15.49:0/3149922560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdac1011b0 msgr2=0x7fcdac196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdac1011b0 0x7fcdac196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 -- 172.21.15.49:0/3149922560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdac103ae0 msgr2=0x7fcdac196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:38.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdac103ae0 0x7fcdac196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 -- 172.21.15.49:0/3149922560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00095f0 con 0x7fcdac106410 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdaad9d700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac106410 0x7fcdac19db70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcda4007ee0 tx=0x7fcda400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcd9b7fe700 1 -- 172.21.15.49:0/3149922560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4017070 con 0x7fcdac106410 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcd9b7fe700 1 -- 172.21.15.49:0/3149922560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcda400f040 con 0x7fcdac106410 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac19e6f0 con 0x7fcdac106410 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcd9b7fe700 1 -- 172.21.15.49:0/3149922560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4005740 con 0x7fcdac106410 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.385+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac19ebb0 con 0x7fcdac106410 2023-10-08T09:12:38.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.387+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac0fc6c0 con 0x7fcdac106410 2023-10-08T09:12:38.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.387+0000 7fcd9b7fe700 1 -- 172.21.15.49:0/3149922560 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcda400dc60 con 0x7fcdac106410 2023-10-08T09:12:38.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.388+0000 7fcd9b7fe700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd940a1480 0x7fcd940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:38.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.388+0000 7fcdaa59c700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd940a1480 0x7fcd940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:38.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.388+0000 7fcd9b7fe700 1 -- 172.21.15.49:0/3149922560 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcda40cc130 con 0x7fcdac106410 2023-10-08T09:12:38.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.389+0000 7fcdaa59c700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd940a1480 0x7fcd940a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcda0006010 tx=0x7fcda00037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.393+0000 7fcd9b7fe700 1 -- 172.21.15.49:0/3149922560 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcda4095b90 con 0x7fcdac106410 2023-10-08T09:12:38.442 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:12:38 smithi049 podman[154192]: 2023-10-08T09:12:38.443 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:12:38 smithi049 ceph-mds[154275]: -- 172.21.15.49:0/2490763758 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5573e56a51e0 con 0x5573e567a800 2023-10-08T09:12:38.443 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:12:38 smithi049 bash[154192]: f31b5393a2a06c637a092afc14db565f33b9ddded29a45240b0f198826c0f44b 2023-10-08T09:12:38.443 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:12:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-d[154214]: starting mds.d at 2023-10-08T09:12:38.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.674+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdac0678a0 con 0x7fcdac106410 2023-10-08T09:12:38.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.675+0000 7fcd9b7fe700 1 -- 172.21.15.49:0/3149922560 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7fcda40991b0 con 0x7fcdac106410 2023-10-08T09:12:38.676 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:38.676 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:12:38.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd940a1480 msgr2=0x7fcd940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:38.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd940a1480 0x7fcd940a3940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcda0006010 tx=0x7fcda00037b0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac106410 msgr2=0x7fcdac19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac106410 0x7fcdac19db70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcda4007ee0 tx=0x7fcda400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 shutdown_connections 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcd940a1480 0x7fcd940a3940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdac1011b0 0x7fcdac196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdac106410 0x7fcdac19db70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 --2- 172.21.15.49:0/3149922560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdac103ae0 0x7fcdac196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 >> 172.21.15.49:0/3149922560 conn(0x7fcdac0fab40 msgr2=0x7fcdac101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 shutdown_connections 2023-10-08T09:12:38.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:38.677+0000 7fcdb0994700 1 -- 172.21.15.49:0/3149922560 wait complete. 2023-10-08T09:12:38.679 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 4 2023-10-08T09:12:38.882 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:12:38 smithi049 systemd[1]: Started Ceph mds.d for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: Deploying daemon mds.e on smithi132 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3149922560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.49:6836/1210414617,v1:172.21.15.49:6837/1210414617] up:boot 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: fsmap 4 up:standby 2023-10-08T09:12:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T09:12:39.786 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:39.786 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.786 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.786 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.786 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:39.786 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:39.786 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:39.787 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: Deploying daemon mds.e on smithi132 2023-10-08T09:12:39.787 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3149922560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:39.787 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.49:6836/1210414617,v1:172.21.15.49:6837/1210414617] up:boot 2023-10-08T09:12:39.787 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: fsmap 4 up:standby 2023-10-08T09:12:39.787 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T09:12:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: Deploying daemon mds.e on smithi132 2023-10-08T09:12:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3149922560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.49:6836/1210414617,v1:172.21.15.49:6837/1210414617] up:boot 2023-10-08T09:12:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: fsmap 4 up:standby 2023-10-08T09:12:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-08T09:12:40.047 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:12:40 smithi132 systemd[1]: Starting Ceph mds.e for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:40.563 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:12:40 smithi132 podman[138729]: 2023-10-08T09:12:40.691 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 -- 172.21.15.49:0/2234317322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79180ff5c0 msgr2=0x7f79180ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 --2- 172.21.15.49:0/2234317322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79180ff5c0 0x7f79180ff9e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79080099c0 tx=0x7f7908009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:40.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 -- 172.21.15.49:0/2234317322 shutdown_connections 2023-10-08T09:12:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 --2- 172.21.15.49:0/2234317322 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7918101150 0x7f7918107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 --2- 172.21.15.49:0/2234317322 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79180ff5c0 0x7f79180ff9e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 --2- 172.21.15.49:0/2234317322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79181007b0 0x7f7918100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 -- 172.21.15.49:0/2234317322 >> 172.21.15.49:0/2234317322 conn(0x7f79180fab30 msgr2=0x7f79180fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.831+0000 7f791e066700 1 -- 172.21.15.49:0/2234317322 shutdown_connections 2023-10-08T09:12:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 -- 172.21.15.49:0/2234317322 wait complete. 2023-10-08T09:12:40.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 Processor -- start 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 -- start start 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79180ff5c0 0x7f79181988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79181007b0 0x7f7918198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7918101150 0x7f791819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79181a03d0 con 0x7f79180ff5c0 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79181a0510 con 0x7f79181007b0 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.832+0000 7f791e066700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79181a06a0 con 0x7f7918101150 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79181007b0 0x7f7918198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:40.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f79177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79180ff5c0 0x7f79181988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79181007b0 0x7f7918198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:37244/0 (socket says 172.21.15.49:37244) 2023-10-08T09:12:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 -- 172.21.15.49:0/997598843 learned_addr learned my addr 172.21.15.49:0/997598843 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7917fff700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7918101150 0x7f791819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 -- 172.21.15.49:0/997598843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7918101150 msgr2=0x7f791819fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7918101150 0x7f791819fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:40.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 -- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79180ff5c0 msgr2=0x7f79181988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79180ff5c0 0x7f79181988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 -- 172.21.15.49:0/997598843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910009c40 con 0x7f79181007b0 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7916ffd700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79181007b0 0x7f7918198de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f790c00e9a0 tx=0x7f790c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7914ff9700 1 -- 172.21.15.49:0/997598843 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c009790 con 0x7f79181007b0 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7914ff9700 1 -- 172.21.15.49:0/997598843 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f790c00ce80 con 0x7f79181007b0 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79080095f0 con 0x7f79181007b0 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f7914ff9700 1 -- 172.21.15.49:0/997598843 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c018680 con 0x7f79181007b0 2023-10-08T09:12:40.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.833+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79181a0d50 con 0x7f79181007b0 2023-10-08T09:12:40.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.834+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7918066bc0 con 0x7f79181007b0 2023-10-08T09:12:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.835+0000 7f7914ff9700 1 -- 172.21.15.49:0/997598843 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f790c010430 con 0x7f79181007b0 2023-10-08T09:12:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.836+0000 7f7914ff9700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f79000a1500 0x7f79000a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:40.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.836+0000 7f79177fe700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f79000a1500 0x7f79000a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:40.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.836+0000 7f7914ff9700 1 -- 172.21.15.49:0/997598843 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f790c014070 con 0x7f79181007b0 2023-10-08T09:12:40.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.836+0000 7f79177fe700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f79000a1500 0x7f79000a39c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7908009990 tx=0x7f7908000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:40.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:40.839+0000 7f7914ff9700 1 -- 172.21.15.49:0/997598843 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f790c096650 con 0x7f79181007b0 2023-10-08T09:12:41.047 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:12:40 smithi132 ceph-mds[138769]: -- 172.21.15.132:0/1250924775 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559ce3b711e0 con 0x559ce3b46c00 2023-10-08T09:12:41.047 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:12:40 smithi132 bash[138729]: fa3baf1f74625630fa9664ddbd405b29ff372def66270dbc2ff4fb573767efda 2023-10-08T09:12:41.047 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:12:40 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-e[138745]: starting mds.e at 2023-10-08T09:12:41.047 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:12:40 smithi132 systemd[1]: Started Ceph mds.e for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:41.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.089+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f791810ae30 con 0x7f79181007b0 2023-10-08T09:12:41.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.090+0000 7f7914ff9700 1 -- 172.21.15.49:0/997598843 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f790c099c70 con 0x7f79181007b0 2023-10-08T09:12:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:41.091 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T09:12:41.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f79000a1500 msgr2=0x7f79000a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:41.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f79000a1500 0x7f79000a39c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7908009990 tx=0x7f7908000bc0 comp rx=0 tx=0).stop 2023-10-08T09:12:41.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79181007b0 msgr2=0x7f7918198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:41.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79181007b0 0x7f7918198de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f790c00e9a0 tx=0x7f790c00ecb0 comp rx=0 tx=0).stop 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 shutdown_connections 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f79000a1500 0x7f79000a39c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7918101150 0x7f791819fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f79180ff5c0 0x7f79181988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 --2- 172.21.15.49:0/997598843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79181007b0 0x7f7918198de0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 >> 172.21.15.49:0/997598843 conn(0x7f79180fab30 msgr2=0x7f7918068aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 shutdown_connections 2023-10-08T09:12:41.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:41.093+0000 7f791e066700 1 -- 172.21.15.49:0/997598843 wait complete. 2023-10-08T09:12:41.095 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 5 2023-10-08T09:12:41.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:41.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:41.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:41.968 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: Deploying daemon mds.f on smithi184 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/997598843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6834/738661164,v1:172.21.15.132:6835/738661164] up:boot 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: fsmap 5 up:standby 2023-10-08T09:12:41.969 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:12:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:42.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:42.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: Deploying daemon mds.f on smithi184 2023-10-08T09:12:42.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/997598843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:42.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6834/738661164,v1:172.21.15.132:6835/738661164] up:boot 2023-10-08T09:12:42.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: fsmap 5 up:standby 2023-10-08T09:12:42.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:12:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:42.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: Deploying daemon mds.f on smithi184 2023-10-08T09:12:42.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/997598843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:42.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6834/738661164,v1:172.21.15.132:6835/738661164] up:boot 2023-10-08T09:12:42.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: fsmap 5 up:standby 2023-10-08T09:12:42.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:12:42.247 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:12:42 smithi184 systemd[1]: Starting Ceph mds.f for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:42.769 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:12:42 smithi184 podman[141250]: 2023-10-08T09:12:43.107 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:43.247 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:12:42 smithi184 bash[141250]: 83239c99ec7059c7f4f5b44d9889b847e286ccdfcaa60900dfda6cf2dcab3082 2023-10-08T09:12:43.247 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:12:42 smithi184 ceph-mds[141290]: -- 172.21.15.184:0/1456482373 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c7c088b1e0 con 0x55c7c0861000 2023-10-08T09:12:43.247 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:12:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-f[141266]: starting mds.f at 2023-10-08T09:12:43.247 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:12:42 smithi184 systemd[1]: Started Ceph mds.f for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2355280101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb81018b0 msgr2=0x7fadb8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2355280101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb81018b0 0x7fadb8101d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fada40099c0 tx=0x7fada4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2355280101 shutdown_connections 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2355280101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb81006c0 0x7fadb8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2355280101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadb8102250 0x7fadb8108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2355280101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb81018b0 0x7fadb8101d10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2355280101 >> 172.21.15.49:0/2355280101 conn(0x7fadb80fbc50 msgr2=0x7fadb80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2355280101 shutdown_connections 2023-10-08T09:12:43.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2355280101 wait complete. 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 Processor -- start 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.247+0000 7fadbeaa8700 1 -- start start 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbeaa8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadb81006c0 0x7fadb8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbeaa8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb81018b0 0x7fadb8196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbeaa8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb8102250 0x7fadb819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbeaa8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb819e1a0 con 0x7fadb8102250 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbeaa8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb819e2e0 con 0x7fadb81006c0 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbeaa8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb819e470 con 0x7fadb81018b0 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb8102250 0x7fadb819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb8102250 0x7fadb819dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47646/0 (socket says 172.21.15.49:47646) 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 -- 172.21.15.49:0/2361907331 learned_addr learned my addr 172.21.15.49:0/2361907331 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:43.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadb7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb81018b0 0x7fadb8196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbc844700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadb81006c0 0x7fadb8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 -- 172.21.15.49:0/2361907331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb81018b0 msgr2=0x7fadb8196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb81018b0 0x7fadb8196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 -- 172.21.15.49:0/2361907331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadb81006c0 msgr2=0x7fadb8196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadb81006c0 0x7fadb8196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 -- 172.21.15.49:0/2361907331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40095f0 con 0x7fadb8102250 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbd045700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb8102250 0x7fadb819dbd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fadb0007ee0 tx=0x7fadb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadb5ffb700 1 -- 172.21.15.49:0/2361907331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0017070 con 0x7fadb8102250 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadb5ffb700 1 -- 172.21.15.49:0/2361907331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadb000f040 con 0x7fadb8102250 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb819e750 con 0x7fadb8102250 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.248+0000 7fadb5ffb700 1 -- 172.21.15.49:0/2361907331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0005740 con 0x7fadb8102250 2023-10-08T09:12:43.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.249+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb819ec60 con 0x7fadb8102250 2023-10-08T09:12:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.250+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb8107b10 con 0x7fadb8102250 2023-10-08T09:12:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.251+0000 7fadb5ffb700 1 -- 172.21.15.49:0/2361907331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fadb000dc60 con 0x7fadb8102250 2023-10-08T09:12:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.251+0000 7fadb5ffb700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fada80a1480 0x7fada80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:43.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.251+0000 7fadb5ffb700 1 -- 172.21.15.49:0/2361907331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fadb00cbff0 con 0x7fadb8102250 2023-10-08T09:12:43.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.251+0000 7fadbc844700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fada80a1480 0x7fada80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:43.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.252+0000 7fadbc844700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fada80a1480 0x7fada80a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadb8101710 tx=0x7fadac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:43.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.256+0000 7fadb5ffb700 1 -- 172.21.15.49:0/2361907331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fadb0095af0 con 0x7fadb8102250 2023-10-08T09:12:43.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.510+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadb8066bc0 con 0x7fadb8102250 2023-10-08T09:12:43.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.510+0000 7fadb5ffb700 1 -- 172.21.15.49:0/2361907331 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7fadb0099110 con 0x7fadb8102250 2023-10-08T09:12:43.511 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:43.511 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-08T09:12:43.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fada80a1480 msgr2=0x7fada80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:43.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fada80a1480 0x7fada80a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fadb8101710 tx=0x7fadac009260 comp rx=0 tx=0).stop 2023-10-08T09:12:43.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb8102250 msgr2=0x7fadb819dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb8102250 0x7fadb819dbd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fadb0007ee0 tx=0x7fadb000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 shutdown_connections 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fada80a1480 0x7fada80a3940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadb8102250 0x7fadb819dbd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fadb81006c0 0x7fadb8196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 --2- 172.21.15.49:0/2361907331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb81018b0 0x7fadb8196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 >> 172.21.15.49:0/2361907331 conn(0x7fadb80fbc50 msgr2=0x7fadb8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 shutdown_connections 2023-10-08T09:12:43.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:43.513+0000 7fadbeaa8700 1 -- 172.21.15.49:0/2361907331 wait complete. 2023-10-08T09:12:43.515 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 6 2023-10-08T09:12:43.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:43.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:43.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:43.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:43.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:43.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:43.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:43.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:43 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2361907331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:43.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:43 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2361907331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:44.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:44.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:44.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:44.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:44.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:44.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:44.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:44.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:43 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2361907331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:44.497 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:12:44 smithi184 systemd[1]: Starting Ceph mds.c for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:44.886 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:44 smithi184 ceph-mon[103675]: Deploying daemon mds.c on smithi184 2023-10-08T09:12:44.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:44 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6834/356478734,v1:172.21.15.184:6835/356478734] up:boot 2023-10-08T09:12:44.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:44 smithi184 ceph-mon[103675]: fsmap 6 up:standby 2023-10-08T09:12:44.938 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:12:44.938 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:12:44 smithi184 podman[141587]: 2023-10-08T09:12:45.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:44 smithi132 ceph-mon[103389]: Deploying daemon mds.c on smithi184 2023-10-08T09:12:45.131 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:44 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6834/356478734,v1:172.21.15.184:6835/356478734] up:boot 2023-10-08T09:12:45.131 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:44 smithi132 ceph-mon[103389]: fsmap 6 up:standby 2023-10-08T09:12:45.131 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:12:45.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:44 smithi049 ceph-mon[99485]: Deploying daemon mds.c on smithi184 2023-10-08T09:12:45.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:44 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6834/356478734,v1:172.21.15.184:6835/356478734] up:boot 2023-10-08T09:12:45.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:44 smithi049 ceph-mon[99485]: fsmap 6 up:standby 2023-10-08T09:12:45.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:12:45.247 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:12:44 smithi184 ceph-mds[141627]: -- 172.21.15.184:0/2569372057 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55da4e21f1e0 con 0x55da4e1f4c00 2023-10-08T09:12:45.268 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:12:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-c[141603]: starting mds.c at 2023-10-08T09:12:45.268 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:12:44 smithi184 bash[141587]: ebe413d6db3024696f770257ad1b9560c8422673e1328387bb05ce96adb88f96 2023-10-08T09:12:45.268 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:12:44 smithi184 systemd[1]: Started Ceph mds.c for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:45.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:46.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:45 smithi049 ceph-mon[99485]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:46.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.590 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:45 smithi184 ceph-mon[103675]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:46.591 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:45 smithi132 ceph-mon[103389]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:46.592 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:12:46 smithi132 systemd[1]: Starting Ceph mds.h for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:46.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 -- 172.21.15.49:0/505019967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3080ff5b0 msgr2=0x7ff3080ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/505019967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3080ff5b0 0x7ff3080ff9d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80099c0 tx=0x7ff2f8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 -- 172.21.15.49:0/505019967 shutdown_connections 2023-10-08T09:12:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/505019967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff308101140 0x7ff3081079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/505019967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3080ff5b0 0x7ff3080ff9d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/505019967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3081007a0 0x7ff308100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 -- 172.21.15.49:0/505019967 >> 172.21.15.49:0/505019967 conn(0x7ff3080fab40 msgr2=0x7ff3080fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 -- 172.21.15.49:0/505019967 shutdown_connections 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.731+0000 7ff30d9f4700 1 -- 172.21.15.49:0/505019967 wait complete. 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 Processor -- start 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 -- start start 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3080ff5b0 0x7ff308196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3081007a0 0x7ff308196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff308101140 0x7ff30819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30819e1b0 con 0x7ff308101140 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30819e2f0 con 0x7ff3080ff5b0 2023-10-08T09:12:47.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff30d9f4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30819e480 con 0x7ff3081007a0 2023-10-08T09:12:47.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff308101140 0x7ff30819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:47.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff308101140 0x7ff30819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47658/0 (socket says 172.21.15.49:47658) 2023-10-08T09:12:47.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 -- 172.21.15.49:0/170751047 learned_addr learned my addr 172.21.15.49:0/170751047 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:47.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff306ffd700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3080ff5b0 0x7ff308196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3067fc700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3081007a0 0x7ff308196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 -- 172.21.15.49:0/170751047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3081007a0 msgr2=0x7ff308196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3081007a0 0x7ff308196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 -- 172.21.15.49:0/170751047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3080ff5b0 msgr2=0x7ff308196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3080ff5b0 0x7ff308196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff3077fe700 1 -- 172.21.15.49:0/170751047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff300009c40 con 0x7ff308101140 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.732+0000 7ff306ffd700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3080ff5b0 0x7ff308196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.733+0000 7ff3077fe700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff308101140 0x7ff30819dbe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c370 tx=0x7ff30000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:47.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.733+0000 7ff30c9f2700 1 -- 172.21.15.49:0/170751047 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300017070 con 0x7ff308101140 2023-10-08T09:12:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.733+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f80095f0 con 0x7ff308101140 2023-10-08T09:12:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.733+0000 7ff30c9f2700 1 -- 172.21.15.49:0/170751047 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff30000f040 con 0x7ff308101140 2023-10-08T09:12:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.733+0000 7ff30c9f2700 1 -- 172.21.15.49:0/170751047 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30000e050 con 0x7ff308101140 2023-10-08T09:12:47.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.733+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30819ea30 con 0x7ff308101140 2023-10-08T09:12:47.108 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:46 smithi132 ceph-mon[103389]: Deploying daemon mds.h on smithi132 2023-10-08T09:12:47.108 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:46 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6836/3605975147,v1:172.21.15.184:6837/3605975147] up:boot 2023-10-08T09:12:47.108 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:46 smithi132 ceph-mon[103389]: fsmap 7 up:standby 2023-10-08T09:12:47.108 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:46 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:12:47.108 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:46 smithi184 ceph-mon[103675]: Deploying daemon mds.h on smithi132 2023-10-08T09:12:47.108 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:46 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6836/3605975147,v1:172.21.15.184:6837/3605975147] up:boot 2023-10-08T09:12:47.108 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:46 smithi184 ceph-mon[103675]: fsmap 7 up:standby 2023-10-08T09:12:47.109 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:46 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:12:47.109 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:12:46 smithi132 podman[139068]: 2023-10-08T09:12:47.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.734+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3080678a0 con 0x7ff308101140 2023-10-08T09:12:47.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.735+0000 7ff30c9f2700 1 -- 172.21.15.49:0/170751047 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff300004bd0 con 0x7ff308101140 2023-10-08T09:12:47.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.736+0000 7ff30c9f2700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2f00a1480 0x7ff2f00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:47.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.736+0000 7ff306ffd700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2f00a1480 0x7ff2f00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:47.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.736+0000 7ff30c9f2700 1 -- 172.21.15.49:0/170751047 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff3000cc950 con 0x7ff308101140 2023-10-08T09:12:47.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.736+0000 7ff306ffd700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2f00a1480 0x7ff2f00a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009990 tx=0x7ff2f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:47.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.739+0000 7ff30c9f2700 1 -- 172.21.15.49:0/170751047 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff300096450 con 0x7ff308101140 2023-10-08T09:12:47.111 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:46 smithi049 ceph-mon[99485]: Deploying daemon mds.h on smithi132 2023-10-08T09:12:47.111 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:46 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6836/3605975147,v1:172.21.15.184:6837/3605975147] up:boot 2023-10-08T09:12:47.111 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:46 smithi049 ceph-mon[99485]: fsmap 7 up:standby 2023-10-08T09:12:47.111 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:46 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:12:47.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.985+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff30810ae30 con 0x7ff308101140 2023-10-08T09:12:47.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.985+0000 7ff30c9f2700 1 -- 172.21.15.49:0/170751047 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7ff300099a70 con 0x7ff308101140 2023-10-08T09:12:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:47.112 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-08T09:12:47.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.987+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2f00a1480 msgr2=0x7ff2f00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:47.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.987+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2f00a1480 0x7ff2f00a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009990 tx=0x7ff2f801a040 comp rx=0 tx=0).stop 2023-10-08T09:12:47.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff308101140 msgr2=0x7ff30819dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:47.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff308101140 0x7ff30819dbe0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff30000c370 tx=0x7ff30000c730 comp rx=0 tx=0).stop 2023-10-08T09:12:47.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 shutdown_connections 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2f00a1480 0x7ff2f00a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff308101140 0x7ff30819dbe0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff3080ff5b0 0x7ff308196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 --2- 172.21.15.49:0/170751047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3081007a0 0x7ff308196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 >> 172.21.15.49:0/170751047 conn(0x7ff3080fab40 msgr2=0x7ff308068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 shutdown_connections 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:46.988+0000 7ff30d9f4700 1 -- 172.21.15.49:0/170751047 wait complete. 2023-10-08T09:12:47.114 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 8 2023-10-08T09:12:47.297 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mds[139108]: -- 172.21.15.132:0/88595850 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5610a8f5b1e0 con 0x5610a8f30c00 2023-10-08T09:12:47.297 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-h[139084]: starting mds.h at 2023-10-08T09:12:47.297 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:12:47 smithi132 bash[139068]: 3778bda780d4dd4844496a44fe3eb9605b4a5c25e87658334be5744edc6bc55f 2023-10-08T09:12:47.297 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:12:47 smithi132 systemd[1]: Started Ceph mds.h for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:48.010 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/170751047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:48.010 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.011 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.011 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.011 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:48.011 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:48.011 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:48.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:48.092 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/170751047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:48.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:48.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:48.093 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:48.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:48.319 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/170751047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:48.319 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.319 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.320 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:48.320 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:48.320 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:48.320 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:48.961 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:12:48 smithi184 systemd[1]: Starting Ceph mds.i for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:48.962 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:48 smithi184 ceph-mon[103675]: Deploying daemon mds.i on smithi184 2023-10-08T09:12:48.962 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:48 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] up:boot 2023-10-08T09:12:48.962 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:48 smithi184 ceph-mon[103675]: fsmap 8 up:standby 2023-10-08T09:12:48.962 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:48 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:12:49.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:48 smithi132 ceph-mon[103389]: Deploying daemon mds.i on smithi184 2023-10-08T09:12:49.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:48 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] up:boot 2023-10-08T09:12:49.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:48 smithi132 ceph-mon[103389]: fsmap 8 up:standby 2023-10-08T09:12:49.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:48 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:12:49.116 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:49.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:48 smithi049 ceph-mon[99485]: Deploying daemon mds.i on smithi184 2023-10-08T09:12:49.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:48 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] up:boot 2023-10-08T09:12:49.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:48 smithi049 ceph-mon[99485]: fsmap 8 up:standby 2023-10-08T09:12:49.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:48 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:12:49.234 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:12:48 smithi184 podman[141928]: 2023-10-08T09:12:49.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 -- 172.21.15.49:0/2048728903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c0ff5d0 msgr2=0x7f650c0ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 --2- 172.21.15.49:0/2048728903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c0ff5d0 0x7f650c0ff9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0099c0 tx=0x7f64fc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 -- 172.21.15.49:0/2048728903 shutdown_connections 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 --2- 172.21.15.49:0/2048728903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f650c101160 0x7f650c107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 --2- 172.21.15.49:0/2048728903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c0ff5d0 0x7f650c0ff9f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 --2- 172.21.15.49:0/2048728903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f650c1007c0 0x7f650c100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 -- 172.21.15.49:0/2048728903 >> 172.21.15.49:0/2048728903 conn(0x7f650c0fab40 msgr2=0x7f650c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 -- 172.21.15.49:0/2048728903 shutdown_connections 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 -- 172.21.15.49:0/2048728903 wait complete. 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 Processor -- start 2023-10-08T09:12:49.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.255+0000 7f6512433700 1 -- start start 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f6512433700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f650c0ff5d0 0x7f650c103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f6512433700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f650c1007c0 0x7f650c101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f6512433700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c101160 0x7f650c1020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f6512433700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c102880 con 0x7f650c101160 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f6512433700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c1029f0 con 0x7f650c1007c0 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f6512433700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c1a0660 con 0x7f650c0ff5d0 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c101160 0x7f650c1020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c101160 0x7f650c1020e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47672/0 (socket says 172.21.15.49:47672) 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 -- 172.21.15.49:0/2712524106 learned_addr learned my addr 172.21.15.49:0/2712524106 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f650bfff700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f650c0ff5d0 0x7f650c103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f650b7fe700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f650c1007c0 0x7f650c101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:49.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 -- 172.21.15.49:0/2712524106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f650c0ff5d0 msgr2=0x7f650c103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:49.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f650c0ff5d0 0x7f650c103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 -- 172.21.15.49:0/2712524106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f650c1007c0 msgr2=0x7f650c101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:49.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f650c1007c0 0x7f650c101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 -- 172.21.15.49:0/2712524106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0095f0 con 0x7f650c101160 2023-10-08T09:12:49.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.256+0000 7f65109d0700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c101160 0x7f650c1020e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6504007d70 tx=0x7f650400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:49.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.257+0000 7f65097fa700 1 -- 172.21.15.49:0/2712524106 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400ad40 con 0x7f650c101160 2023-10-08T09:12:49.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.257+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c1a08f0 con 0x7f650c101160 2023-10-08T09:12:49.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.257+0000 7f65097fa700 1 -- 172.21.15.49:0/2712524106 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6504007e60 con 0x7f650c101160 2023-10-08T09:12:49.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.257+0000 7f65097fa700 1 -- 172.21.15.49:0/2712524106 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504011620 con 0x7f650c101160 2023-10-08T09:12:49.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.257+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c1a0e00 con 0x7f650c101160 2023-10-08T09:12:49.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.258+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c0678a0 con 0x7f650c101160 2023-10-08T09:12:49.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.259+0000 7f65097fa700 1 -- 172.21.15.49:0/2712524106 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6504012970 con 0x7f650c101160 2023-10-08T09:12:49.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.259+0000 7f65097fa700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f64f80a1480 0x7f64f80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:49.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.259+0000 7f65097fa700 1 -- 172.21.15.49:0/2712524106 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f65040ccbb0 con 0x7f650c101160 2023-10-08T09:12:49.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.259+0000 7f650bfff700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f64f80a1480 0x7f64f80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:49.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.260+0000 7f650bfff700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f64f80a1480 0x7f64f80a3940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64fc003730 tx=0x7f64fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:49.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.264+0000 7f65097fa700 1 -- 172.21.15.49:0/2712524106 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f65040966b0 con 0x7f650c101160 2023-10-08T09:12:49.497 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mds[141969]: -- 172.21.15.184:0/458974841 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584be9cd1e0 con 0x5584be9a3000 2023-10-08T09:12:49.497 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:12:49 smithi184 bash[141928]: 47018045fe636bb2577cdd44753dc9aeb3c4be511df9b1b451c5eb2abb444f6d 2023-10-08T09:12:49.497 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-i[141944]: starting mds.i at 2023-10-08T09:12:49.497 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:12:49 smithi184 systemd[1]: Started Ceph mds.i for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:49.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.540+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f650c10afb0 con 0x7f650c101160 2023-10-08T09:12:49.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.541+0000 7f65097fa700 1 -- 172.21.15.49:0/2712524106 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6874 (secure 0 0 0) 0x7f6504099cd0 con 0x7f650c101160 2023-10-08T09:12:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:49.543 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24608,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-10-08T09:12:49.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f64f80a1480 msgr2=0x7f64f80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:49.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f64f80a1480 0x7f64f80a3940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64fc003730 tx=0x7f64fc000bc0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c101160 msgr2=0x7f650c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:49.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c101160 0x7f650c1020e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6504007d70 tx=0x7f650400a340 comp rx=0 tx=0).stop 2023-10-08T09:12:49.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 shutdown_connections 2023-10-08T09:12:49.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f64f80a1480 0x7f64f80a3940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f650c0ff5d0 0x7f650c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f650c101160 0x7f650c1020e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 --2- 172.21.15.49:0/2712524106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f650c1007c0 0x7f650c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:49.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 >> 172.21.15.49:0/2712524106 conn(0x7f650c0fab40 msgr2=0x7f650c0fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:49.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 shutdown_connections 2023-10-08T09:12:49.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:49.543+0000 7f6512433700 1 -- 172.21.15.49:0/2712524106 wait complete. 2023-10-08T09:12:49.547 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 9 2023-10-08T09:12:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:49.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:49 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2712524106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:49.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:49 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2712524106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:50.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:50.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:49 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2712524106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:50.994 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:50 smithi049 ceph-mon[99485]: Deploying daemon mds.g on smithi049 2023-10-08T09:12:51.023 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:50 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] up:boot 2023-10-08T09:12:51.023 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:50 smithi049 ceph-mon[99485]: fsmap 9 up:standby 2023-10-08T09:12:51.023 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:12:51.024 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:50 smithi184 ceph-mon[103675]: Deploying daemon mds.g on smithi049 2023-10-08T09:12:51.024 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:50 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] up:boot 2023-10-08T09:12:51.024 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:50 smithi184 ceph-mon[103675]: fsmap 9 up:standby 2023-10-08T09:12:51.024 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:12:51.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:50 smithi132 ceph-mon[103389]: Deploying daemon mds.g on smithi049 2023-10-08T09:12:51.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:50 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] up:boot 2023-10-08T09:12:51.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:50 smithi132 ceph-mon[103389]: fsmap 9 up:standby 2023-10-08T09:12:51.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:12:51.322 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:12:50 smithi049 systemd[1]: Starting Ceph mds.g for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:51.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:51.581 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:12:51 smithi049 podman[154759]: 2023-10-08T09:12:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.693+0000 7fdcd66bc700 1 -- 172.21.15.49:0/405999638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0103ad0 msgr2=0x7fdcd0105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.693+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/405999638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0103ad0 0x7fdcd0105ec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009990 tx=0x7fdcb8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 -- 172.21.15.49:0/405999638 shutdown_connections 2023-10-08T09:12:51.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/405999638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd01011a0 0x7fdcd0103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/405999638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd0106400 0x7fdcd0108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/405999638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd0103ad0 0x7fdcd0105ec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 -- 172.21.15.49:0/405999638 >> 172.21.15.49:0/405999638 conn(0x7fdcd00fab30 msgr2=0x7fdcd00fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:51.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 -- 172.21.15.49:0/405999638 shutdown_connections 2023-10-08T09:12:51.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 -- 172.21.15.49:0/405999638 wait complete. 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 Processor -- start 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.694+0000 7fdcd66bc700 1 -- start start 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcd66bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd01011a0 0x7fdcd0196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcd66bc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd0103ad0 0x7fdcd0196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcd66bc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd0106400 0x7fdcd019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcd66bc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019e180 con 0x7fdcd01011a0 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcd66bc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019e2c0 con 0x7fdcd0106400 2023-10-08T09:12:51.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcd66bc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019e450 con 0x7fdcd0103ad0 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd01011a0 0x7fdcd0196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd01011a0 0x7fdcd0196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47694/0 (socket says 172.21.15.49:47694) 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 -- 172.21.15.49:0/3706564585 learned_addr learned my addr 172.21.15.49:0/3706564585 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcd4c59700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd0106400 0x7fdcd019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcc7fff700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd0103ad0 0x7fdcd0196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 -- 172.21.15.49:0/3706564585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd0103ad0 msgr2=0x7fdcd0196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd0103ad0 0x7fdcd0196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 -- 172.21.15.49:0/3706564585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd0106400 msgr2=0x7fdcd019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd0106400 0x7fdcd019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 -- 172.21.15.49:0/3706564585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80095f0 con 0x7fdcd01011a0 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdcc7fff700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd0103ad0 0x7fdcd0196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:12:51.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccffff700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd01011a0 0x7fdcd0196650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000d980 tx=0x7fdcc000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:51.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.695+0000 7fdccdffb700 1 -- 172.21.15.49:0/3706564585 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc000bb60 con 0x7fdcd01011a0 2023-10-08T09:12:51.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.696+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd00feff0 con 0x7fdcd01011a0 2023-10-08T09:12:51.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.696+0000 7fdccdffb700 1 -- 172.21.15.49:0/3706564585 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcc0007640 con 0x7fdcd01011a0 2023-10-08T09:12:51.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.696+0000 7fdccdffb700 1 -- 172.21.15.49:0/3706564585 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0004e90 con 0x7fdcd01011a0 2023-10-08T09:12:51.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.696+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd00ff550 con 0x7fdcd01011a0 2023-10-08T09:12:51.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.697+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd00fc6b0 con 0x7fdcd01011a0 2023-10-08T09:12:51.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.698+0000 7fdccdffb700 1 -- 172.21.15.49:0/3706564585 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdcc000bcc0 con 0x7fdcd01011a0 2023-10-08T09:12:51.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.698+0000 7fdccdffb700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdcbc0a1480 0x7fdcbc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:51.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.699+0000 7fdcc7fff700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdcbc0a1480 0x7fdcbc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:51.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.699+0000 7fdccdffb700 1 -- 172.21.15.49:0/3706564585 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdcc00cc7a0 con 0x7fdcd01011a0 2023-10-08T09:12:51.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.699+0000 7fdcc7fff700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdcbc0a1480 0x7fdcbc0a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80037f0 tx=0x7fdcb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:51.704 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.703+0000 7fdccdffb700 1 -- 172.21.15.49:0/3706564585 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdcc00962a0 con 0x7fdcd01011a0 2023-10-08T09:12:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:51 smithi049 ceph-mon[99485]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:51.882 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:12:51 smithi049 ceph-mds[154799]: -- 172.21.15.49:0/3840644302 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ea12a71e0 con 0x561ea127cc00 2023-10-08T09:12:51.882 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:12:51 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g[154775]: starting mds.g at 2023-10-08T09:12:51.882 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:12:51 smithi049 bash[154759]: 9b0e6224674a66eaccf243691619fa3be4337f284c2cf19c0ffebb6400bf833c 2023-10-08T09:12:51.882 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:12:51 smithi049 systemd[1]: Started Ceph mds.g for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.958+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcd0066bc0 con 0x7fdcd01011a0 2023-10-08T09:12:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.959+0000 7fdccdffb700 1 -- 172.21.15.49:0/3706564585 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7fdcc00998c0 con 0x7fdcd01011a0 2023-10-08T09:12:51.960 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:51.960 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24608,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24616,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T09:12:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.961+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdcbc0a1480 msgr2=0x7fdcbc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.961+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdcbc0a1480 0x7fdcbc0a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80037f0 tx=0x7fdcb8005fb0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.961+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd01011a0 msgr2=0x7fdcd0196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.961+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd01011a0 0x7fdcd0196650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000d980 tx=0x7fdcc000dd40 comp rx=0 tx=0).stop 2023-10-08T09:12:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 shutdown_connections 2023-10-08T09:12:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdcbc0a1480 0x7fdcbc0a3940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcd0106400 0x7fdcd019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdcd01011a0 0x7fdcd0196650 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 --2- 172.21.15.49:0/3706564585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd0103ad0 0x7fdcd0196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 >> 172.21.15.49:0/3706564585 conn(0x7fdcd00fab30 msgr2=0x7fdcd0103030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 shutdown_connections 2023-10-08T09:12:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:51.962+0000 7fdcd66bc700 1 -- 172.21.15.49:0/3706564585 wait complete. 2023-10-08T09:12:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 10 2023-10-08T09:12:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:51 smithi184 ceph-mon[103675]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:51 smithi132 ceph-mon[103389]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:52.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: Deploying daemon mds.a on smithi049 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3706564585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.49:6838/3863085428,v1:172.21.15.49:6839/3863085428] up:boot 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: fsmap 10 up:standby 2023-10-08T09:12:52.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:12:53.040 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:53.040 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:53.040 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:53.040 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: Deploying daemon mds.a on smithi049 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3706564585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.49:6838/3863085428,v1:172.21.15.49:6839/3863085428] up:boot 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: fsmap 10 up:standby 2023-10-08T09:12:53.041 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: Deploying daemon mds.a on smithi049 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3706564585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:53.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.49:6838/3863085428,v1:172.21.15.49:6839/3863085428] up:boot 2023-10-08T09:12:53.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: fsmap 10 up:standby 2023-10-08T09:12:53.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:12:53.290 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:12:53 smithi049 systemd[1]: Starting Ceph mds.a for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:53.861 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:12:53 smithi049 podman[155152]: 2023-10-08T09:12:53.980 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 -- 172.21.15.49:0/3164900555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5400ff5d0 msgr2=0x7fc5400ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:54.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 --2- 172.21.15.49:0/3164900555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5400ff5d0 0x7fc5400ff9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc5300099c0 tx=0x7fc530009cd0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 -- 172.21.15.49:0/3164900555 shutdown_connections 2023-10-08T09:12:54.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 --2- 172.21.15.49:0/3164900555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc540101160 0x7fc540107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 --2- 172.21.15.49:0/3164900555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5400ff5d0 0x7fc5400ff9f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 --2- 172.21.15.49:0/3164900555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc5401007c0 0x7fc540100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 -- 172.21.15.49:0/3164900555 >> 172.21.15.49:0/3164900555 conn(0x7fc5400fab40 msgr2=0x7fc5400fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 -- 172.21.15.49:0/3164900555 shutdown_connections 2023-10-08T09:12:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.127+0000 7fc545e07700 1 -- 172.21.15.49:0/3164900555 wait complete. 2023-10-08T09:12:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 Processor -- start 2023-10-08T09:12:54.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 -- start start 2023-10-08T09:12:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5400ff5d0 0x7fc540196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5401007c0 0x7fc540196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc540101160 0x7fc54019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54019e210 con 0x7fc5401007c0 2023-10-08T09:12:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54019e350 con 0x7fc5400ff5d0 2023-10-08T09:12:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc545e07700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54019e4e0 con 0x7fc540101160 2023-10-08T09:12:54.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc537fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5401007c0 0x7fc540196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc537fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5401007c0 0x7fc540196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47724/0 (socket says 172.21.15.49:47724) 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.128+0000 7fc537fff700 1 -- 172.21.15.49:0/657251664 learned_addr learned my addr 172.21.15.49:0/657251664 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc53ffff700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc540101160 0x7fc54019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc53f7fe700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5400ff5d0 0x7fc540196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc537fff700 1 -- 172.21.15.49:0/657251664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc540101160 msgr2=0x7fc54019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc537fff700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc540101160 0x7fc54019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc537fff700 1 -- 172.21.15.49:0/657251664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5400ff5d0 msgr2=0x7fc540196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc537fff700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5400ff5d0 0x7fc540196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc537fff700 1 -- 172.21.15.49:0/657251664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5300095f0 con 0x7fc5401007c0 2023-10-08T09:12:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc53ffff700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc540101160 0x7fc54019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc53f7fe700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5400ff5d0 0x7fc540196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc537fff700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5401007c0 0x7fc540196c70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c9b0 tx=0x7fc52800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.129+0000 7fc53d7fa700 1 -- 172.21.15.49:0/657251664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528007b00 con 0x7fc5401007c0 2023-10-08T09:12:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.130+0000 7fc53d7fa700 1 -- 172.21.15.49:0/657251664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc528007c60 con 0x7fc5401007c0 2023-10-08T09:12:54.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.130+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54019e7c0 con 0x7fc5401007c0 2023-10-08T09:12:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.130+0000 7fc53d7fa700 1 -- 172.21.15.49:0/657251664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528018660 con 0x7fc5401007c0 2023-10-08T09:12:54.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.130+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54019ecd0 con 0x7fc5401007c0 2023-10-08T09:12:54.137 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:12:53 smithi049 ceph-mds[155192]: -- 172.21.15.49:0/3782636027 <== mon.0 v2:172.21.15.49:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557786fdb1e0 con 0x557786fb0c00 2023-10-08T09:12:54.138 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:12:53 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-a[155168]: starting mds.a at 2023-10-08T09:12:54.138 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:12:53 smithi049 bash[155152]: 4431e06f879ac3f601fb6d18325e0254a87cd3e9f945114b7a4acdf137a04f06 2023-10-08T09:12:54.138 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:12:54 smithi049 systemd[1]: Started Ceph mds.a for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:54.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.131+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc540066bc0 con 0x7fc5401007c0 2023-10-08T09:12:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.132+0000 7fc53d7fa700 1 -- 172.21.15.49:0/657251664 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc528018880 con 0x7fc5401007c0 2023-10-08T09:12:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.132+0000 7fc53d7fa700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc52c0a1480 0x7fc52c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.133+0000 7fc53f7fe700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc52c0a1480 0x7fc52c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.133+0000 7fc53d7fa700 1 -- 172.21.15.49:0/657251664 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc5280ccba0 con 0x7fc5401007c0 2023-10-08T09:12:54.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.133+0000 7fc53f7fe700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc52c0a1480 0x7fc52c0a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc530000c00 tx=0x7fc530005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:54.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.138+0000 7fc53d7fa700 1 -- 172.21.15.49:0/657251664 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc5280cce60 con 0x7fc5401007c0 2023-10-08T09:12:54.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.388+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc54010ae40 con 0x7fc5401007c0 2023-10-08T09:12:54.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.389+0000 7fc53d7fa700 1 -- 172.21.15.49:0/657251664 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7fc528096730 con 0x7fc5401007c0 2023-10-08T09:12:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:54.391 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14721,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24608,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24616,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T09:12:54.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc52c0a1480 msgr2=0x7fc52c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc52c0a1480 0x7fc52c0a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc530000c00 tx=0x7fc530005fb0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5401007c0 msgr2=0x7fc540196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5401007c0 0x7fc540196c70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc52800c9b0 tx=0x7fc52800cd70 comp rx=0 tx=0).stop 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 shutdown_connections 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc52c0a1480 0x7fc52c0a3940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc540101160 0x7fc54019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc5400ff5d0 0x7fc540196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 --2- 172.21.15.49:0/657251664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc5401007c0 0x7fc540196c70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:54.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 >> 172.21.15.49:0/657251664 conn(0x7fc5400fab40 msgr2=0x7fc5400fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:54.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 shutdown_connections 2023-10-08T09:12:54.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:54.391+0000 7fc545e07700 1 -- 172.21.15.49:0/657251664 wait complete. 2023-10-08T09:12:54.394 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 12 2023-10-08T09:12:55.121 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:55.121 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.49:6840/1275166284,v1:172.21.15.49:6841/1275166284] up:boot 2023-10-08T09:12:55.121 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: fsmap 11 up:standby 2023-10-08T09:12:55.121 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:12:55.121 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.121 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:55.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:55.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:55.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: Deploying daemon mds.b on smithi132 2023-10-08T09:12:55.122 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:54 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/657251664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:55.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:55.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.49:6840/1275166284,v1:172.21.15.49:6841/1275166284] up:boot 2023-10-08T09:12:55.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: fsmap 11 up:standby 2023-10-08T09:12:55.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: Deploying daemon mds.b on smithi132 2023-10-08T09:12:55.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:54 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/657251664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:55.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:55.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.49:6840/1275166284,v1:172.21.15.49:6841/1275166284] up:boot 2023-10-08T09:12:55.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: fsmap 11 up:standby 2023-10-08T09:12:55.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:12:55.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:55.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-08T09:12:55.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-08T09:12:55.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:12:55.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: Deploying daemon mds.b on smithi132 2023-10-08T09:12:55.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:54 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/657251664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:55.798 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:12:55 smithi132 systemd[1]: Starting Ceph mds.b for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:12:56.211 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:12:55 smithi132 podman[139407]: 2023-10-08T09:12:56.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:56.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:12:56 smithi132 ceph-mds[139447]: -- 172.21.15.132:0/2724059160 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a022eb1e0 con 0x558a022c0800 2023-10-08T09:12:56.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:12:56 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: starting mds.b at 2023-10-08T09:12:56.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:12:56 smithi132 bash[139407]: 05752a862bf7b5bace93264169dd952e9b52f2efb237b3672ce53182e8ce0f7b 2023-10-08T09:12:56.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:12:56 smithi132 systemd[1]: Started Ceph mds.b for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:12:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 -- 172.21.15.49:0/3800052628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef4101180 msgr2=0x7f2ef4103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 --2- 172.21.15.49:0/3800052628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef4101180 0x7f2ef4103570 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009990 tx=0x7f2ee8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 -- 172.21.15.49:0/3800052628 shutdown_connections 2023-10-08T09:12:56.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 --2- 172.21.15.49:0/3800052628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ef41063e0 0x7f2ef41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 --2- 172.21.15.49:0/3800052628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef4101180 0x7f2ef4103570 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 --2- 172.21.15.49:0/3800052628 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ef4103ab0 0x7f2ef4105ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 -- 172.21.15.49:0/3800052628 >> 172.21.15.49:0/3800052628 conn(0x7f2ef40fab10 msgr2=0x7f2ef40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:56.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 -- 172.21.15.49:0/3800052628 shutdown_connections 2023-10-08T09:12:56.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 -- 172.21.15.49:0/3800052628 wait complete. 2023-10-08T09:12:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 Processor -- start 2023-10-08T09:12:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.547+0000 7f2efb0db700 1 -- start start 2023-10-08T09:12:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2efb0db700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ef4101180 0x7f2ef4100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2efb0db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ef4103ab0 0x7f2ef40ff130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2efb0db700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef41063e0 0x7f2ef40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2efb0db700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40ffe10 con 0x7f2ef41063e0 2023-10-08T09:12:56.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2efb0db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40fff80 con 0x7f2ef4103ab0 2023-10-08T09:12:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2efb0db700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41a06f0 con 0x7f2ef4101180 2023-10-08T09:12:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef41063e0 0x7f2ef40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef41063e0 0x7f2ef40ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47740/0 (socket says 172.21.15.49:47740) 2023-10-08T09:12:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 -- 172.21.15.49:0/8467338 learned_addr learned my addr 172.21.15.49:0/8467338 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:56.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef3fff700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ef4103ab0 0x7f2ef40ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef8e77700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ef4101180 0x7f2ef4100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 -- 172.21.15.49:0/8467338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ef4101180 msgr2=0x7f2ef4100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ef4101180 0x7f2ef4100aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 -- 172.21.15.49:0/8467338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ef4103ab0 msgr2=0x7f2ef40ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ef4103ab0 0x7f2ef40ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 -- 172.21.15.49:0/8467338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80095f0 con 0x7f2ef41063e0 2023-10-08T09:12:56.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef8e77700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ef4101180 0x7f2ef4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:56.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef9678700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef41063e0 0x7f2ef40ff670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b970 tx=0x7f2eec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:56.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef1ffb700 1 -- 172.21.15.49:0/8467338 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00d8e0 con 0x7f2ef41063e0 2023-10-08T09:12:56.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef1ffb700 1 -- 172.21.15.49:0/8467338 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eec0098b0 con 0x7f2ef41063e0 2023-10-08T09:12:56.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef41a0980 con 0x7f2ef41063e0 2023-10-08T09:12:56.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.548+0000 7f2ef1ffb700 1 -- 172.21.15.49:0/8467338 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00c400 con 0x7f2ef41063e0 2023-10-08T09:12:56.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.549+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef41a0f10 con 0x7f2ef41063e0 2023-10-08T09:12:56.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.549+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef40fc690 con 0x7f2ef41063e0 2023-10-08T09:12:56.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.550+0000 7f2ef1ffb700 1 -- 172.21.15.49:0/8467338 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2eec00c5a0 con 0x7f2ef41063e0 2023-10-08T09:12:56.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.551+0000 7f2ef1ffb700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ee40a1480 0x7f2ee40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:56.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.551+0000 7f2ef8e77700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ee40a1480 0x7f2ee40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:56.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.551+0000 7f2ef1ffb700 1 -- 172.21.15.49:0/8467338 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2eec011070 con 0x7f2ef41063e0 2023-10-08T09:12:56.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.552+0000 7f2ef8e77700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ee40a1480 0x7f2ee40a3940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800b2e0 tx=0x7f2ee8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:56.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.556+0000 7f2ef1ffb700 1 -- 172.21.15.49:0/8467338 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2eec04f100 con 0x7f2ef41063e0 2023-10-08T09:12:56.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.816+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ef4067450 con 0x7f2ef41063e0 2023-10-08T09:12:56.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.817+0000 7f2ef1ffb700 1 -- 172.21.15.49:0/8467338 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f2eec099840 con 0x7f2ef41063e0 2023-10-08T09:12:56.818 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:56.818 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14721,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24608,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24616,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-08T09:12:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ee40a1480 msgr2=0x7f2ee40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:56.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ee40a1480 0x7f2ee40a3940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800b2e0 tx=0x7f2ee8005fb0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef41063e0 msgr2=0x7f2ef40ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef41063e0 0x7f2ef40ff670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b970 tx=0x7f2eec00bc80 comp rx=0 tx=0).stop 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 shutdown_connections 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ee40a1480 0x7f2ee40a3940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ef4101180 0x7f2ef4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ef41063e0 0x7f2ef40ff670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 --2- 172.21.15.49:0/8467338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ef4103ab0 0x7f2ef40ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 >> 172.21.15.49:0/8467338 conn(0x7f2ef40fab10 msgr2=0x7f2ef4101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 shutdown_connections 2023-10-08T09:12:56.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:56.819+0000 7f2efb0db700 1 -- 172.21.15.49:0/8467338 wait complete. 2023-10-08T09:12:56.822 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 12 2023-10-08T09:12:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:57.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/8467338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:57.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] up:boot 2023-10-08T09:12:57.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: fsmap 12 up:standby 2023-10-08T09:12:57.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:57 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:12:57.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:57.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/8467338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] up:boot 2023-10-08T09:12:57.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: fsmap 12 up:standby 2023-10-08T09:12:57.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:57 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:12:57.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:57.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:57.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:12:57.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/8467338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:57.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] up:boot 2023-10-08T09:12:57.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: fsmap 12 up:standby 2023-10-08T09:12:57.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:57 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:12:58.835 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:12:58.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 -- 172.21.15.49:0/1961320762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 msgr2=0x7fb7fc103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:58.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 --2- 172.21.15.49:0/1961320762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc103540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009990 tx=0x7fb7ec009ca0 comp rx=0 tx=0).stop 2023-10-08T09:12:58.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 -- 172.21.15.49:0/1961320762 shutdown_connections 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 --2- 172.21.15.49:0/1961320762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7fc1063b0 0x7fb7fc108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 --2- 172.21.15.49:0/1961320762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc103540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 --2- 172.21.15.49:0/1961320762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7fc103a80 0x7fb7fc105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 -- 172.21.15.49:0/1961320762 >> 172.21.15.49:0/1961320762 conn(0x7fb7fc0fab40 msgr2=0x7fb7fc0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 -- 172.21.15.49:0/1961320762 shutdown_connections 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.964+0000 7fb800d63700 1 -- 172.21.15.49:0/1961320762 wait complete. 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 Processor -- start 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 -- start start 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7fc103a80 0x7fb7fc198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7fc1063b0 0x7fb7fc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:58.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1a0330 con 0x7fb7fc101150 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1a0470 con 0x7fb7fc1063b0 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb800d63700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1a0600 con 0x7fb7fc103a80 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb7fa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb7fa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47750/0 (socket says 172.21.15.49:47750) 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb7fa59c700 1 -- 172.21.15.49:0/1691374106 learned_addr learned my addr 172.21.15.49:0/1691374106 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb7f9d9b700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7fc103a80 0x7fb7fc198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.965+0000 7fb7fad9d700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7fc1063b0 0x7fb7fc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7fa59c700 1 -- 172.21.15.49:0/1691374106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7fc103a80 msgr2=0x7fb7fc198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7fa59c700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7fc103a80 0x7fb7fc198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:58.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7fa59c700 1 -- 172.21.15.49:0/1691374106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7fc1063b0 msgr2=0x7fb7fc19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7fa59c700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7fc1063b0 0x7fb7fc19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7fa59c700 1 -- 172.21.15.49:0/1691374106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec0095f0 con 0x7fb7fc101150 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7fa59c700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc198800 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00b680 tx=0x7fb7ec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7eb7fe700 1 -- 172.21.15.49:0/1691374106 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec01d070 con 0x7fb7fc101150 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7eb7fe700 1 -- 172.21.15.49:0/1691374106 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7ec022470 con 0x7fb7fc101150 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc1a0880 con 0x7fb7fc101150 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb7eb7fe700 1 -- 172.21.15.49:0/1691374106 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec017650 con 0x7fb7fc101150 2023-10-08T09:12:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.966+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc1a0d00 con 0x7fb7fc101150 2023-10-08T09:12:58.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.967+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc066bc0 con 0x7fb7fc101150 2023-10-08T09:12:58.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.968+0000 7fb7eb7fe700 1 -- 172.21.15.49:0/1691374106 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb7ec0225e0 con 0x7fb7fc101150 2023-10-08T09:12:58.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.969+0000 7fb7eb7fe700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7e40a1500 0x7fb7e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:58.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.970+0000 7fb7f9d9b700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7e40a1500 0x7fb7e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:58.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.970+0000 7fb7eb7fe700 1 -- 172.21.15.49:0/1691374106 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7ec0ce000 con 0x7fb7fc101150 2023-10-08T09:12:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.970+0000 7fb7f9d9b700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7e40a1500 0x7fb7e40a39c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009c60 tx=0x7fb7f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:58.973+0000 7fb7eb7fe700 1 -- 172.21.15.49:0/1691374106 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb7ec097b90 con 0x7fb7fc101150 2023-10-08T09:12:59.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.242+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7fc0ffe60 con 0x7fb7fc101150 2023-10-08T09:12:59.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.243+0000 7fb7eb7fe700 1 -- 172.21.15.49:0/1691374106 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10049 (secure 0 0 0) 0x7fb7ec09b1b0 con 0x7fb7fc101150 2023-10-08T09:12:59.244 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:59.244 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14721,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24608,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24616,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-08T09:12:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7e40a1500 msgr2=0x7fb7e40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7e40a1500 0x7fb7e40a39c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009c60 tx=0x7fb7f0009260 comp rx=0 tx=0).stop 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 msgr2=0x7fb7fc198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc198800 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00b680 tx=0x7fb7ec009580 comp rx=0 tx=0).stop 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 shutdown_connections 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7e40a1500 0x7fb7e40a39c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7fc1063b0 0x7fb7fc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7fc101150 0x7fb7fc198800 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 --2- 172.21.15.49:0/1691374106 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7fc103a80 0x7fb7fc198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 >> 172.21.15.49:0/1691374106 conn(0x7fb7fc0fab40 msgr2=0x7fb7fc106810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:59.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 shutdown_connections 2023-10-08T09:12:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.245+0000 7fb800d63700 1 -- 172.21.15.49:0/1691374106 wait complete. 2023-10-08T09:12:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 13 2023-10-08T09:12:59.259 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:12:59.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 -- 172.21.15.49:0/892982271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 msgr2=0x7ff5e01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/892982271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e01079e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800b0d0 tx=0x7ff5d800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 -- 172.21.15.49:0/892982271 shutdown_connections 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/892982271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e01079e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/892982271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e00ff5a0 0x7ff5e00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/892982271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0100790 0x7ff5e0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 -- 172.21.15.49:0/892982271 >> 172.21.15.49:0/892982271 conn(0x7ff5e00fab10 msgr2=0x7ff5e00fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 -- 172.21.15.49:0/892982271 shutdown_connections 2023-10-08T09:12:59.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.396+0000 7ff5e7e33700 1 -- 172.21.15.49:0/892982271 wait complete. 2023-10-08T09:12:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 Processor -- start 2023-10-08T09:12:59.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 -- start start 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e00ff5a0 0x7ff5e01955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0100790 0x7ff5e0195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e019d120 con 0x7ff5e0101130 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e019d260 con 0x7ff5e00ff5a0 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e7e33700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e019d3f0 con 0x7ff5e0100790 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47768/0 (socket says 172.21.15.49:47768) 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e5bcf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e00ff5a0 0x7ff5e01955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 -- 172.21.15.49:0/701349931 learned_addr learned my addr 172.21.15.49:0/701349931 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:59.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e53ce700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0100790 0x7ff5e0195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 -- 172.21.15.49:0/701349931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0100790 msgr2=0x7ff5e0195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0100790 0x7ff5e0195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 -- 172.21.15.49:0/701349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e00ff5a0 msgr2=0x7ff5e01955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e00ff5a0 0x7ff5e01955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.397+0000 7ff5e63d0700 1 -- 172.21.15.49:0/701349931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d8009d00 con 0x7ff5e0101130 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5e5bcf700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e00ff5a0 0x7ff5e01955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5e53ce700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0100790 0x7ff5e0195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:12:59.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5e63d0700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e019cb50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8007bd0 tx=0x7ff5d8009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5d6ffd700 1 -- 172.21.15.49:0/701349931 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d800e040 con 0x7ff5e0101130 2023-10-08T09:12:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5d6ffd700 1 -- 172.21.15.49:0/701349931 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5d801dd50 con 0x7ff5e0101130 2023-10-08T09:12:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e019d670 con 0x7ff5e0101130 2023-10-08T09:12:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5d6ffd700 1 -- 172.21.15.49:0/701349931 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80123f0 con 0x7ff5e0101130 2023-10-08T09:12:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.398+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e019db40 con 0x7ff5e0101130 2023-10-08T09:12:59.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.399+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e0106950 con 0x7ff5e0101130 2023-10-08T09:12:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.400+0000 7ff5d6ffd700 1 -- 172.21.15.49:0/701349931 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff5d801d890 con 0x7ff5e0101130 2023-10-08T09:12:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.401+0000 7ff5d6ffd700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff5cc0a1500 0x7ff5cc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.401+0000 7ff5d6ffd700 1 -- 172.21.15.49:0/701349931 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff5d80ce480 con 0x7ff5e0101130 2023-10-08T09:12:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.401+0000 7ff5e5bcf700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff5cc0a1500 0x7ff5cc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.401+0000 7ff5e5bcf700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff5cc0a1500 0x7ff5cc0a39c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff5e01005f0 tx=0x7ff5dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:59.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.406+0000 7ff5d6ffd700 1 -- 172.21.15.49:0/701349931 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff5d8098010 con 0x7ff5e0101130 2023-10-08T09:12:59.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.613+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5e010adc0 con 0x7ff5e0101130 2023-10-08T09:12:59.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.614+0000 7ff5d6ffd700 1 -- 172.21.15.49:0/701349931 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18431 (secure 0 0 0) 0x7ff5d809b630 con 0x7ff5e0101130 2023-10-08T09:12:59.616 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:12:59.616 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":77,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:12:21.320137+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff5cc0a1500 msgr2=0x7ff5cc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff5cc0a1500 0x7ff5cc0a39c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff5e01005f0 tx=0x7ff5dc00b410 comp rx=0 tx=0).stop 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 msgr2=0x7ff5e019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e019cb50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8007bd0 tx=0x7ff5d8009440 comp rx=0 tx=0).stop 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 shutdown_connections 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff5cc0a1500 0x7ff5cc0a39c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5e0101130 0x7ff5e019cb50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5e00ff5a0 0x7ff5e01955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 --2- 172.21.15.49:0/701349931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5e0100790 0x7ff5e0195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 >> 172.21.15.49:0/701349931 conn(0x7ff5e00fab10 msgr2=0x7ff5e0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 shutdown_connections 2023-10-08T09:12:59.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.617+0000 7ff5e7e33700 1 -- 172.21.15.49:0/701349931 wait complete. 2023-10-08T09:12:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:59 smithi049 ceph-mon[99485]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:59 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1691374106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.630 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:12:59 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T09:12:59.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:12:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:59 smithi184 ceph-mon[103675]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:59 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1691374106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:12:59 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.761+0000 7f0e242ea700 1 -- 172.21.15.49:0/3232854217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 msgr2=0x7f0e1c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.761+0000 7f0e242ea700 1 --2- 172.21.15.49:0/3232854217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c108c00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400b0d0 tx=0x7f0e1400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 -- 172.21.15.49:0/3232854217 shutdown_connections 2023-10-08T09:12:59.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 --2- 172.21.15.49:0/3232854217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1007c0 0x7f0e1c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 --2- 172.21.15.49:0/3232854217 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c108c00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 --2- 172.21.15.49:0/3232854217 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e1c1019b0 0x7f0e1c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 -- 172.21.15.49:0/3232854217 >> 172.21.15.49:0/3232854217 conn(0x7f0e1c0fbd30 msgr2=0x7f0e1c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 -- 172.21.15.49:0/3232854217 shutdown_connections 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 -- 172.21.15.49:0/3232854217 wait complete. 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 Processor -- start 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.762+0000 7f0e242ea700 1 -- start start 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e242ea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e1c1007c0 0x7f0e1c074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e242ea700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1019b0 0x7f0e1c072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e242ea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e242ea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0738c0 con 0x7f0e1c102350 2023-10-08T09:12:59.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e242ea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c073a30 con 0x7f0e1c1007c0 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e242ea700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1a0890 con 0x7f0e1c1019b0 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1019b0 0x7f0e1c072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e22887700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1019b0 0x7f0e1c072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:34814/0 (socket says 172.21.15.49:34814) 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 -- 172.21.15.49:0/4249957973 learned_addr learned my addr 172.21.15.49:0/4249957973 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e22887700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47782/0 (socket says 172.21.15.49:47782) 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e22086700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e1c1007c0 0x7f0e1c074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 -- 172.21.15.49:0/4249957973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e1c1007c0 msgr2=0x7f0e1c074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e1c1007c0 0x7f0e1c074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 -- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 msgr2=0x7f0e1c073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 -- 172.21.15.49:0/4249957973 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14009d00 con 0x7f0e1c1019b0 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e22086700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e1c1007c0 0x7f0e1c074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e22887700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e21885700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1019b0 0x7f0e1c072be0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00e9a0 tx=0x7f0e0c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.763+0000 7f0e137fe700 1 -- 172.21.15.49:0/4249957973 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c009790 con 0x7f0e1c1019b0 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.764+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c1a0ac0 con 0x7f0e1c1019b0 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.764+0000 7f0e137fe700 1 -- 172.21.15.49:0/4249957973 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e0c00cbd0 con 0x7f0e1c1019b0 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.764+0000 7f0e137fe700 1 -- 172.21.15.49:0/4249957973 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c018680 con 0x7f0e1c1019b0 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.764+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c1a0f70 con 0x7f0e1c1019b0 2023-10-08T09:12:59.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.764+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c107b70 con 0x7f0e1c1019b0 2023-10-08T09:12:59.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.766+0000 7f0e137fe700 1 -- 172.21.15.49:0/4249957973 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0e0c00cd40 con 0x7f0e1c1019b0 2023-10-08T09:12:59.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.766+0000 7f0e137fe700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0e080a1480 0x7f0e080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:12:59.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.766+0000 7f0e22086700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0e080a1480 0x7f0e080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:12:59.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.766+0000 7f0e137fe700 1 -- 172.21.15.49:0/4249957973 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0e0c014070 con 0x7f0e1c1019b0 2023-10-08T09:12:59.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.767+0000 7f0e22086700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0e080a1480 0x7f0e080a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c101810 tx=0x7f0e18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:12:59.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.770+0000 7f0e137fe700 1 -- 172.21.15.49:0/4249957973 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0e0c052100 con 0x7f0e1c1019b0 2023-10-08T09:12:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:59 smithi132 ceph-mon[103389]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:12:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:59 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1691374106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:12:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:12:59 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:12:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.995+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e1c10c230 con 0x7f0e1c1019b0 2023-10-08T09:12:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.996+0000 7f0e137fe700 1 -- 172.21.15.49:0/4249957973 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e0c096660 con 0x7f0e1c1019b0 2023-10-08T09:12:59.997 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:13:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0e080a1480 msgr2=0x7f0e080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0e080a1480 0x7f0e080a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c101810 tx=0x7f0e18009260 comp rx=0 tx=0).stop 2023-10-08T09:13:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1019b0 msgr2=0x7f0e1c072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1019b0 0x7f0e1c072be0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00e9a0 tx=0x7f0e0c00ecb0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 shutdown_connections 2023-10-08T09:13:00.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0e080a1480 0x7f0e080a3940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0e1c102350 0x7f0e1c073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0e1c1007c0 0x7f0e1c074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 --2- 172.21.15.49:0/4249957973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0e1c1019b0 0x7f0e1c072be0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 >> 172.21.15.49:0/4249957973 conn(0x7f0e1c0fbd30 msgr2=0x7f0e1c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 shutdown_connections 2023-10-08T09:13:00.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:12:59.998+0000 7f0e242ea700 1 -- 172.21.15.49:0/4249957973 wait complete. 2023-10-08T09:13:00.012 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T09:13:00.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/1829901474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 msgr2=0x7f6ff40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/1829901474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 0x7f6ff40ff9c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009920 tx=0x7f6ff0009c30 comp rx=0 tx=0).stop 2023-10-08T09:13:00.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/1829901474 shutdown_connections 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/1829901474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 0x7f6ff41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/1829901474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 0x7f6ff40ff9c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/1829901474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ff4100790 0x7f6ff4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/1829901474 >> 172.21.15.49:0/1829901474 conn(0x7f6ff40fab10 msgr2=0x7f6ff40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/1829901474 shutdown_connections 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.140+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/1829901474 wait complete. 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 Processor -- start 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 -- start start 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 0x7f6ff4103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ff4100790 0x7f6ff4101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 0x7f6ff4102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4102ae0 con 0x7f6ff4101130 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4102c50 con 0x7f6ff40ff5a0 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ffb5d4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff41a0720 con 0x7f6ff4100790 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 0x7f6ff4102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 0x7f6ff4102340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47796/0 (socket says 172.21.15.49:47796) 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 -- 172.21.15.49:0/3394516834 learned_addr learned my addr 172.21.15.49:0/3394516834 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9370700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 0x7f6ff4103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff8b6f700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ff4100790 0x7f6ff4101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 -- 172.21.15.49:0/3394516834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ff4100790 msgr2=0x7f6ff4101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ff4100790 0x7f6ff4101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 -- 172.21.15.49:0/3394516834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 msgr2=0x7f6ff4103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 0x7f6ff4103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.141+0000 7f6ff9b71700 1 -- 172.21.15.49:0/3394516834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009520 con 0x7f6ff4101130 2023-10-08T09:13:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.142+0000 7f6ff9b71700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 0x7f6ff4102340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c110 tx=0x7f6fec00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.142+0000 7f6fea7fc700 1 -- 172.21.15.49:0/3394516834 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec017070 con 0x7f6ff4101130 2023-10-08T09:13:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.142+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff41a0950 con 0x7f6ff4101130 2023-10-08T09:13:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.142+0000 7f6fea7fc700 1 -- 172.21.15.49:0/3394516834 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fec00e050 con 0x7f6ff4101130 2023-10-08T09:13:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.142+0000 7f6fea7fc700 1 -- 172.21.15.49:0/3394516834 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec007840 con 0x7f6ff4101130 2023-10-08T09:13:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.142+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff41a0e10 con 0x7f6ff4101130 2023-10-08T09:13:00.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.143+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff4067450 con 0x7f6ff4101130 2023-10-08T09:13:00.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.144+0000 7f6fea7fc700 1 -- 172.21.15.49:0/3394516834 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6fec007a60 con 0x7f6ff4101130 2023-10-08T09:13:00.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.145+0000 7f6fea7fc700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6fe00a1480 0x7f6fe00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.145+0000 7f6fea7fc700 1 -- 172.21.15.49:0/3394516834 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6fec0ccd00 con 0x7f6ff4101130 2023-10-08T09:13:00.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.145+0000 7f6ff9370700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6fe00a1480 0x7f6fe00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.145+0000 7f6ff9370700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6fe00a1480 0x7f6fe00a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00098f0 tx=0x7f6ff0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:00.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.149+0000 7f6fea7fc700 1 -- 172.21.15.49:0/3394516834 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6fec09a290 con 0x7f6ff4101130 2023-10-08T09:13:00.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.361+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6ff410af80 con 0x7f6ff4101130 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.361+0000 7f6fea7fc700 1 -- 172.21.15.49:0/3394516834 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6fec099e20 con 0x7f6ff4101130 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:13:00.363 INFO:teuthology.orchestra.run.smithi049.stdout:] 2023-10-08T09:13:00.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6fe00a1480 msgr2=0x7f6fe00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6fe00a1480 0x7f6fe00a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00098f0 tx=0x7f6ff0019040 comp rx=0 tx=0).stop 2023-10-08T09:13:00.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 msgr2=0x7f6ff4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 0x7f6ff4102340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c110 tx=0x7f6fec00c420 comp rx=0 tx=0).stop 2023-10-08T09:13:00.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 shutdown_connections 2023-10-08T09:13:00.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6fe00a1480 0x7f6fe00a3940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ff4101130 0x7f6ff4102340 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.363+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff40ff5a0 0x7f6ff4103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.364+0000 7f6ffb5d4700 1 --2- 172.21.15.49:0/3394516834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ff4100790 0x7f6ff4101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.364+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 >> 172.21.15.49:0/3394516834 conn(0x7f6ff40fab10 msgr2=0x7f6ff4067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:00.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.364+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 shutdown_connections 2023-10-08T09:13:00.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.364+0000 7f6ffb5d4700 1 -- 172.21.15.49:0/3394516834 wait complete. 2023-10-08T09:13:00.377 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T09:13:00.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 -- 172.21.15.49:0/69934097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 msgr2=0x7f8cb0100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 --2- 172.21.15.49:0/69934097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb0100be0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0099c0 tx=0x7f8cac009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 -- 172.21.15.49:0/69934097 shutdown_connections 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 --2- 172.21.15.49:0/69934097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb0102350 0x7f8cb0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 --2- 172.21.15.49:0/69934097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb0100be0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 --2- 172.21.15.49:0/69934097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cb01019b0 0x7f8cb0101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 -- 172.21.15.49:0/69934097 >> 172.21.15.49:0/69934097 conn(0x7f8cb00fbd30 msgr2=0x7f8cb00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 -- 172.21.15.49:0/69934097 shutdown_connections 2023-10-08T09:13:00.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.511+0000 7f8cb846e700 1 -- 172.21.15.49:0/69934097 wait complete. 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 Processor -- start 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 -- start start 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb01019b0 0x7f8cb0196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cb0102350 0x7f8cb019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019e210 con 0x7f8cb01007c0 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019e350 con 0x7f8cb0102350 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb846e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb019e4e0 con 0x7f8cb01019b0 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb01966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47806/0 (socket says 172.21.15.49:47806) 2023-10-08T09:13:00.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 -- 172.21.15.49:0/2578834416 learned_addr learned my addr 172.21.15.49:0/2578834416 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb5a09700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb01019b0 0x7f8cb0196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb6a0b700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cb0102350 0x7f8cb019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 -- 172.21.15.49:0/2578834416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb01019b0 msgr2=0x7f8cb0196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb01019b0 0x7f8cb0196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 -- 172.21.15.49:0/2578834416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cb0102350 msgr2=0x7f8cb019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cb0102350 0x7f8cb019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 -- 172.21.15.49:0/2578834416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0095f0 con 0x7f8cb01007c0 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8cb620a700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb01966e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009fd0 tx=0x7f8cac005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.512+0000 7f8ca77fe700 1 -- 172.21.15.49:0/2578834416 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac01d070 con 0x7f8cb01007c0 2023-10-08T09:13:00.517 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.513+0000 7f8ca77fe700 1 -- 172.21.15.49:0/2578834416 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cac0059a0 con 0x7f8cb01007c0 2023-10-08T09:13:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.513+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb019e760 con 0x7f8cb01007c0 2023-10-08T09:13:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.513+0000 7f8ca77fe700 1 -- 172.21.15.49:0/2578834416 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac0173f0 con 0x7f8cb01007c0 2023-10-08T09:13:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.513+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb019ec60 con 0x7f8cb01007c0 2023-10-08T09:13:00.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.513+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb0107bf0 con 0x7f8cb01007c0 2023-10-08T09:13:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.515+0000 7f8ca77fe700 1 -- 172.21.15.49:0/2578834416 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8cac022600 con 0x7f8cb01007c0 2023-10-08T09:13:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.515+0000 7f8ca77fe700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c9c0a1500 0x7f8c9c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.515+0000 7f8cb5a09700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c9c0a1500 0x7f8c9c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.515+0000 7f8ca77fe700 1 -- 172.21.15.49:0/2578834416 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8cac0ce0d0 con 0x7f8cb01007c0 2023-10-08T09:13:00.519 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.516+0000 7f8cb5a09700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c9c0a1500 0x7f8c9c0a39c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000ac50 tx=0x7f8ca000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:00.520 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.519+0000 7f8ca77fe700 1 -- 172.21.15.49:0/2578834416 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8cac097c60 con 0x7f8cb01007c0 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/701349931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4249957973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:00.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:00 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:13:00.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:00.726+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8cb0066ca0 con 0x7f8cb01007c0 2023-10-08T09:13:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/701349931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:13:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:13:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4249957973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:00.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:00 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/701349931' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4249957973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:00 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:13:01.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.350+0000 7f8ca77fe700 1 -- 172.21.15.49:0/2578834416 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8cac09b280 con 0x7f8cb01007c0 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c9c0a1500 msgr2=0x7f8c9c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c9c0a1500 0x7f8c9c0a39c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000ac50 tx=0x7f8ca000a380 comp rx=0 tx=0).stop 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 msgr2=0x7f8cb01966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb01966e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009fd0 tx=0x7f8cac005f10 comp rx=0 tx=0).stop 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 shutdown_connections 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c9c0a1500 0x7f8c9c0a39c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cb0102350 0x7f8cb019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8cb01007c0 0x7f8cb01966e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 --2- 172.21.15.49:0/2578834416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8cb01019b0 0x7f8cb0196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 >> 172.21.15.49:0/2578834416 conn(0x7f8cb00fbd30 msgr2=0x7f8cb0104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:01.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 shutdown_connections 2023-10-08T09:13:01.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.353+0000 7f8cb846e700 1 -- 172.21.15.49:0/2578834416 wait complete. 2023-10-08T09:13:01.360 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2023-10-08T09:13:01.370 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-08T09:13:01.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 -- 172.21.15.49:0/1268672849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 msgr2=0x7f3944100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 --2- 172.21.15.49:0/1268672849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 0x7f3944100d20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39380098f0 tx=0x7f3938009c00 comp rx=0 tx=0).stop 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 -- 172.21.15.49:0/1268672849 shutdown_connections 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 --2- 172.21.15.49:0/1268672849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3944101260 0x7f3944107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 --2- 172.21.15.49:0/1268672849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 0x7f39440ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 --2- 172.21.15.49:0/1268672849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 0x7f3944100d20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 -- 172.21.15.49:0/1268672849 >> 172.21.15.49:0/1268672849 conn(0x7f39440fac40 msgr2=0x7f39440fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.507+0000 7f394a8c6700 1 -- 172.21.15.49:0/1268672849 shutdown_connections 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.508+0000 7f394a8c6700 1 -- 172.21.15.49:0/1268672849 wait complete. 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.508+0000 7f394a8c6700 1 Processor -- start 2023-10-08T09:13:01.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.508+0000 7f394a8c6700 1 -- start start 2023-10-08T09:13:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.508+0000 7f394a8c6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 0x7f39441967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.508+0000 7f394a8c6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 0x7f3944196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.508+0000 7f394a8c6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3944101260 0x7f394419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:01.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f394a8c6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394419e2f0 con 0x7f3944101260 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f394a8c6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394419e430 con 0x7f39440ff6d0 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f394a8c6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394419e5c0 con 0x7f39441008c0 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3948e63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3944101260 0x7f394419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 0x7f39441967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f39437fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 0x7f3944196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 0x7f39441967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:37434/0 (socket says 172.21.15.49:37434) 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3948e63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3944101260 0x7f394419dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47824/0 (socket says 172.21.15.49:47824) 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 -- 172.21.15.49:0/2684915543 learned_addr learned my addr 172.21.15.49:0/2684915543 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 -- 172.21.15.49:0/2684915543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 msgr2=0x7f3944196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 0x7f3944196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 -- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3944101260 msgr2=0x7f394419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3944101260 0x7f394419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 -- 172.21.15.49:0/2684915543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c009d00 con 0x7f39440ff6d0 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f39437fe700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 0x7f3944196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f3943fff700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 0x7f39441967c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f393400e970 tx=0x7f393400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f39417fa700 1 -- 172.21.15.49:0/2684915543 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934009790 con 0x7f39440ff6d0 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3938009520 con 0x7f39440ff6d0 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f39417fa700 1 -- 172.21.15.49:0/2684915543 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f393400eec0 con 0x7f39440ff6d0 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.509+0000 7f39417fa700 1 -- 172.21.15.49:0/2684915543 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934003cf0 con 0x7f39440ff6d0 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.510+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394419ec20 con 0x7f39440ff6d0 2023-10-08T09:13:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.510+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3944066ca0 con 0x7f39440ff6d0 2023-10-08T09:13:01.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.512+0000 7f39417fa700 1 -- 172.21.15.49:0/2684915543 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f393400ccd0 con 0x7f39440ff6d0 2023-10-08T09:13:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.513+0000 7f39417fa700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f392c0a1480 0x7f392c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.513+0000 7f39437fe700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f392c0a1480 0x7f392c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.513+0000 7f39417fa700 1 -- 172.21.15.49:0/2684915543 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f3934014070 con 0x7f39440ff6d0 2023-10-08T09:13:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.513+0000 7f39437fe700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f392c0a1480 0x7f392c0a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f393800fa70 tx=0x7f3938019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.515+0000 7f39417fa700 1 -- 172.21.15.49:0/2684915543 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f39340969b0 con 0x7f39440ff6d0 2023-10-08T09:13:01.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:01 smithi049 ceph-mon[99485]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:01.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:01 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3394516834' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T09:13:01.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:01 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2578834416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T09:13:01.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:01 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:01.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:01.718+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f394410aee0 con 0x7f39440ff6d0 2023-10-08T09:13:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:01 smithi184 ceph-mon[103675]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:01 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3394516834' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T09:13:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:01 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2578834416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T09:13:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:01 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:01 smithi132 ceph-mon[103389]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:01 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3394516834' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-08T09:13:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:01 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2578834416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-08T09:13:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:01 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.366+0000 7f39417fa700 1 -- 172.21.15.49:0/2684915543 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) v1 ==== 171+0+0 (secure 0 0 0) 0x7f3934099fd0 con 0x7f39440ff6d0 2023-10-08T09:13:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.377+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f392c0a1480 msgr2=0x7f392c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.377+0000 7f394a8c6700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f392c0a1480 0x7f392c0a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f393800fa70 tx=0x7f3938019040 comp rx=0 tx=0).stop 2023-10-08T09:13:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.377+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 msgr2=0x7f39441967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.377+0000 7f394a8c6700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 0x7f39441967c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f393400e970 tx=0x7f393400ec80 comp rx=0 tx=0).stop 2023-10-08T09:13:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 shutdown_connections 2023-10-08T09:13:02.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f392c0a1480 0x7f392c0a3940 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3944101260 0x7f394419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39440ff6d0 0x7f39441967c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 --2- 172.21.15.49:0/2684915543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f39441008c0 0x7f3944196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 >> 172.21.15.49:0/2684915543 conn(0x7f39440fac40 msgr2=0x7f3944068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 shutdown_connections 2023-10-08T09:13:02.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.378+0000 7f394a8c6700 1 -- 172.21.15.49:0/2684915543 wait complete. 2023-10-08T09:13:02.381 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2023-10-08T09:13:02.394 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T09:13:02.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 -- 172.21.15.49:0/774388290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e700ff960 msgr2=0x7f4e700ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 --2- 172.21.15.49:0/774388290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e700ff960 0x7f4e700ffdc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009990 tx=0x7f4e64009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 -- 172.21.15.49:0/774388290 shutdown_connections 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 --2- 172.21.15.49:0/774388290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e70100300 0x7f4e70107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 --2- 172.21.15.49:0/774388290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e70100c30 0x7f4e70101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 --2- 172.21.15.49:0/774388290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e700ff960 0x7f4e700ffdc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 -- 172.21.15.49:0/774388290 >> 172.21.15.49:0/774388290 conn(0x7f4e700fa390 msgr2=0x7f4e700fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 -- 172.21.15.49:0/774388290 shutdown_connections 2023-10-08T09:13:02.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.527+0000 7f4e782ed700 1 -- 172.21.15.49:0/774388290 wait complete. 2023-10-08T09:13:02.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 Processor -- start 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 -- start start 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e700ff960 0x7f4e70195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e70100300 0x7f4e70195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e70100c30 0x7f4e7019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7019d140 con 0x7f4e70100c30 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7019d280 con 0x7f4e70100300 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e782ed700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7019d410 con 0x7f4e700ff960 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e70100c30 0x7f4e7019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e70100c30 0x7f4e7019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47838/0 (socket says 172.21.15.49:47838) 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e76089700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e700ff960 0x7f4e70195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:02.532 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 -- 172.21.15.49:0/3990669258 learned_addr learned my addr 172.21.15.49:0/3990669258 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e75888700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e70100300 0x7f4e70195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 -- 172.21.15.49:0/3990669258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e700ff960 msgr2=0x7f4e70195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e700ff960 0x7f4e70195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 -- 172.21.15.49:0/3990669258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e70100300 msgr2=0x7f4e70195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e70100300 0x7f4e70195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e7688a700 1 -- 172.21.15.49:0/3990669258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640095f0 con 0x7f4e70100c30 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.528+0000 7f4e75888700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e70100300 0x7f4e70195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.529+0000 7f4e7688a700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e70100c30 0x7f4e7019cb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b970 tx=0x7f4e6800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.529+0000 7f4e637fe700 1 -- 172.21.15.49:0/3990669258 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800d860 con 0x7f4e70100c30 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.529+0000 7f4e637fe700 1 -- 172.21.15.49:0/3990669258 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e6800de80 con 0x7f4e70100c30 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.529+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7019d6f0 con 0x7f4e70100c30 2023-10-08T09:13:02.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.529+0000 7f4e637fe700 1 -- 172.21.15.49:0/3990669258 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68017400 con 0x7f4e70100c30 2023-10-08T09:13:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.529+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7019dbb0 con 0x7f4e70100c30 2023-10-08T09:13:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.530+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e70106bb0 con 0x7f4e70100c30 2023-10-08T09:13:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.532+0000 7f4e637fe700 1 -- 172.21.15.49:0/3990669258 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4e6800e630 con 0x7f4e70100c30 2023-10-08T09:13:02.534 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.533+0000 7f4e637fe700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e5c0a1480 0x7f4e5c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.533+0000 7f4e637fe700 1 -- 172.21.15.49:0/3990669258 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f4e68006e40 con 0x7f4e70100c30 2023-10-08T09:13:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.533+0000 7f4e76089700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e5c0a1480 0x7f4e5c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:02.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.533+0000 7f4e76089700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e5c0a1480 0x7f4e5c0a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e700675b0 tx=0x7f4e6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:02.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.538+0000 7f4e637fe700 1 -- 172.21.15.49:0/3990669258 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e68096150 con 0x7f4e70100c30 2023-10-08T09:13:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:02 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2578834416' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T09:13:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:02 smithi049 ceph-mon[99485]: osdmap e78: 12 total, 12 up, 12 in 2023-10-08T09:13:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:02 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2684915543' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:13:02.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:02 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:13:02.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:02 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2578834416' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T09:13:02.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:02 smithi184 ceph-mon[103675]: osdmap e78: 12 total, 12 up, 12 in 2023-10-08T09:13:02.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:02 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2684915543' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:13:02.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:02 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:13:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:02 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2578834416' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-08T09:13:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:02 smithi132 ceph-mon[103389]: osdmap e78: 12 total, 12 up, 12 in 2023-10-08T09:13:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:02 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2684915543' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:13:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:02 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-08T09:13:02.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:02.802+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4e70066ca0 con 0x7f4e70100c30 2023-10-08T09:13:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: pgmap v261: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T09:13:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T09:13:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:13:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3990669258' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:13:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.717+0000 7f4e637fe700 1 -- 172.21.15.49:0/3990669258 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4e68099770 con 0x7f4e70100c30 2023-10-08T09:13:03.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e5c0a1480 msgr2=0x7f4e5c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e5c0a1480 0x7f4e5c0a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4e700675b0 tx=0x7f4e6c009260 comp rx=0 tx=0).stop 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e70100c30 msgr2=0x7f4e7019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e70100c30 0x7f4e7019cb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b970 tx=0x7f4e6800bc80 comp rx=0 tx=0).stop 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 shutdown_connections 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e5c0a1480 0x7f4e5c0a3940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e700ff960 0x7f4e70195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e70100c30 0x7f4e7019cb70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 --2- 172.21.15.49:0/3990669258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e70100300 0x7f4e70195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 >> 172.21.15.49:0/3990669258 conn(0x7f4e700fa390 msgr2=0x7f4e70103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 shutdown_connections 2023-10-08T09:13:03.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.720+0000 7f4e782ed700 1 -- 172.21.15.49:0/3990669258 wait complete. 2023-10-08T09:13:03.723 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T09:13:03.740 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-08T09:13:03.740 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-08T09:13:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: pgmap v261: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T09:13:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T09:13:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:13:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3990669258' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:13:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: pgmap v261: 33 pgs: 2 creating+peering, 1 active+clean, 30 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-08T09:13:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: osdmap e79: 12 total, 12 up, 12 in 2023-10-08T09:13:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:13:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3990669258' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: osdmap e80: 12 total, 12 up, 12 in 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:13:03.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:13:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 -- 172.21.15.49:0/578152064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad941064c0 msgr2=0x7fad941088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 --2- 172.21.15.49:0/578152064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad941064c0 0x7fad941088f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00b0a0 tx=0x7fad8c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 -- 172.21.15.49:0/578152064 shutdown_connections 2023-10-08T09:13:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 --2- 172.21.15.49:0/578152064 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad94101260 0x7fad94103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 --2- 172.21.15.49:0/578152064 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad941064c0 0x7fad941088f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 --2- 172.21.15.49:0/578152064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad94103b90 0x7fad94105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 -- 172.21.15.49:0/578152064 >> 172.21.15.49:0/578152064 conn(0x7fad940fac10 msgr2=0x7fad940fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 -- 172.21.15.49:0/578152064 shutdown_connections 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.878+0000 7fad9ba90700 1 -- 172.21.15.49:0/578152064 wait complete. 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 Processor -- start 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 -- start start 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad94101260 0x7fad94100b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad94103b90 0x7fad940ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad941064c0 0x7fad940ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940ffef0 con 0x7fad94101260 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94100060 con 0x7fad941064c0 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9ba90700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad941a0890 con 0x7fad94103b90 2023-10-08T09:13:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9982c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad94101260 0x7fad94100b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9982c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad94101260 0x7fad94100b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47852/0 (socket says 172.21.15.49:47852) 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.879+0000 7fad9982c700 1 -- 172.21.15.49:0/3953169825 learned_addr learned my addr 172.21.15.49:0/3953169825 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9902b700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad94103b90 0x7fad940ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9a02d700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad941064c0 0x7fad940ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9982c700 1 -- 172.21.15.49:0/3953169825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad94103b90 msgr2=0x7fad940ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9982c700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad94103b90 0x7fad940ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9982c700 1 -- 172.21.15.49:0/3953169825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad941064c0 msgr2=0x7fad940ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9982c700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad941064c0 0x7fad940ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9982c700 1 -- 172.21.15.49:0/3953169825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c009d00 con 0x7fad94101260 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9982c700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad94101260 0x7fad94100b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad8400d980 tx=0x7fad8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad8affd700 1 -- 172.21.15.49:0/3953169825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84004220 con 0x7fad94101260 2023-10-08T09:13:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad8affd700 1 -- 172.21.15.49:0/3953169825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad84007640 con 0x7fad94101260 2023-10-08T09:13:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad941a0b20 con 0x7fad94101260 2023-10-08T09:13:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad8affd700 1 -- 172.21.15.49:0/3953169825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84005690 con 0x7fad94101260 2023-10-08T09:13:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.880+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad941a1030 con 0x7fad94101260 2023-10-08T09:13:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.881+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad940fc790 con 0x7fad94101260 2023-10-08T09:13:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.882+0000 7fad8affd700 1 -- 172.21.15.49:0/3953169825 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fad84004750 con 0x7fad94101260 2023-10-08T09:13:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.883+0000 7fad8affd700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad800a1480 0x7fad800a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.883+0000 7fad8affd700 1 -- 172.21.15.49:0/3953169825 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fad840cceb0 con 0x7fad94101260 2023-10-08T09:13:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.883+0000 7fad9902b700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad800a1480 0x7fad800a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.883+0000 7fad9902b700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad800a1480 0x7fad800a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad90007b60 tx=0x7fad900073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:03.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:03.887+0000 7fad8affd700 1 -- 172.21.15.49:0/3953169825 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fad840fa100 con 0x7fad94101260 2023-10-08T09:13:04.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.097+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fad94067500 con 0x7fad94101260 2023-10-08T09:13:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.829+0000 7fad8affd700 1 -- 172.21.15.49:0/3953169825 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v82) v1 ==== 157+0+0 (secure 0 0 0) 0x7fad8409a0f0 con 0x7fad94101260 2023-10-08T09:13:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad800a1480 msgr2=0x7fad800a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad800a1480 0x7fad800a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad90007b60 tx=0x7fad900073d0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad94101260 msgr2=0x7fad94100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad94101260 0x7fad94100b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad8400d980 tx=0x7fad8400dc90 comp rx=0 tx=0).stop 2023-10-08T09:13:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 shutdown_connections 2023-10-08T09:13:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad800a1480 0x7fad800a3940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad941064c0 0x7fad940ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fad94101260 0x7fad94100b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 --2- 172.21.15.49:0/3953169825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad94103b90 0x7fad940ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 >> 172.21.15.49:0/3953169825 conn(0x7fad940fac10 msgr2=0x7fad94101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 shutdown_connections 2023-10-08T09:13:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.833+0000 7fad9ba90700 1 -- 172.21.15.49:0/3953169825 wait complete. 2023-10-08T09:13:04.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T09:13:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 -- 172.21.15.49:0/2133046353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc1001e0 msgr2=0x7f55cc107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 --2- 172.21.15.49:0/2133046353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc1001e0 0x7f55cc107a80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f55c400b0a0 tx=0x7f55c400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 -- 172.21.15.49:0/2133046353 shutdown_connections 2023-10-08T09:13:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 --2- 172.21.15.49:0/2133046353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55cc100c60 0x7f55cc101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 --2- 172.21.15.49:0/2133046353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc1001e0 0x7f55cc107a80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 --2- 172.21.15.49:0/2133046353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc0ff820 0x7f55cc0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 -- 172.21.15.49:0/2133046353 >> 172.21.15.49:0/2133046353 conn(0x7f55cc0fa2d0 msgr2=0x7f55cc0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 -- 172.21.15.49:0/2133046353 shutdown_connections 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.989+0000 7f55d3824700 1 -- 172.21.15.49:0/2133046353 wait complete. 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 Processor -- start 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 -- start start 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc0ff820 0x7f55cc0ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55cc1001e0 0x7f55cc19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc100c60 0x7f55cc19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc19e290 con 0x7f55cc100c60 2023-10-08T09:13:04.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc19e400 con 0x7f55cc0ff820 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d3824700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc19e5c0 con 0x7f55cc1001e0 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc100c60 0x7f55cc19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc100c60 0x7f55cc19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47870/0 (socket says 172.21.15.49:47870) 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 -- 172.21.15.49:0/1082003283 learned_addr learned my addr 172.21.15.49:0/1082003283 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d0dbf700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55cc1001e0 0x7f55cc19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d15c0700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc0ff820 0x7f55cc0ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 -- 172.21.15.49:0/1082003283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55cc1001e0 msgr2=0x7f55cc19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55cc1001e0 0x7f55cc19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 -- 172.21.15.49:0/1082003283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc0ff820 msgr2=0x7f55cc0ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc0ff820 0x7f55cc0ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 -- 172.21.15.49:0/1082003283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4009d00 con 0x7f55cc100c60 2023-10-08T09:13:04.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d15c0700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc0ff820 0x7f55cc0ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d0dbf700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55cc1001e0 0x7f55cc19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.990+0000 7f55d1dc1700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc100c60 0x7f55cc19daa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f55c4004d00 tx=0x7f55c4004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.991+0000 7f55c27fc700 1 -- 172.21.15.49:0/1082003283 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c400e040 con 0x7f55cc100c60 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.991+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc19e7e0 con 0x7f55cc100c60 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.991+0000 7f55c27fc700 1 -- 172.21.15.49:0/1082003283 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c4009440 con 0x7f55cc100c60 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.991+0000 7f55c27fc700 1 -- 172.21.15.49:0/1082003283 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c401ba80 con 0x7f55cc100c60 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.991+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc19ed30 con 0x7f55cc100c60 2023-10-08T09:13:04.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.992+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc106a70 con 0x7f55cc100c60 2023-10-08T09:13:04.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.993+0000 7f55c27fc700 1 -- 172.21.15.49:0/1082003283 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f55c401bbe0 con 0x7f55cc100c60 2023-10-08T09:13:04.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.993+0000 7f55c27fc700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55b80a1470 0x7f55b80a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:04.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.993+0000 7f55c27fc700 1 -- 172.21.15.49:0/1082003283 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f55c40cf320 con 0x7f55cc100c60 2023-10-08T09:13:04.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.993+0000 7f55d15c0700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55b80a1470 0x7f55b80a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:04.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.994+0000 7f55d15c0700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55b80a1470 0x7f55b80a3930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:04.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:04 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3990669258' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T09:13:04.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:04 smithi184 ceph-mon[103675]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T09:13:04.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:04 smithi184 ceph-mon[103675]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T09:13:04.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:04 smithi184 ceph-mon[103675]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T09:13:04.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:04 smithi184 ceph-mon[103675]: pgmap v265: 97 pgs: 10 creating+peering, 22 active+clean, 65 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:04.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:04 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3953169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T09:13:04.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:04.997+0000 7f55c27fc700 1 -- 172.21.15.49:0/1082003283 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f55c4098a30 con 0x7f55cc100c60 2023-10-08T09:13:05.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:04 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3990669258' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T09:13:05.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:04 smithi132 ceph-mon[103389]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T09:13:05.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:04 smithi132 ceph-mon[103389]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T09:13:05.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:04 smithi132 ceph-mon[103389]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T09:13:05.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:04 smithi132 ceph-mon[103389]: pgmap v265: 97 pgs: 10 creating+peering, 22 active+clean, 65 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:05.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:04 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3953169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T09:13:05.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:04 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3990669258' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-08T09:13:05.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:04 smithi049 ceph-mon[99485]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-08T09:13:05.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:04 smithi049 ceph-mon[99485]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-08T09:13:05.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:04 smithi049 ceph-mon[99485]: osdmap e81: 12 total, 12 up, 12 in 2023-10-08T09:13:05.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:04 smithi049 ceph-mon[99485]: pgmap v265: 97 pgs: 10 creating+peering, 22 active+clean, 65 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:13:05.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:04 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3953169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-08T09:13:05.200 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:05.199+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f55cc10afe0 con 0x7f55cc100c60 2023-10-08T09:13:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] up:active 2023-10-08T09:13:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:13:05.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: Cluster is now healthy 2023-10-08T09:13:05.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3953169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T09:13:05.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T09:13:05.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:05 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1082003283' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] up:active 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: Cluster is now healthy 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3953169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T09:13:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:05 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1082003283' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] up:active 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: Cluster is now healthy 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3953169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: osdmap e82: 12 total, 12 up, 12 in 2023-10-08T09:13:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:05 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1082003283' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-08T09:13:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.838+0000 7f55c27fc700 1 -- 172.21.15.49:0/1082003283 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v84) v1 ==== 228+0+0 (secure 0 0 0) 0x7f55c409c050 con 0x7f55cc100c60 2023-10-08T09:13:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55b80a1470 msgr2=0x7f55b80a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55b80a1470 0x7f55b80a3930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc005dc0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc100c60 msgr2=0x7f55cc19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc100c60 0x7f55cc19daa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f55c4004d00 tx=0x7f55c4004de0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 shutdown_connections 2023-10-08T09:13:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f55b80a1470 0x7f55b80a3930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f55cc100c60 0x7f55cc19daa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f55cc0ff820 0x7f55cc0ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 --2- 172.21.15.49:0/1082003283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55cc1001e0 0x7f55cc19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 >> 172.21.15.49:0/1082003283 conn(0x7f55cc0fa2d0 msgr2=0x7f55cc103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:06.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 shutdown_connections 2023-10-08T09:13:06.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.841+0000 7f55d3824700 1 -- 172.21.15.49:0/1082003283 wait complete. 2023-10-08T09:13:06.846 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data_ec' created 2023-10-08T09:13:06.862 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-08T09:13:06.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.992+0000 7f7af0684700 1 -- 172.21.15.49:0/1507393001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae81008c0 msgr2=0x7f7ae8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:06.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.992+0000 7f7af0684700 1 --2- 172.21.15.49:0/1507393001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae81008c0 0x7f7ae8100d20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ad80099c0 tx=0x7f7ad8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- 172.21.15.49:0/1507393001 shutdown_connections 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 --2- 172.21.15.49:0/1507393001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ae80ff6d0 0x7f7ae80ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 --2- 172.21.15.49:0/1507393001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ae8101260 0x7f7ae8107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 --2- 172.21.15.49:0/1507393001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae81008c0 0x7f7ae8100d20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- 172.21.15.49:0/1507393001 >> 172.21.15.49:0/1507393001 conn(0x7f7ae80fac40 msgr2=0x7f7ae80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- 172.21.15.49:0/1507393001 shutdown_connections 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- 172.21.15.49:0/1507393001 wait complete. 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 Processor -- start 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- start start 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae80ff6d0 0x7f7ae8069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:06.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ae81008c0 0x7f7ae819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ae8101260 0x7f7ae819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae819e2c0 con 0x7f7ae80ff6d0 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae819e430 con 0x7f7ae81008c0 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.993+0000 7f7af0684700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae819e5f0 con 0x7f7ae8101260 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae80ff6d0 0x7f7ae8069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae80ff6d0 0x7f7ae8069810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47888/0 (socket says 172.21.15.49:47888) 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 -- 172.21.15.49:0/262762396 learned_addr learned my addr 172.21.15.49:0/262762396 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aeec21700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ae8101260 0x7f7ae819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aedc1f700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ae81008c0 0x7f7ae819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 -- 172.21.15.49:0/262762396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ae8101260 msgr2=0x7f7ae819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ae8101260 0x7f7ae819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 -- 172.21.15.49:0/262762396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ae81008c0 msgr2=0x7f7ae819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ae81008c0 0x7f7ae819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 -- 172.21.15.49:0/262762396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad80095f0 con 0x7f7ae80ff6d0 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7aee420700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae80ff6d0 0x7f7ae8069810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400e970 tx=0x7f7ae400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7adf7fe700 1 -- 172.21.15.49:0/262762396 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4009790 con 0x7f7ae80ff6d0 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae819e870 con 0x7f7ae80ff6d0 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7adf7fe700 1 -- 172.21.15.49:0/262762396 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae400c7e0 con 0x7f7ae80ff6d0 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7adf7fe700 1 -- 172.21.15.49:0/262762396 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003c70 con 0x7f7ae80ff6d0 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.994+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae819eda0 con 0x7f7ae80ff6d0 2023-10-08T09:13:06.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.995+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae8106af0 con 0x7f7ae80ff6d0 2023-10-08T09:13:07.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.997+0000 7f7adf7fe700 1 -- 172.21.15.49:0/262762396 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7ae400cc50 con 0x7f7ae80ff6d0 2023-10-08T09:13:07.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.998+0000 7f7adf7fe700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7ad40a1480 0x7f7ad40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:07.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.998+0000 7f7aedc1f700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7ad40a1480 0x7f7ad40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:07.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.998+0000 7f7adf7fe700 1 -- 172.21.15.49:0/262762396 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f7ae4014070 con 0x7f7ae80ff6d0 2023-10-08T09:13:07.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:06.998+0000 7f7aedc1f700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7ad40a1480 0x7f7ad40a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8003730 tx=0x7f7ad800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:07.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.002+0000 7f7adf7fe700 1 -- 172.21.15.49:0/262762396 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7ae4096660 con 0x7f7ae80ff6d0 2023-10-08T09:13:07.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:06 smithi049 ceph-mon[99485]: pgmap v268: 97 pgs: 10 creating+peering, 68 active+clean, 19 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 4 op/s 2023-10-08T09:13:07.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:06 smithi049 ceph-mon[99485]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T09:13:07.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.205+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f7ae8067980 con 0x7f7ae80ff6d0 2023-10-08T09:13:07.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:06 smithi184 ceph-mon[103675]: pgmap v268: 97 pgs: 10 creating+peering, 68 active+clean, 19 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 4 op/s 2023-10-08T09:13:07.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:06 smithi184 ceph-mon[103675]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T09:13:07.297 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:13:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: 2023-10-08T09:13:06.939+0000 7fb4a80a8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T09:13:07.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:06 smithi132 ceph-mon[103389]: pgmap v268: 97 pgs: 10 creating+peering, 68 active+clean, 19 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s wr, 4 op/s 2023-10-08T09:13:07.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:06 smithi132 ceph-mon[103389]: osdmap e83: 12 total, 12 up, 12 in 2023-10-08T09:13:07.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.840+0000 7f7adf7fe700 1 -- 172.21.15.49:0/262762396 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7ae4099c80 con 0x7f7ae80ff6d0 2023-10-08T09:13:07.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.844+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7ad40a1480 msgr2=0x7f7ad40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:07.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.844+0000 7f7af0684700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7ad40a1480 0x7f7ad40a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8003730 tx=0x7f7ad800b540 comp rx=0 tx=0).stop 2023-10-08T09:13:07.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.844+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae80ff6d0 msgr2=0x7f7ae8069810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:07.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.844+0000 7f7af0684700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae80ff6d0 0x7f7ae8069810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400e970 tx=0x7f7ae400ec80 comp rx=0 tx=0).stop 2023-10-08T09:13:07.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.845+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 shutdown_connections 2023-10-08T09:13:07.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.845+0000 7f7af0684700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7ad40a1480 0x7f7ad40a3940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:07.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.845+0000 7f7af0684700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ae8101260 0x7f7ae819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:07.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.845+0000 7f7af0684700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae80ff6d0 0x7f7ae8069810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:07.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.845+0000 7f7af0684700 1 --2- 172.21.15.49:0/262762396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ae81008c0 0x7f7ae819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:07.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.845+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 >> 172.21.15.49:0/262762396 conn(0x7f7ae80fac40 msgr2=0x7f7ae8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:07.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.845+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 shutdown_connections 2023-10-08T09:13:07.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:07.846+0000 7f7af0684700 1 -- 172.21.15.49:0/262762396 wait complete. 2023-10-08T09:13:07.866 INFO:teuthology.orchestra.run.smithi049.stderr:set pool 4 allow_ec_overwrites to true 2023-10-08T09:13:07.878 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 -- 172.21.15.49:0/1162860128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04981002a0 msgr2=0x7f0498107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 --2- 172.21.15.49:0/1162860128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04981002a0 0x7f0498107b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f049000b0a0 tx=0x7f049000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 -- 172.21.15.49:0/1162860128 shutdown_connections 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 --2- 172.21.15.49:0/1162860128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0498100d20 0x7f0498101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 --2- 172.21.15.49:0/1162860128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04981002a0 0x7f0498107b40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 --2- 172.21.15.49:0/1162860128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04980ff8e0 0x7f04980ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 -- 172.21.15.49:0/1162860128 >> 172.21.15.49:0/1162860128 conn(0x7f04980fa390 msgr2=0x7f04980fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 -- 172.21.15.49:0/1162860128 shutdown_connections 2023-10-08T09:13:08.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 -- 172.21.15.49:0/1162860128 wait complete. 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 Processor -- start 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.010+0000 7f04a066f700 1 -- start start 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f04a066f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04980ff8e0 0x7f04980ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f04a066f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04981002a0 0x7f049819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f04a066f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0498100d20 0x7f049819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f04a066f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049819e370 con 0x7f0498100d20 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f04a066f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049819e4e0 con 0x7f04980ff8e0 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f04a066f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049819e6a0 con 0x7f04981002a0 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0498100d20 0x7f049819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0498100d20 0x7f049819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47910/0 (socket says 172.21.15.49:47910) 2023-10-08T09:13:08.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 -- 172.21.15.49:0/2521117756 learned_addr learned my addr 172.21.15.49:0/2521117756 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049e40b700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04980ff8e0 0x7f04980ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049dc0a700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04981002a0 0x7f049819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 -- 172.21.15.49:0/2521117756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04981002a0 msgr2=0x7f049819b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04981002a0 0x7f049819b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 -- 172.21.15.49:0/2521117756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04980ff8e0 msgr2=0x7f04980ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04980ff8e0 0x7f04980ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 -- 172.21.15.49:0/2521117756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490009d00 con 0x7f0498100d20 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049dc0a700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04981002a0 0x7f049819b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f049ec0c700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0498100d20 0x7f049819db80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0490007a60 tx=0x7f049000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f048f7fe700 1 -- 172.21.15.49:0/2521117756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049000e040 con 0x7f0498100d20 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f048f7fe700 1 -- 172.21.15.49:0/2521117756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04900094c0 con 0x7f0498100d20 2023-10-08T09:13:08.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049819e8c0 con 0x7f0498100d20 2023-10-08T09:13:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.011+0000 7f048f7fe700 1 -- 172.21.15.49:0/2521117756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04900052a0 con 0x7f0498100d20 2023-10-08T09:13:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.012+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049819ed40 con 0x7f0498100d20 2023-10-08T09:13:08.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.012+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498106ab0 con 0x7f0498100d20 2023-10-08T09:13:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.013+0000 7f048f7fe700 1 -- 172.21.15.49:0/2521117756 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0490005400 con 0x7f0498100d20 2023-10-08T09:13:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.014+0000 7f048f7fe700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f04840a1480 0x7f04840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.014+0000 7f049e40b700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f04840a1480 0x7f04840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.014+0000 7f048f7fe700 1 -- 172.21.15.49:0/2521117756 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f04900ce850 con 0x7f0498100d20 2023-10-08T09:13:08.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.014+0000 7f049e40b700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f04840a1480 0x7f04840a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0494009c60 tx=0x7f0494009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:08.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.018+0000 7f048f7fe700 1 -- 172.21.15.49:0/2521117756 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0490097d70 con 0x7f0498100d20 2023-10-08T09:13:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:07 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1082003283' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T09:13:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:07 smithi049 ceph-mon[99485]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T09:13:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:07 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/262762396' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T09:13:08.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:07 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1082003283' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T09:13:08.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:07 smithi184 ceph-mon[103675]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T09:13:08.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:07 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/262762396' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T09:13:08.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.272+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f049810b000 con 0x7f0498100d20 2023-10-08T09:13:08.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:07 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1082003283' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-08T09:13:08.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:07 smithi132 ceph-mon[103389]: osdmap e84: 12 total, 12 up, 12 in 2023-10-08T09:13:08.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:07 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/262762396' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-08T09:13:08.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.852+0000 7f048f7fe700 1 -- 172.21.15.49:0/2521117756 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f049009b390 con 0x7f0498100d20 2023-10-08T09:13:08.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f04840a1480 msgr2=0x7f04840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f04840a1480 0x7f04840a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0494009c60 tx=0x7f0494009260 comp rx=0 tx=0).stop 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0498100d20 msgr2=0x7f049819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0498100d20 0x7f049819db80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0490007a60 tx=0x7f049000bce0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 shutdown_connections 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f04840a1480 0x7f04840a3940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0498100d20 0x7f049819db80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04980ff8e0 0x7f04980ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 --2- 172.21.15.49:0/2521117756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04981002a0 0x7f049819b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 >> 172.21.15.49:0/2521117756 conn(0x7f04980fa390 msgr2=0x7f0498103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:08.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 shutdown_connections 2023-10-08T09:13:08.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:08.856+0000 7f04a066f700 1 -- 172.21.15.49:0/2521117756 wait complete. 2023-10-08T09:13:08.859 INFO:teuthology.orchestra.run.smithi049.stderr:added data pool 4 to fsmap 2023-10-08T09:13:08.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:13:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.004+0000 7ff07b226700 1 -- 172.21.15.49:0/3655888701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0740ff5a0 msgr2=0x7ff0740ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.004+0000 7ff07b226700 1 --2- 172.21.15.49:0/3655888701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0740ff5a0 0x7ff0740ff9c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff0680099c0 tx=0x7ff068009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.004+0000 7ff07b226700 1 -- 172.21.15.49:0/3655888701 shutdown_connections 2023-10-08T09:13:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.004+0000 7ff07b226700 1 --2- 172.21.15.49:0/3655888701 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff074101130 0x7ff0741079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.004+0000 7ff07b226700 1 --2- 172.21.15.49:0/3655888701 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0740ff5a0 0x7ff0740ff9c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.004+0000 7ff07b226700 1 --2- 172.21.15.49:0/3655888701 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff074100790 0x7ff074100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.004+0000 7ff07b226700 1 -- 172.21.15.49:0/3655888701 >> 172.21.15.49:0/3655888701 conn(0x7ff0740fab10 msgr2=0x7ff0740fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 -- 172.21.15.49:0/3655888701 shutdown_connections 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 -- 172.21.15.49:0/3655888701 wait complete. 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 Processor -- start 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 -- start start 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0740ff5a0 0x7ff074196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff074100790 0x7ff074196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff074101130 0x7ff07419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07419e1c0 con 0x7ff074100790 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07419e300 con 0x7ff0740ff5a0 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff07b226700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07419e490 con 0x7ff074101130 2023-10-08T09:13:09.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff073fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff074100790 0x7ff074196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff073fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff074100790 0x7ff074196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47922/0 (socket says 172.21.15.49:47922) 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff073fff700 1 -- 172.21.15.49:0/1205748789 learned_addr learned my addr 172.21.15.49:0/1205748789 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.005+0000 7ff078fc2700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0740ff5a0 0x7ff074196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff0797c3700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff074101130 0x7ff07419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff073fff700 1 -- 172.21.15.49:0/1205748789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff074101130 msgr2=0x7ff07419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff073fff700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff074101130 0x7ff07419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff073fff700 1 -- 172.21.15.49:0/1205748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0740ff5a0 msgr2=0x7ff074196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff073fff700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0740ff5a0 0x7ff074196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff073fff700 1 -- 172.21.15.49:0/1205748789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0680095f0 con 0x7ff074100790 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff073fff700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff074100790 0x7ff074196bd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d950 tx=0x7ff06000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff071ffb700 1 -- 172.21.15.49:0/1205748789 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06000f890 con 0x7ff074100790 2023-10-08T09:13:09.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff071ffb700 1 -- 172.21.15.49:0/1205748789 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff060010460 con 0x7ff074100790 2023-10-08T09:13:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07419e770 con 0x7ff074100790 2023-10-08T09:13:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07419ec80 con 0x7ff074100790 2023-10-08T09:13:09.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.006+0000 7ff071ffb700 1 -- 172.21.15.49:0/1205748789 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06001b650 con 0x7ff074100790 2023-10-08T09:13:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.007+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff074067450 con 0x7ff074100790 2023-10-08T09:13:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.009+0000 7ff071ffb700 1 -- 172.21.15.49:0/1205748789 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff0600108d0 con 0x7ff074100790 2023-10-08T09:13:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.009+0000 7ff071ffb700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0640a1480 0x7ff0640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.009+0000 7ff071ffb700 1 -- 172.21.15.49:0/1205748789 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff0600cd130 con 0x7ff074100790 2023-10-08T09:13:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.009+0000 7ff078fc2700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0640a1480 0x7ff0640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.010+0000 7ff078fc2700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0640a1480 0x7ff0640a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff068003820 tx=0x7ff068005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:09.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.013+0000 7ff071ffb700 1 -- 172.21.15.49:0/1205748789 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff0600965a0 con 0x7ff074100790 2023-10-08T09:13:09.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:08 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/262762396' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T09:13:09.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:08 smithi049 ceph-mon[99485]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T09:13:09.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:08 smithi049 ceph-mon[99485]: pgmap v271: 129 pgs: 8 creating+peering, 97 active+clean, 24 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-10-08T09:13:09.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:08 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2521117756' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T09:13:09.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:08 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/262762396' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T09:13:09.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:08 smithi184 ceph-mon[103675]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T09:13:09.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:08 smithi184 ceph-mon[103675]: pgmap v271: 129 pgs: 8 creating+peering, 97 active+clean, 24 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-10-08T09:13:09.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:08 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2521117756' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T09:13:09.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.265+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff07410af30 con 0x7ff074100790 2023-10-08T09:13:09.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.266+0000 7ff071ffb700 1 -- 172.21.15.49:0/1205748789 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11267 (secure 0 0 0) 0x7ff060020090 con 0x7ff074100790 2023-10-08T09:13:09.268 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:09.268 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14721,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24608,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24616,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:08.274602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T09:13:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.268+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0640a1480 msgr2=0x7ff0640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.268+0000 7ff07b226700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0640a1480 0x7ff0640a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff068003820 tx=0x7ff068005ad0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.268+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff074100790 msgr2=0x7ff074196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.268+0000 7ff07b226700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff074100790 0x7ff074196bd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff06000d950 tx=0x7ff06000dc60 comp rx=0 tx=0).stop 2023-10-08T09:13:09.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 shutdown_connections 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0640a1480 0x7ff0640a3940 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff074101130 0x7ff07419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0740ff5a0 0x7ff074196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 --2- 172.21.15.49:0/1205748789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff074100790 0x7ff074196bd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 >> 172.21.15.49:0/1205748789 conn(0x7ff0740fab10 msgr2=0x7ff074067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 shutdown_connections 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.269+0000 7ff07b226700 1 -- 172.21.15.49:0/1205748789 wait complete. 2023-10-08T09:13:09.271 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2023-10-08T09:13:09.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:09.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:08 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/262762396' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-08T09:13:09.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:08 smithi132 ceph-mon[103389]: osdmap e85: 12 total, 12 up, 12 in 2023-10-08T09:13:09.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:08 smithi132 ceph-mon[103389]: pgmap v271: 129 pgs: 8 creating+peering, 97 active+clean, 24 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-10-08T09:13:09.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:08 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2521117756' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-08T09:13:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 -- 172.21.15.49:0/3760045706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 msgr2=0x7fa4941020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/3760045706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4941020b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0099c0 tx=0x7fa47c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 -- 172.21.15.49:0/3760045706 shutdown_connections 2023-10-08T09:13:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/3760045706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4940ff0a0 0x7fa494108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/3760045706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4941020b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/3760045706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4940fe700 0x7fa4940feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 -- 172.21.15.49:0/3760045706 >> 172.21.15.49:0/3760045706 conn(0x7fa4940fa2c0 msgr2=0x7fa4940fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 -- 172.21.15.49:0/3760045706 shutdown_connections 2023-10-08T09:13:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.414+0000 7fa49a8d5700 1 -- 172.21.15.49:0/3760045706 wait complete. 2023-10-08T09:13:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 Processor -- start 2023-10-08T09:13:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 -- start start 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4940fe700 0x7fa494101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4940ff0a0 0x7fa4940ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4940fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941006b0 con 0x7fa494101c90 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa494100820 con 0x7fa4940ff0a0 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa49a8d5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941a06d0 con 0x7fa4940fe700 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa498e72700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4940fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa498e72700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4940fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47938/0 (socket says 172.21.15.49:47938) 2023-10-08T09:13:09.418 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa498e72700 1 -- 172.21.15.49:0/2572772347 learned_addr learned my addr 172.21.15.49:0/2572772347 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa4937fe700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4940ff0a0 0x7fa4940ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa493fff700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4940fe700 0x7fa494101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.415+0000 7fa498e72700 1 -- 172.21.15.49:0/2572772347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4940fe700 msgr2=0x7fa494101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa498e72700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4940fe700 0x7fa494101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa498e72700 1 -- 172.21.15.49:0/2572772347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4940ff0a0 msgr2=0x7fa4940ff9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa498e72700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4940ff0a0 0x7fa4940ff9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa498e72700 1 -- 172.21.15.49:0/2572772347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c0095f0 con 0x7fa494101c90 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa493fff700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4940fe700 0x7fa494101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa498e72700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4940fff10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00ef30 tx=0x7fa48c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa4917fa700 1 -- 172.21.15.49:0/2572772347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c018070 con 0x7fa494101c90 2023-10-08T09:13:09.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa4917fa700 1 -- 172.21.15.49:0/2572772347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa48c00f040 con 0x7fa494101c90 2023-10-08T09:13:09.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4941a0900 con 0x7fa494101c90 2023-10-08T09:13:09.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa4917fa700 1 -- 172.21.15.49:0/2572772347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00ce00 con 0x7fa494101c90 2023-10-08T09:13:09.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.416+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4941a0dc0 con 0x7fa494101c90 2023-10-08T09:13:09.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.417+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa494107d00 con 0x7fa494101c90 2023-10-08T09:13:09.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.419+0000 7fa4917fa700 1 -- 172.21.15.49:0/2572772347 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa48c003a00 con 0x7fa494101c90 2023-10-08T09:13:09.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.419+0000 7fa4917fa700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4800a1480 0x7fa4800a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.419+0000 7fa4917fa700 1 -- 172.21.15.49:0/2572772347 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa48c0cd490 con 0x7fa494101c90 2023-10-08T09:13:09.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.419+0000 7fa493fff700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4800a1480 0x7fa4800a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.420+0000 7fa493fff700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4800a1480 0x7fa4800a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009990 tx=0x7fa47c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:09.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.423+0000 7fa4917fa700 1 -- 172.21.15.49:0/2572772347 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa48c096990 con 0x7fa494101c90 2023-10-08T09:13:09.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.625+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa494066bc0 con 0x7fa494101c90 2023-10-08T09:13:09.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.627+0000 7fa4917fa700 1 -- 172.21.15.49:0/2572772347 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7fa48c099fb0 con 0x7fa494101c90 2023-10-08T09:13:09.628 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:09.628 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4800a1480 msgr2=0x7fa4800a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4800a1480 0x7fa4800a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009990 tx=0x7fa47c01a040 comp rx=0 tx=0).stop 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 msgr2=0x7fa4940fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4940fff10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00ef30 tx=0x7fa48c00c450 comp rx=0 tx=0).stop 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 shutdown_connections 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4800a1480 0x7fa4800a3940 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4940fe700 0x7fa494101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa494101c90 0x7fa4940fff10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 --2- 172.21.15.49:0/2572772347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4940ff0a0 0x7fa4940ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 >> 172.21.15.49:0/2572772347 conn(0x7fa4940fa2c0 msgr2=0x7fa494104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:09.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 shutdown_connections 2023-10-08T09:13:09.631 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.629+0000 7fa49a8d5700 1 -- 172.21.15.49:0/2572772347 wait complete. 2023-10-08T09:13:09.643 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:09.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.768+0000 7fd43a207700 1 -- 172.21.15.49:0/4278314713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4341018b0 msgr2=0x7fd434101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.768+0000 7fd43a207700 1 --2- 172.21.15.49:0/4278314713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4341018b0 0x7fd434101d10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd4280099c0 tx=0x7fd428009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 -- 172.21.15.49:0/4278314713 shutdown_connections 2023-10-08T09:13:09.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 --2- 172.21.15.49:0/4278314713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4341006c0 0x7fd434100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 --2- 172.21.15.49:0/4278314713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd434102250 0x7fd434108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 --2- 172.21.15.49:0/4278314713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4341018b0 0x7fd434101d10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 -- 172.21.15.49:0/4278314713 >> 172.21.15.49:0/4278314713 conn(0x7fd4340fbc50 msgr2=0x7fd4340fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 -- 172.21.15.49:0/4278314713 shutdown_connections 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 -- 172.21.15.49:0/4278314713 wait complete. 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 Processor -- start 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 -- start start 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.769+0000 7fd43a207700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4341006c0 0x7fd434074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd43a207700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4341018b0 0x7fd434072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd43a207700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd434102250 0x7fd434073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd43a207700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340738c0 con 0x7fd434102250 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd43a207700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd434073a30 con 0x7fd4341006c0 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd43a207700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4341a0670 con 0x7fd4341018b0 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd434102250 0x7fd434073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd434102250 0x7fd434073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47952/0 (socket says 172.21.15.49:47952) 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 -- 172.21.15.49:0/168429197 learned_addr learned my addr 172.21.15.49:0/168429197 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd432ffd700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4341018b0 0x7fd434072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd4337fe700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4341006c0 0x7fd434074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 -- 172.21.15.49:0/168429197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4341018b0 msgr2=0x7fd434072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4341018b0 0x7fd434072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 -- 172.21.15.49:0/168429197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4341006c0 msgr2=0x7fd434074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4341006c0 0x7fd434074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 -- 172.21.15.49:0/168429197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4280095f0 con 0x7fd434102250 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd433fff700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd434102250 0x7fd434073120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007ee0 tx=0x7fd42c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd430ff9700 1 -- 172.21.15.49:0/168429197 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00dd70 con 0x7fd434102250 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd430ff9700 1 -- 172.21.15.49:0/168429197 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd42c00f040 con 0x7fd434102250 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4341a0900 con 0x7fd434102250 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd430ff9700 1 -- 172.21.15.49:0/168429197 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0048a0 con 0x7fd434102250 2023-10-08T09:13:09.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.770+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4341a0dc0 con 0x7fd434102250 2023-10-08T09:13:09.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.771+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd434107ae0 con 0x7fd434102250 2023-10-08T09:13:09.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.773+0000 7fd430ff9700 1 -- 172.21.15.49:0/168429197 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd42c003a00 con 0x7fd434102250 2023-10-08T09:13:09.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.773+0000 7fd430ff9700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd41c0a1480 0x7fd41c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:09.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.773+0000 7fd430ff9700 1 -- 172.21.15.49:0/168429197 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd42c0cd050 con 0x7fd434102250 2023-10-08T09:13:09.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.773+0000 7fd4337fe700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd41c0a1480 0x7fd41c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:09.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.774+0000 7fd4337fe700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd41c0a1480 0x7fd41c0a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd424005fd0 tx=0x7fd424005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:09.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.776+0000 7fd430ff9700 1 -- 172.21.15.49:0/168429197 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd42c096460 con 0x7fd434102250 2023-10-08T09:13:09.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.982+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd434066bc0 con 0x7fd434102250 2023-10-08T09:13:09.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.984+0000 7fd430ff9700 1 -- 172.21.15.49:0/168429197 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7fd42c099a80 con 0x7fd434102250 2023-10-08T09:13:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:09.985 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:09.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd41c0a1480 msgr2=0x7fd41c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd41c0a1480 0x7fd41c0a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd424005fd0 tx=0x7fd424005dd0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd434102250 msgr2=0x7fd434073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd434102250 0x7fd434073120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007ee0 tx=0x7fd42c00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 shutdown_connections 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd41c0a1480 0x7fd41c0a3940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd434102250 0x7fd434073120 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4341006c0 0x7fd434074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 --2- 172.21.15.49:0/168429197 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4341018b0 0x7fd434072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:09.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 >> 172.21.15.49:0/168429197 conn(0x7fd4340fbc50 msgr2=0x7fd434104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:09.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 shutdown_connections 2023-10-08T09:13:09.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:09.986+0000 7fd43a207700 1 -- 172.21.15.49:0/168429197 wait complete. 2023-10-08T09:13:09.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:09 smithi049 ceph-mon[99485]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:13:09.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:09 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2521117756' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T09:13:09.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:09 smithi049 ceph-mon[99485]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T09:13:09.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:09 smithi049 ceph-mon[99485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:09.998 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:09 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1205748789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:09.999 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:09 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2572772347' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:10.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:13:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 -- 172.21.15.49:0/2304299703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda001018a0 msgr2=0x7fda00101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 --2- 172.21.15.49:0/2304299703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda001018a0 0x7fda00101d00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80099c0 tx=0x7fd9e8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 -- 172.21.15.49:0/2304299703 shutdown_connections 2023-10-08T09:13:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 --2- 172.21.15.49:0/2304299703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda001006b0 0x7fda00100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 --2- 172.21.15.49:0/2304299703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda00102240 0x7fda00108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 --2- 172.21.15.49:0/2304299703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda001018a0 0x7fda00101d00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 -- 172.21.15.49:0/2304299703 >> 172.21.15.49:0/2304299703 conn(0x7fda000fbc20 msgr2=0x7fda000fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 -- 172.21.15.49:0/2304299703 shutdown_connections 2023-10-08T09:13:10.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.146+0000 7fda05878700 1 -- 172.21.15.49:0/2304299703 wait complete. 2023-10-08T09:13:10.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 Processor -- start 2023-10-08T09:13:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 -- start start 2023-10-08T09:13:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda001006b0 0x7fda00074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda001018a0 0x7fda00072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda00102240 0x7fda00073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda000738c0 con 0x7fda00102240 2023-10-08T09:13:10.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda00073a30 con 0x7fda001018a0 2023-10-08T09:13:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fda05878700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda001a06c0 con 0x7fda001006b0 2023-10-08T09:13:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fd9ff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda00102240 0x7fda00073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fd9ff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda00102240 0x7fda00073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47966/0 (socket says 172.21.15.49:47966) 2023-10-08T09:13:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fd9ff7fe700 1 -- 172.21.15.49:0/445765700 learned_addr learned my addr 172.21.15.49:0/445765700 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fd9fe7fc700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda001018a0 0x7fda00072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.147+0000 7fd9feffd700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda001006b0 0x7fda00074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fd9ff7fe700 1 -- 172.21.15.49:0/445765700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda001006b0 msgr2=0x7fda00074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fd9ff7fe700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda001006b0 0x7fda00074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fd9ff7fe700 1 -- 172.21.15.49:0/445765700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda001018a0 msgr2=0x7fda00072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fd9ff7fe700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda001018a0 0x7fda00072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fd9ff7fe700 1 -- 172.21.15.49:0/445765700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e80095f0 con 0x7fda00102240 2023-10-08T09:13:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fd9ff7fe700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda00102240 0x7fda00073120 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8007ee0 tx=0x7fd9f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fda04876700 1 -- 172.21.15.49:0/445765700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8017070 con 0x7fda00102240 2023-10-08T09:13:10.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fda04876700 1 -- 172.21.15.49:0/445765700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9f800f040 con 0x7fda00102240 2023-10-08T09:13:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda001a0950 con 0x7fda00102240 2023-10-08T09:13:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fda04876700 1 -- 172.21.15.49:0/445765700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8005740 con 0x7fda00102240 2023-10-08T09:13:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.148+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda001a0e60 con 0x7fda00102240 2023-10-08T09:13:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.149+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda00107ae0 con 0x7fda00102240 2023-10-08T09:13:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.150+0000 7fda04876700 1 -- 172.21.15.49:0/445765700 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd9f80058a0 con 0x7fda00102240 2023-10-08T09:13:10.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.150+0000 7fda04876700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9ec0a1480 0x7fd9ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.151+0000 7fd9feffd700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9ec0a1480 0x7fd9ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.151+0000 7fda04876700 1 -- 172.21.15.49:0/445765700 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd9f80ccd50 con 0x7fda00102240 2023-10-08T09:13:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.151+0000 7fd9feffd700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9ec0a1480 0x7fd9ec0a3940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0005950 tx=0x7fd9f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.154+0000 7fda04876700 1 -- 172.21.15.49:0/445765700 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9f80961c0 con 0x7fda00102240 2023-10-08T09:13:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:09 smithi184 ceph-mon[103675]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:13:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:09 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2521117756' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T09:13:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:09 smithi184 ceph-mon[103675]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T09:13:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:09 smithi184 ceph-mon[103675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:09 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1205748789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:10.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:09 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2572772347' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:09 smithi132 ceph-mon[103389]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-08T09:13:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:09 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2521117756' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-08T09:13:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:09 smithi132 ceph-mon[103389]: osdmap e86: 12 total, 12 up, 12 in 2023-10-08T09:13:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:09 smithi132 ceph-mon[103389]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:09 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1205748789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:09 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2572772347' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.408+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda000678a0 con 0x7fda00102240 2023-10-08T09:13:10.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.409+0000 7fda04876700 1 -- 172.21.15.49:0/445765700 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11267 (secure 0 0 0) 0x7fd9f80997e0 con 0x7fda00102240 2023-10-08T09:13:10.410 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:10.410 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14721,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24553,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24575,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24592,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24608,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24616,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:08.274602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24638},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T09:13:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9ec0a1480 msgr2=0x7fd9ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9ec0a1480 0x7fd9ec0a3940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0005950 tx=0x7fd9f00058e0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda00102240 msgr2=0x7fda00073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda00102240 0x7fda00073120 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8007ee0 tx=0x7fd9f800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 shutdown_connections 2023-10-08T09:13:10.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9ec0a1480 0x7fd9ec0a3940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda001006b0 0x7fda00074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda00102240 0x7fda00073120 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 --2- 172.21.15.49:0/445765700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda001018a0 0x7fda00072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.411+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 >> 172.21.15.49:0/445765700 conn(0x7fda000fbc20 msgr2=0x7fda00104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.412+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 shutdown_connections 2023-10-08T09:13:10.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.412+0000 7fda05878700 1 -- 172.21.15.49:0/445765700 wait complete. 2023-10-08T09:13:10.413 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 17 2023-10-08T09:13:10.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 -- 172.21.15.49:0/2417145650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 msgr2=0x7ffa680ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 --2- 172.21.15.49:0/2417145650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa680ff960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffa580099c0 tx=0x7ffa58009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 -- 172.21.15.49:0/2417145650 shutdown_connections 2023-10-08T09:13:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 --2- 172.21.15.49:0/2417145650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa681010d0 0x7ffa68107980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 --2- 172.21.15.49:0/2417145650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa680ff960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 --2- 172.21.15.49:0/2417145650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa68100730 0x7ffa68100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 -- 172.21.15.49:0/2417145650 >> 172.21.15.49:0/2417145650 conn(0x7ffa680faa90 msgr2=0x7ffa680fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 -- 172.21.15.49:0/2417145650 shutdown_connections 2023-10-08T09:13:10.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.551+0000 7ffa6e092700 1 -- 172.21.15.49:0/2417145650 wait complete. 2023-10-08T09:13:10.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 Processor -- start 2023-10-08T09:13:10.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 -- start start 2023-10-08T09:13:10.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa68068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa68100730 0x7ffa6819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa681010d0 0x7ffa6819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681a0350 con 0x7ffa680ff540 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681a04c0 con 0x7ffa68100730 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa6e092700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681a0680 con 0x7ffa681010d0 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa677fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa68068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa677fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa68068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47976/0 (socket says 172.21.15.49:47976) 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa677fe700 1 -- 172.21.15.49:0/320753258 learned_addr learned my addr 172.21.15.49:0/320753258 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa66ffd700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa68100730 0x7ffa6819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa677fe700 1 -- 172.21.15.49:0/320753258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa681010d0 msgr2=0x7ffa6819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa677fe700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa681010d0 0x7ffa6819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa677fe700 1 -- 172.21.15.49:0/320753258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa68100730 msgr2=0x7ffa6819d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.552+0000 7ffa677fe700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa68100730 0x7ffa6819d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa677fe700 1 -- 172.21.15.49:0/320753258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa580095f0 con 0x7ffa680ff540 2023-10-08T09:13:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa66ffd700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa68100730 0x7ffa6819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa677fe700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa68068350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006010 tx=0x7ffa58005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa64ff9700 1 -- 172.21.15.49:0/320753258 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5801d070 con 0x7ffa680ff540 2023-10-08T09:13:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa681a0900 con 0x7ffa680ff540 2023-10-08T09:13:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa64ff9700 1 -- 172.21.15.49:0/320753258 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa58004150 con 0x7ffa680ff540 2023-10-08T09:13:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa64ff9700 1 -- 172.21.15.49:0/320753258 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58021410 con 0x7ffa680ff540 2023-10-08T09:13:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.553+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa681a0d80 con 0x7ffa680ff540 2023-10-08T09:13:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.554+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa681069e0 con 0x7ffa680ff540 2023-10-08T09:13:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.555+0000 7ffa64ff9700 1 -- 172.21.15.49:0/320753258 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ffa5800f6f0 con 0x7ffa680ff540 2023-10-08T09:13:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.555+0000 7ffa64ff9700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffa540a1480 0x7ffa540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.556+0000 7ffa66ffd700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffa540a1480 0x7ffa540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.556+0000 7ffa64ff9700 1 -- 172.21.15.49:0/320753258 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffa580cded0 con 0x7ffa680ff540 2023-10-08T09:13:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.556+0000 7ffa66ffd700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffa540a1480 0x7ffa540a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa68068910 tx=0x7ffa5000b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.560+0000 7ffa64ff9700 1 -- 172.21.15.49:0/320753258 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffa58097370 con 0x7ffa680ff540 2023-10-08T09:13:10.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.765+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa680673e0 con 0x7ffa680ff540 2023-10-08T09:13:10.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.767+0000 7ffa64ff9700 1 -- 172.21.15.49:0/320753258 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7ffa5809a990 con 0x7ffa680ff540 2023-10-08T09:13:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:10.768 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffa540a1480 msgr2=0x7ffa540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffa540a1480 0x7ffa540a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa68068910 tx=0x7ffa5000b2c0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 msgr2=0x7ffa68068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa68068350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006010 tx=0x7ffa58005af0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 shutdown_connections 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffa540a1480 0x7ffa540a3940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa681010d0 0x7ffa6819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa680ff540 0x7ffa68068350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 --2- 172.21.15.49:0/320753258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa68100730 0x7ffa6819d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 >> 172.21.15.49:0/320753258 conn(0x7ffa680faa90 msgr2=0x7ffa68103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 shutdown_connections 2023-10-08T09:13:10.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.769+0000 7ffa6e092700 1 -- 172.21.15.49:0/320753258 wait complete. 2023-10-08T09:13:10.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T09:13:02.804952+0000', 'modified': '2023-10-08T09:13:08.274602+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24638}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/1629770277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 1629770277}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 1629770277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-08T09:13:10.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T09:13:10.783 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T09:13:10.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 -- 172.21.15.49:0/38494874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc98101230 msgr2=0x7efc98107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 --2- 172.21.15.49:0/38494874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc98101230 0x7efc98107ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efc9000b0d0 tx=0x7efc9000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 -- 172.21.15.49:0/38494874 shutdown_connections 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 --2- 172.21.15.49:0/38494874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc98101230 0x7efc98107ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 --2- 172.21.15.49:0/38494874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc980ff6a0 0x7efc980ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 --2- 172.21.15.49:0/38494874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc98100890 0x7efc98100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 -- 172.21.15.49:0/38494874 >> 172.21.15.49:0/38494874 conn(0x7efc980fac10 msgr2=0x7efc980fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 -- 172.21.15.49:0/38494874 shutdown_connections 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.914+0000 7efc9ecee700 1 -- 172.21.15.49:0/38494874 wait complete. 2023-10-08T09:13:10.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 Processor -- start 2023-10-08T09:13:10.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 -- start start 2023-10-08T09:13:10.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc980ff6a0 0x7efc98195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc98100890 0x7efc98195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc98101230 0x7efc9819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9819d270 con 0x7efc980ff6a0 2023-10-08T09:13:10.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9819d3b0 con 0x7efc98100890 2023-10-08T09:13:10.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ecee700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9819d540 con 0x7efc98101230 2023-10-08T09:13:10.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc980ff6a0 0x7efc98195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc980ff6a0 0x7efc98195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47990/0 (socket says 172.21.15.49:47990) 2023-10-08T09:13:10.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 -- 172.21.15.49:0/1609615137 learned_addr learned my addr 172.21.15.49:0/1609615137 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:10.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc97fff700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc98100890 0x7efc98195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9d28b700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc98101230 0x7efc9819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 -- 172.21.15.49:0/1609615137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc98101230 msgr2=0x7efc9819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc98101230 0x7efc9819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 -- 172.21.15.49:0/1609615137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc98100890 msgr2=0x7efc98195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc98100890 0x7efc98195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc9ca8a700 1 -- 172.21.15.49:0/1609615137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc90009d00 con 0x7efc980ff6a0 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.915+0000 7efc97fff700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc98100890 0x7efc98195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.916+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc980ff6a0 0x7efc98195740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00d980 tx=0x7efc8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.916+0000 7efc95ffb700 1 -- 172.21.15.49:0/1609615137 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c004020 con 0x7efc980ff6a0 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.916+0000 7efc95ffb700 1 -- 172.21.15.49:0/1609615137 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8c009730 con 0x7efc980ff6a0 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.916+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9819d820 con 0x7efc980ff6a0 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.916+0000 7efc95ffb700 1 -- 172.21.15.49:0/1609615137 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c003a80 con 0x7efc980ff6a0 2023-10-08T09:13:10.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.916+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9819dd30 con 0x7efc980ff6a0 2023-10-08T09:13:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.918+0000 7efc95ffb700 1 -- 172.21.15.49:0/1609615137 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efc8c0098a0 con 0x7efc980ff6a0 2023-10-08T09:13:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.918+0000 7efc95ffb700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc880a1480 0x7efc880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.918+0000 7efc95ffb700 1 -- 172.21.15.49:0/1609615137 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc8c0ccc00 con 0x7efc980ff6a0 2023-10-08T09:13:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.918+0000 7efc97fff700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc880a1480 0x7efc880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.919+0000 7efc97fff700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc880a1480 0x7efc880a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc84009cb0 tx=0x7efc84009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.919+0000 7efcaaed7b80 1 Processor -- start 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.919+0000 7efcaaed7b80 1 -- start start 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0970 0x5601eb9d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d17b0 0x5601eb9d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb938410 con 0x5601eb9d17b0 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb929bd0 con 0x5601eb9d0010 2023-10-08T09:13:10.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb9274f0 con 0x5601eb9d0970 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d17b0 0x5601eb9d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d17b0 0x5601eb9d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48002/0 (socket says 172.21.15.49:48002) 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 -- 172.21.15.49:0/134624708 learned_addr learned my addr 172.21.15.49:0/134624708 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc97fff700 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0970 0x5601eb9d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 -- 172.21.15.49:0/134624708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0970 msgr2=0x5601eb9d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0970 0x5601eb9d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 -- 172.21.15.49:0/134624708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 msgr2=0x5601eb9d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 -- 172.21.15.49:0/134624708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601eb74dd90 con 0x5601eb9d17b0 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.920+0000 7efc9d28b700 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d17b0 0x5601eb9d1c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc90015000 tx=0x7efc90005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df1664daf21dcea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efc82ffd700 1 -- 172.21.15.49:0/134624708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000e040 con 0x5601eb9d17b0 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efc82ffd700 1 -- 172.21.15.49:0/134624708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc9000baa0 con 0x5601eb9d17b0 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efc82ffd700 1 -- 172.21.15.49:0/134624708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90005070 con 0x5601eb9d17b0 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 -- 172.21.15.49:0/134624708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d17b0 msgr2=0x5601eb9d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d17b0 0x5601eb9d1c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efc90015000 tx=0x7efc90005a20 comp rx=0 tx=0).stop 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 -- 172.21.15.49:0/134624708 shutdown_connections 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d17b0 0x5601eb9d1c40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/134624708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0970 0x5601eb9d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 -- 172.21.15.49:0/134624708 >> 172.21.15.49:0/134624708 conn(0x5601eb944210 msgr2=0x5601eb9cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 -- 172.21.15.49:0/134624708 shutdown_connections 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.921+0000 7efcaaed7b80 1 -- 172.21.15.49:0/134624708 wait complete. 2023-10-08T09:13:10.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 -- start start 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 Processor -- start 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eb9e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb922de0 con 0x5601eb9d0970 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb8bf6c0 con 0x5601eb9d0010 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb8bf360 con 0x5601eb9d17b0 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efcaaed7b80 10 client.0 _ll_register_callbacks cb 0x5601eb71c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:13:10.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efc97fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efc97fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9da260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48006/0 (socket says 172.21.15.49:48006) 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efc97fff700 1 -- 172.21.15.49:0/87795756 learned_addr learned my addr 172.21.15.49:0/87795756 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.922+0000 7efc9d28b700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eb9e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc97fff700 1 -- 172.21.15.49:0/87795756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 msgr2=0x5601eb9e12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc97fff700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eb9e12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc97fff700 1 -- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 msgr2=0x5601eb9d9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc97fff700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc97fff700 1 -- 172.21.15.49:0/87795756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc98108430 con 0x5601eb9d0970 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc9d28b700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eb9e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc97fff700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9da260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efc840045a0 tx=0x7efc840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaa715a4148e260 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc77fff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc77fff700 1 -- 172.21.15.49:0/87795756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84012450 con 0x5601eb9d0970 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc77fff700 1 -- 172.21.15.49:0/87795756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc84018070 con 0x5601eb9d0970 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601eb8eae60 con 0x5601eb9d0970 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc77fff700 1 -- 172.21.15.49:0/87795756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc840125b0 con 0x5601eb9d0970 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efcaaed7b80 10 client.14859 fetch_fsmap learned FSMap version 17 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5601eb736c10 con 0x5601eb9d0970 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc77fff700 1 -- 172.21.15.49:0/87795756 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601eb8eae60 con 0x5601eb9d0970 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efc77fff700 1 -- 172.21.15.49:0/87795756 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc84021730 con 0x5601eb9d0970 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efcaaed7b80 10 client.14859 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.923+0000 7efcaaed7b80 10 client.14859 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.924+0000 7efcaaed7b80 20 client.14859 populate_metadata read hostname 'smithi049' 2023-10-08T09:13:10.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.924+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eb9e59f0 0x5601eb9e4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.924+0000 7efcaaed7b80 4 client.14859 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.924+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601eb91bdc0 con 0x5601eb9e59f0 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.924+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eb9e59f0 0x5601eb9e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.924+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eb9e59f0 0x5601eb9e4450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f125ababfcef49 server_cookie=5351a99a5f667239 in_seq=0 out_seq=0 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.924+0000 7efc77fff700 10 client.14859 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efc77fff700 1 -- 172.21.15.49:0/87795756 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5601eb91bdc0 con 0x5601eb9e59f0 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efc77fff700 10 client.14859 handle_command_reply: tid=0 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 client.14859 shutdown 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 msgr2=0x5601eb9da260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9da260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efc840045a0 tx=0x7efc840047c0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 shutdown_connections 2023-10-08T09:13:10.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eb9e59f0 0x5601eb9e4450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eb9e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9d9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/87795756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9da260 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efc77fff700 0 client.14859 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 >> 172.21.15.49:0/87795756 conn(0x5601eb944210 msgr2=0x5601eb9ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 shutdown_connections 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 1 -- 172.21.15.49:0/87795756 wait complete. 2023-10-08T09:13:10.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.925+0000 7efcaaed7b80 20 client.14859 trim_cache size 0 max 16384 2023-10-08T09:13:10.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 Processor -- start 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 -- start start 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0010 0x5601eb9d92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d17b0 0x5601eb9d7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb7a38d0 con 0x5601eb9d0970 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb91eed0 con 0x5601eb9d17b0 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.943+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eb8c94d0 con 0x5601eb9d0010 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9e1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48014/0 (socket says 172.21.15.49:48014) 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 -- 172.21.15.49:0/1453561540 learned_addr learned my addr 172.21.15.49:0/1453561540 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:10.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0010 0x5601eb9d92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc9d28b700 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d17b0 0x5601eb9d7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 -- 172.21.15.49:0/1453561540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0010 msgr2=0x5601eb9d92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0010 0x5601eb9d92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 -- 172.21.15.49:0/1453561540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d17b0 msgr2=0x5601eb9d7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d17b0 0x5601eb9d7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 -- 172.21.15.49:0/1453561540 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601eb774200 con 0x5601eb9d0970 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0010 0x5601eb9d92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc97fff700 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9e1260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efc84005b60 tx=0x7efc84005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b56a11afbb4cf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc837fe700 1 -- 172.21.15.49:0/1453561540 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84003960 con 0x5601eb9d0970 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc837fe700 1 -- 172.21.15.49:0/1453561540 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc840218d0 con 0x5601eb9d0970 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.944+0000 7efc837fe700 1 -- 172.21.15.49:0/1453561540 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84012cb0 con 0x5601eb9d0970 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 -- 172.21.15.49:0/1453561540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 msgr2=0x5601eb9e1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9e1260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efc84005b60 tx=0x7efc84005980 comp rx=0 tx=0).stop 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 -- 172.21.15.49:0/1453561540 shutdown_connections 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d0010 0x5601eb9d92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d17b0 0x5601eb9d7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/1453561540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9e1260 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 -- 172.21.15.49:0/1453561540 >> 172.21.15.49:0/1453561540 conn(0x5601eb944210 msgr2=0x5601eb9c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 -- 172.21.15.49:0/1453561540 shutdown_connections 2023-10-08T09:13:10.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 -- 172.21.15.49:0/1453561540 wait complete. 2023-10-08T09:13:10.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 -- start start 2023-10-08T09:13:10.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.945+0000 7efcaaed7b80 1 Processor -- start 2023-10-08T09:13:10.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9de170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eba393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eba39920 con 0x5601eb9d0970 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eba39a60 con 0x5601eb9d0010 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601eba39ba0 con 0x5601eb9d17b0 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 10 client.0 _ll_register_callbacks cb 0x7efc840043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9de6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48020/0 (socket says 172.21.15.49:48020) 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 -- 172.21.15.49:0/3114603211 learned_addr learned my addr 172.21.15.49:0/3114603211 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc9d28b700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eba393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 -- 172.21.15.49:0/3114603211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 msgr2=0x5601eba393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eba393e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 -- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 msgr2=0x5601eb9de170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9de170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 -- 172.21.15.49:0/3114603211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601eb736c10 con 0x5601eb9d0970 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9de170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc97fff700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9de6b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efc84009620 tx=0x7efc840099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d7b45cc9a776e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc77fff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc77fff700 1 -- 172.21.15.49:0/3114603211 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84021610 con 0x5601eb9d0970 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc77fff700 1 -- 172.21.15.49:0/3114603211 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc84018030 con 0x5601eb9d0970 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc9000baa0 con 0x5601eb9d0970 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.946+0000 7efc77fff700 1 -- 172.21.15.49:0/3114603211 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc84021770 con 0x5601eb9d0970 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 10 client.14865 fetch_fsmap learned FSMap version 17 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5601eb74c260 con 0x5601eb9d0970 2023-10-08T09:13:10.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efc77fff700 1 -- 172.21.15.49:0/3114603211 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc84054080 con 0x5601eb9d0970 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efc77fff700 1 -- 172.21.15.49:0/3114603211 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efc84047030 con 0x5601eb9d0970 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 10 client.14865 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 10 client.14865 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 20 client.14865 populate_metadata read hostname 'smithi049' 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eba3e930 0x5601eba3adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 4 client.14865 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5601eb91bae0 con 0x5601eba3e930 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.947+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eba3e930 0x5601eba3adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efc77fff700 10 client.14865 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efc9ca8a700 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eba3e930 0x5601eba3adb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d20ec58449a64c server_cookie=2c023a205c641f56 in_seq=0 out_seq=0 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efc77fff700 1 -- 172.21.15.49:0/3114603211 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5601eb91bae0 con 0x5601eba3e930 2023-10-08T09:13:10.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efc77fff700 10 client.14865 handle_command_reply: tid=0 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 client.14865 shutdown 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 msgr2=0x5601eb9de6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9de6b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7efc84009620 tx=0x7efc840099e0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 shutdown_connections 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5601eba3e930 0x5601eba3adb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5601eb9d17b0 0x5601eba393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efc77fff700 0 client.14865 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5601eb9d0010 0x5601eb9de170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 --2- 172.21.15.49:0/3114603211 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5601eb9d0970 0x5601eb9de6b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 >> 172.21.15.49:0/3114603211 conn(0x5601eb944210 msgr2=0x5601eb9d7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 shutdown_connections 2023-10-08T09:13:10.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 1 -- 172.21.15.49:0/3114603211 wait complete. 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.948+0000 7efcaaed7b80 20 client.14865 trim_cache size 0 max 16384 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24638, 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:13:10.957 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 7.2289274209999999, 2023-10-08T09:13:10.958 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 17, 2023-10-08T09:13:10.958 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 86, 2023-10-08T09:13:10.958 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 85, 2023-10-08T09:13:10.958 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 14.734335572000001 2023-10-08T09:13:10.958 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:13:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc880a1480 msgr2=0x7efc880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc880a1480 0x7efc880a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc84009cb0 tx=0x7efc84009260 comp rx=0 tx=0).stop 2023-10-08T09:13:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc980ff6a0 msgr2=0x7efc98195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc980ff6a0 0x7efc98195740 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc8c00d980 tx=0x7efc8c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:13:10.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 shutdown_connections 2023-10-08T09:13:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc880a1480 0x7efc880a3940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc98101230 0x7efc9819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc980ff6a0 0x7efc98195740 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 --2- 172.21.15.49:0/1609615137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc98100890 0x7efc98195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 >> 172.21.15.49:0/1609615137 conn(0x7efc980fac10 msgr2=0x7efc98103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 shutdown_connections 2023-10-08T09:13:10.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:10.949+0000 7efc9ecee700 1 -- 172.21.15.49:0/1609615137 wait complete. 2023-10-08T09:13:10.961 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T09:13:10.962 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:10.962 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-08T09:13:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:10 smithi049 ceph-mon[99485]: pgmap v273: 129 pgs: 8 creating+peering, 108 active+clean, 13 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 764 B/s wr, 7 op/s 2023-10-08T09:13:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:10 smithi049 ceph-mon[99485]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:13:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:10 smithi049 ceph-mon[99485]: Cluster is now healthy 2023-10-08T09:13:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/168429197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/445765700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/320753258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:10 smithi184 ceph-mon[103675]: pgmap v273: 129 pgs: 8 creating+peering, 108 active+clean, 13 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 764 B/s wr, 7 op/s 2023-10-08T09:13:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:10 smithi184 ceph-mon[103675]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:13:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:10 smithi184 ceph-mon[103675]: Cluster is now healthy 2023-10-08T09:13:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:10 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/168429197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:10 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/445765700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:10 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/320753258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:10 smithi132 ceph-mon[103389]: pgmap v273: 129 pgs: 8 creating+peering, 108 active+clean, 13 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 764 B/s wr, 7 op/s 2023-10-08T09:13:11.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:10 smithi132 ceph-mon[103389]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-08T09:13:11.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:10 smithi132 ceph-mon[103389]: Cluster is now healthy 2023-10-08T09:13:11.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:10 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/168429197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:10 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/445765700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:10 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/320753258' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:11.520 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:13.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 -- 172.21.15.49:0/3610653543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1008a0 msgr2=0x7fde6c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:13.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 --2- 172.21.15.49:0/3610653543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1008a0 0x7fde6c100d20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fde68009990 tx=0x7fde68009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 -- 172.21.15.49:0/3610653543 shutdown_connections 2023-10-08T09:13:13.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 --2- 172.21.15.49:0/3610653543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6c1057c0 0x7fde6c105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 --2- 172.21.15.49:0/3610653543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde6c101260 0x7fde6c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 --2- 172.21.15.49:0/3610653543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1008a0 0x7fde6c100d20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 -- 172.21.15.49:0/3610653543 >> 172.21.15.49:0/3610653543 conn(0x7fde6c0779e0 msgr2=0x7fde6c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:13.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 -- 172.21.15.49:0/3610653543 shutdown_connections 2023-10-08T09:13:13.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.047+0000 7fde738e9700 1 -- 172.21.15.49:0/3610653543 wait complete. 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 Processor -- start 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 -- start start 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde6c1008a0 0x7fde6c19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6c101260 0x7fde6c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1057c0 0x7fde6c196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c197130 con 0x7fde6c1057c0 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c1972a0 con 0x7fde6c1008a0 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.048+0000 7fde738e9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c197460 con 0x7fde6c101260 2023-10-08T09:13:13.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1057c0 0x7fde6c196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1057c0 0x7fde6c196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48034/0 (socket says 172.21.15.49:48034) 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 -- 172.21.15.49:0/2966737311 learned_addr learned my addr 172.21.15.49:0/2966737311 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde70e84700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6c101260 0x7fde6c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71685700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde6c1008a0 0x7fde6c19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 -- 172.21.15.49:0/2966737311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6c101260 msgr2=0x7fde6c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6c101260 0x7fde6c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 -- 172.21.15.49:0/2966737311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde6c1008a0 msgr2=0x7fde6c19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde6c1008a0 0x7fde6c19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 -- 172.21.15.49:0/2966737311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680095f0 con 0x7fde6c1057c0 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde70e84700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6c101260 0x7fde6c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde71e86700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1057c0 0x7fde6c196ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde6400ab50 tx=0x7fde6400af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:13.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde627fc700 1 -- 172.21.15.49:0/2966737311 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6400f070 con 0x7fde6c1057c0 2023-10-08T09:13:13.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c197740 con 0x7fde6c1057c0 2023-10-08T09:13:13.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde627fc700 1 -- 172.21.15.49:0/2966737311 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde64007a80 con 0x7fde6c1057c0 2023-10-08T09:13:13.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde627fc700 1 -- 172.21.15.49:0/2966737311 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64005310 con 0x7fde6c1057c0 2023-10-08T09:13:13.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.049+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c1a3930 con 0x7fde6c1057c0 2023-10-08T09:13:13.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.050+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c108da0 con 0x7fde6c1057c0 2023-10-08T09:13:13.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.051+0000 7fde627fc700 1 -- 172.21.15.49:0/2966737311 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fde64004ad0 con 0x7fde6c1057c0 2023-10-08T09:13:13.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.052+0000 7fde627fc700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fde580a1480 0x7fde580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:13.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.052+0000 7fde71685700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fde580a1480 0x7fde580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:13.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.052+0000 7fde627fc700 1 -- 172.21.15.49:0/2966737311 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fde640ccaf0 con 0x7fde6c1057c0 2023-10-08T09:13:13.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.052+0000 7fde71685700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fde580a1480 0x7fde580a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005fd0 tx=0x7fde5c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:13.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.056+0000 7fde627fc700 1 -- 172.21.15.49:0/2966737311 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fde64095f90 con 0x7fde6c1057c0 2023-10-08T09:13:13.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.301+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fde6c101f30 con 0x7fde6c1057c0 2023-10-08T09:13:13.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.301+0000 7fde627fc700 1 -- 172.21.15.49:0/2966737311 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fde640995b0 con 0x7fde6c1057c0 2023-10-08T09:13:13.302 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fde580a1480 msgr2=0x7fde580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fde580a1480 0x7fde580a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fde5c005fd0 tx=0x7fde5c00a4c0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1057c0 msgr2=0x7fde6c196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1057c0 0x7fde6c196ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde6400ab50 tx=0x7fde6400af10 comp rx=0 tx=0).stop 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 shutdown_connections 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fde580a1480 0x7fde580a3940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fde6c1057c0 0x7fde6c196ad0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde6c1008a0 0x7fde6c19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 --2- 172.21.15.49:0/2966737311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6c101260 0x7fde6c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:13.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 >> 172.21.15.49:0/2966737311 conn(0x7fde6c0779e0 msgr2=0x7fde6c0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:13.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 shutdown_connections 2023-10-08T09:13:13.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:13.303+0000 7fde738e9700 1 -- 172.21.15.49:0/2966737311 wait complete. 2023-10-08T09:13:13.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:13 smithi049 ceph-mon[99485]: pgmap v274: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2023-10-08T09:13:13.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:13 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2966737311' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:13:13.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:13 smithi184 ceph-mon[103675]: pgmap v274: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2023-10-08T09:13:13.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:13 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2966737311' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:13:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:13 smithi132 ceph-mon[103389]: pgmap v274: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2023-10-08T09:13:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:13 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2966737311' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:13:13.885 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T09:13:13.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T09:13:13.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-10-08T09:13:13.885 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-08T09:13:13.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:13.886 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T09:13:13.886 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T09:13:13.886 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T09:13:13.886 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T09:13:13.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:13.886 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-10-08T09:13:13.905 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T09:13:13.905 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T09:13:13.905 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T09:13:13.905 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:13:13.906 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-08T09:13:13.907 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 41867sec preferred_lft 41867sec 2023-10-08T09:13:13.907 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-10-08T09:13:13.907 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:13:13.907 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:13:13.907 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:13:13.907 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T09:13:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:13.908 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:13:13.908 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:13:13.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2023-10-08T09:13:13.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2023-10-08T09:13:13.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2023-10-08T09:13:13.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T09:13:13.908 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:13:14.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.106 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T09:13:14.136 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-10-08T09:13:14.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.138 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-10-08T09:13:14.217 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-10-08T09:13:14.218 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T09:13:14.218 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-08T09:13:14.218 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-08T09:13:14.218 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T09:13:14.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.219 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:13:14.219 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:13:14.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-08T09:13:14.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-08T09:13:14.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-08T09:13:14.219 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:13:14.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.527 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-10-08T09:13:14.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.545 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2023-10-08T09:13:14.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.602 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:13:14.602 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:13:14.602 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:13:14.602 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-08T09:13:14.602 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:13:14.699 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-08T09:13:14.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.699 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:13:14.699 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:13:14.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-08T09:13:14.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T09:13:14.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-08T09:13:14.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-08T09:13:14.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-08T09:13:14.700 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:13:14.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:14.920 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:13:14.920 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:13:14.920 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-10-08T09:13:14.920 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T09:13:14.920 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:13:15.019 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-08T09:13:15.019 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:13:15.019 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:15.036 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T09:13:15.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:13:15.037 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:15.094 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-10-08T09:13:15.162 DEBUG:teuthology.orchestra.run.smithi049:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T09:13:15.220 INFO:teuthology.orchestra.run.smithi049.stdout:/sys 2023-10-08T09:13:15.220 INFO:teuthology.orchestra.run.smithi049.stdout:/proc 2023-10-08T09:13:15.220 INFO:teuthology.orchestra.run.smithi049.stdout:/dev 2023-10-08T09:13:15.220 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/security 2023-10-08T09:13:15.220 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/shm 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/pts 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/run 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/systemd 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/pstore 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/bpf 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/blkio 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/devices 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/perf_event 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/cpuset 2023-10-08T09:13:15.221 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/memory 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/hugetlb 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/freezer 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/rdma 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/pids 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/tracing 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/config 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/ 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/selinux 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:13:15.222 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/hugepages 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/mqueue 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/debug 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/fuse/connections 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/1000 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/ceph 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/aae671c9bfdd41db3a1384a4325637d92ff6fe73ea717772e716d806d2a75e36/merged 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/9a40fe4ce77576d3cc430a50c70c4a7bc548927390e9a76e644892c3d11a5dc9/merged 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/0 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:13:15.223 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/13e03bee4ecfa3554a51432b5f4da9585010836f5a4e44034259d1eca14ad3b1/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/448ab56d6c1c10dd8104807e7d5822b3e63af0eab9eb5649cb63e2a0e52c96c0/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/0cb2676e16a521c84671eb21e2cac6ede13921d455d4ebafb52a6b6de942b990/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a3d1d298d04649df05c40d8233ebbfd19918fe655a73ae5371b9fca2436fcf21/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a98d8f0a56a249aa607050ce752dc51b184b106cd044a96dc22beef5ea8cfefa/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/70d36ee1014f6cf8f0b02d001b287bf12f7bc809120a80dbe2ef3826099db98a/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/655e3cd98062bb2c06227530851bd537aca2a03abcfd3cad4cdb43298ead26ef/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/3e77ff115712886bc64a575566e0fa1d0e608902e90f1c121c11956dce6cab2d/merged 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:13:15.224 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:13:15.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:15.225 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-10-08T09:13:15.278 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T09:13:15.278 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-08T09:13:15.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo modprobe fuse 2023-10-08T09:13:15.351 DEBUG:teuthology.orchestra.run.smithi049:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T09:13:15.410 INFO:teuthology.orchestra.run.smithi049.stdout:/sys 2023-10-08T09:13:15.410 INFO:teuthology.orchestra.run.smithi049.stdout:/proc 2023-10-08T09:13:15.410 INFO:teuthology.orchestra.run.smithi049.stdout:/dev 2023-10-08T09:13:15.410 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/security 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/shm 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/pts 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/run 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/systemd 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/pstore 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/bpf 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/blkio 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/devices 2023-10-08T09:13:15.411 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/perf_event 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/cpuset 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/memory 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/hugetlb 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/freezer 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/rdma 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/cgroup/pids 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/tracing 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/config 2023-10-08T09:13:15.412 INFO:teuthology.orchestra.run.smithi049.stdout:/ 2023-10-08T09:13:15.413 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/selinux 2023-10-08T09:13:15.413 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:13:15.413 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/hugepages 2023-10-08T09:13:15.413 INFO:teuthology.orchestra.run.smithi049.stdout:/dev/mqueue 2023-10-08T09:13:15.413 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/kernel/debug 2023-10-08T09:13:15.413 INFO:teuthology.orchestra.run.smithi049.stdout:/sys/fs/fuse/connections 2023-10-08T09:13:15.413 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/1000 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/ceph 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/aae671c9bfdd41db3a1384a4325637d92ff6fe73ea717772e716d806d2a75e36/merged 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/9a40fe4ce77576d3cc430a50c70c4a7bc548927390e9a76e644892c3d11a5dc9/merged 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/run/user/0 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/13e03bee4ecfa3554a51432b5f4da9585010836f5a4e44034259d1eca14ad3b1/merged 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/448ab56d6c1c10dd8104807e7d5822b3e63af0eab9eb5649cb63e2a0e52c96c0/merged 2023-10-08T09:13:15.414 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/0cb2676e16a521c84671eb21e2cac6ede13921d455d4ebafb52a6b6de942b990/merged 2023-10-08T09:13:15.415 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a3d1d298d04649df05c40d8233ebbfd19918fe655a73ae5371b9fca2436fcf21/merged 2023-10-08T09:13:15.415 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/a98d8f0a56a249aa607050ce752dc51b184b106cd044a96dc22beef5ea8cfefa/merged 2023-10-08T09:13:15.415 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/70d36ee1014f6cf8f0b02d001b287bf12f7bc809120a80dbe2ef3826099db98a/merged 2023-10-08T09:13:15.415 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/655e3cd98062bb2c06227530851bd537aca2a03abcfd3cad4cdb43298ead26ef/merged 2023-10-08T09:13:15.415 INFO:teuthology.orchestra.run.smithi049.stdout:/var/lib/containers/storage/overlay/3e77ff115712886bc64a575566e0fa1d0e608902e90f1c121c11956dce6cab2d/merged 2023-10-08T09:13:15.416 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns 2023-10-08T09:13:15.416 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:13:15.416 INFO:teuthology.orchestra.run.smithi049.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:13:15.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:15.416 DEBUG:teuthology.orchestra.run.smithi049:> ls /sys/fs/fuse/connections 2023-10-08T09:13:15.453 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:2023-10-08T09:13:15.452+0000 7f7e783d75c0 -1 init, newargv = 0x5594b05cc940 newargc=15 2023-10-08T09:13:15.453 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:ceph-fuse[157776]: starting ceph client 2023-10-08T09:13:15.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:ceph-fuse[157776]: starting fuse 2023-10-08T09:13:15.470 INFO:teuthology.orchestra.run.smithi049.stdout:124 2023-10-08T09:13:15.471 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-10-08T09:13:15.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo stdin-killer -- python3 -c ' 2023-10-08T09:13:15.471 DEBUG:teuthology.orchestra.run.smithi049:> import glob 2023-10-08T09:13:15.471 DEBUG:teuthology.orchestra.run.smithi049:> import re 2023-10-08T09:13:15.471 DEBUG:teuthology.orchestra.run.smithi049:> import os 2023-10-08T09:13:15.471 DEBUG:teuthology.orchestra.run.smithi049:> import subprocess 2023-10-08T09:13:15.471 DEBUG:teuthology.orchestra.run.smithi049:> 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> def _find_admin_socket(client_name): 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> files = glob.glob(asok_path) 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> # Given a non-glob path, it better be there 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> if "*" not in asok_path: 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> assert(len(files) == 1) 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> return files[0] 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> for f in files: 2023-10-08T09:13:15.472 DEBUG:teuthology.orchestra.run.smithi049:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> contents = proc_f.read() 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> if mountpoint in contents: 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> return f 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> print(_find_admin_socket("client.admin")) 2023-10-08T09:13:15.473 DEBUG:teuthology.orchestra.run.smithi049:> ' 2023-10-08T09:13:15.597 INFO:teuthology.orchestra.run.smithi049.stdout:/var/run/ceph/ceph-client.admin.157776.asok 2023-10-08T09:13:15.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-08T09:13:15.606 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:15 smithi049 ceph-mon[99485]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 438 B/s wr, 4 op/s 2023-10-08T09:13:15.607 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.157776.asok 2023-10-08T09:13:15.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:15.607 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.157776.asok status 2023-10-08T09:13:15.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:15 smithi184 ceph-mon[103675]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 438 B/s wr, 4 op/s 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": { 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_sha1": "7b3143bae3b6870a749bfb9810ed6e7d0e89d7de", 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_version": "ceph version 18.0.0-6639-g7b3143ba (7b3143bae3b6870a749bfb9810ed6e7d0e89d7de) reef (dev)", 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "entity_id": "admin", 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": "smithi049", 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "pid": "157776", 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: "root": "/" 2023-10-08T09:13:15.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_count": 0, 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "dentry_pinned_count": 0, 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 14877, 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "inst": { 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "name": { 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "client", 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "num": 14877 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-10-08T09:13:15.770 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4144967197 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": { 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "192.168.0.1:0", 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4144967197 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "inst_str": "client.14877 192.168.0.1:0/4144967197", 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "addr_str": "192.168.0.1:0/4144967197", 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "inode_count": 1, 2023-10-08T09:13:15.771 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_epoch": 17, 2023-10-08T09:13:15.772 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch": 86, 2023-10-08T09:13:15.772 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_epoch_barrier": 0, 2023-10-08T09:13:15.772 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklisted": false, 2023-10-08T09:13:15.772 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs" 2023-10-08T09:13:15.772 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:13:15.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:15.779 DEBUG:teuthology.orchestra.run.smithi049:> stat --file-system '--printf=%T 2023-10-08T09:13:15.779 DEBUG:teuthology.orchestra.run.smithi049:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:15.797 INFO:teuthology.orchestra.run.smithi049.stdout:fuseblk 2023-10-08T09:13:15.797 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:15.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:15.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:15.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:15 smithi132 ceph-mon[103389]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 438 B/s wr, 4 op/s 2023-10-08T09:13:15.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:15.868 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-08T09:13:15.977 INFO:teuthology.orchestra.run.smithi049.stdout:# file: . 2023-10-08T09:13:15.978 INFO:teuthology.orchestra.run.smithi049.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-08T09:13:15.978 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:15.978 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:13:15.978 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:13:15.997 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:16.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 -- 172.21.15.49:0/2178035893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 msgr2=0x7ff448107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 --2- 172.21.15.49:0/2178035893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff448107aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b0a0 tx=0x7ff44000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 -- 172.21.15.49:0/2178035893 shutdown_connections 2023-10-08T09:13:16.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 --2- 172.21.15.49:0/2178035893 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff448107aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 --2- 172.21.15.49:0/2178035893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff448100bf0 0x7ff448101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 --2- 172.21.15.49:0/2178035893 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4480ff840 0x7ff4480ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 -- 172.21.15.49:0/2178035893 >> 172.21.15.49:0/2178035893 conn(0x7ff4480fa2f0 msgr2=0x7ff4480fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:16.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 -- 172.21.15.49:0/2178035893 shutdown_connections 2023-10-08T09:13:16.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.162+0000 7ff44d790700 1 -- 172.21.15.49:0/2178035893 wait complete. 2023-10-08T09:13:16.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 Processor -- start 2023-10-08T09:13:16.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 -- start start 2023-10-08T09:13:16.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4480ff840 0x7ff4480ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff44819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff448100bf0 0x7ff44819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819e200 con 0x7ff448100200 2023-10-08T09:13:16.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819e370 con 0x7ff448100bf0 2023-10-08T09:13:16.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.163+0000 7ff44d790700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44819e530 con 0x7ff4480ff840 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff44819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff44819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48070/0 (socket says 172.21.15.49:48070) 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 -- 172.21.15.49:0/2419266374 learned_addr learned my addr 172.21.15.49:0/2419266374 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff446ffd700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4480ff840 0x7ff4480ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4477fe700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff448100bf0 0x7ff44819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 -- 172.21.15.49:0/2419266374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4480ff840 msgr2=0x7ff4480ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4480ff840 0x7ff4480ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 -- 172.21.15.49:0/2419266374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff448100bf0 msgr2=0x7ff44819da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff448100bf0 0x7ff44819da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 -- 172.21.15.49:0/2419266374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009d00 con 0x7ff448100200 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4477fe700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff448100bf0 0x7ff44819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff4467fc700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff44819b3f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d980 tx=0x7ff43800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff43ffff700 1 -- 172.21.15.49:0/2419266374 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438004a30 con 0x7ff448100200 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff43ffff700 1 -- 172.21.15.49:0/2419266374 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4380054e0 con 0x7ff448100200 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44819e7b0 con 0x7ff448100200 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff43ffff700 1 -- 172.21.15.49:0/2419266374 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438009c90 con 0x7ff448100200 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.164+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44819ec70 con 0x7ff448100200 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.165+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448106a10 con 0x7ff448100200 2023-10-08T09:13:16.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.166+0000 7ff43ffff700 1 -- 172.21.15.49:0/2419266374 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff438004c20 con 0x7ff448100200 2023-10-08T09:13:16.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.167+0000 7ff43ffff700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff4340a1480 0x7ff4340a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.167+0000 7ff43ffff700 1 -- 172.21.15.49:0/2419266374 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff4380cd030 con 0x7ff448100200 2023-10-08T09:13:16.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.167+0000 7ff446ffd700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff4340a1480 0x7ff4340a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.168+0000 7ff446ffd700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff4340a1480 0x7ff4340a3940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff448068a90 tx=0x7ff43000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:16.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.171+0000 7ff43ffff700 1 -- 172.21.15.49:0/2419266374 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff4380964a0 con 0x7ff448100200 2023-10-08T09:13:16.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.375+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff44810ae70 con 0x7ff448100200 2023-10-08T09:13:16.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.377+0000 7ff43ffff700 1 -- 172.21.15.49:0/2419266374 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7ff438017020 con 0x7ff448100200 2023-10-08T09:13:16.378 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:16.378 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff4340a1480 msgr2=0x7ff4340a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff4340a1480 0x7ff4340a3940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff448068a90 tx=0x7ff43000a3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 msgr2=0x7ff44819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff44819b3f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d980 tx=0x7ff43800dc90 comp rx=0 tx=0).stop 2023-10-08T09:13:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 shutdown_connections 2023-10-08T09:13:16.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff4340a1480 0x7ff4340a3940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4480ff840 0x7ff4480ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff448100bf0 0x7ff44819da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 --2- 172.21.15.49:0/2419266374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff448100200 0x7ff44819b3f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 >> 172.21.15.49:0/2419266374 conn(0x7ff4480fa2f0 msgr2=0x7ff448103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:16.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 shutdown_connections 2023-10-08T09:13:16.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.379+0000 7ff44d790700 1 -- 172.21.15.49:0/2419266374 wait complete. 2023-10-08T09:13:16.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:13:16.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 -- 172.21.15.49:0/3434938793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 msgr2=0x7ffbe41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/3434938793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe41088f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00b0a0 tx=0x7ffbdc00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 -- 172.21.15.49:0/3434938793 shutdown_connections 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/3434938793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe41088f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/3434938793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbe4101220 0x7ffbe4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/3434938793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe4103b50 0x7ffbe4105f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 -- 172.21.15.49:0/3434938793 >> 172.21.15.49:0/3434938793 conn(0x7ffbe40fac10 msgr2=0x7ffbe40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 -- 172.21.15.49:0/3434938793 shutdown_connections 2023-10-08T09:13:16.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.522+0000 7ffbeaa76700 1 -- 172.21.15.49:0/3434938793 wait complete. 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 Processor -- start 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 -- start start 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe4101220 0x7ffbe4196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbe4103b50 0x7ffbe4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe419dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe419e330 con 0x7ffbe4106480 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe419e470 con 0x7ffbe4103b50 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbeaa76700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe419e600 con 0x7ffbe4101220 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe419dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe419dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48086/0 (socket says 172.21.15.49:48086) 2023-10-08T09:13:16.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 -- 172.21.15.49:0/2969029840 learned_addr learned my addr 172.21.15.49:0/2969029840 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe8812700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe4101220 0x7ffbe4196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe3fff700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbe4103b50 0x7ffbe4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 -- 172.21.15.49:0/2969029840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe4101220 msgr2=0x7ffbe4196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe4101220 0x7ffbe4196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 -- 172.21.15.49:0/2969029840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbe4103b50 msgr2=0x7ffbe4196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbe4103b50 0x7ffbe4196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.523+0000 7ffbe9013700 1 -- 172.21.15.49:0/2969029840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc009d00 con 0x7ffbe4106480 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.524+0000 7ffbe9013700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe419dd60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc007ab0 tx=0x7ffbdc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.524+0000 7ffbe1ffb700 1 -- 172.21.15.49:0/2969029840 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00e040 con 0x7ffbe4106480 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.524+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe419e880 con 0x7ffbe4106480 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.524+0000 7ffbe1ffb700 1 -- 172.21.15.49:0/2969029840 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbdc004e70 con 0x7ffbe4106480 2023-10-08T09:13:16.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.524+0000 7ffbe1ffb700 1 -- 172.21.15.49:0/2969029840 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc01da70 con 0x7ffbe4106480 2023-10-08T09:13:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.524+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe419eca0 con 0x7ffbe4106480 2023-10-08T09:13:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.525+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe40fc790 con 0x7ffbe4106480 2023-10-08T09:13:16.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.526+0000 7ffbe1ffb700 1 -- 172.21.15.49:0/2969029840 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ffbdc0058a0 con 0x7ffbe4106480 2023-10-08T09:13:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.526+0000 7ffbe1ffb700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffbd40a1480 0x7ffbd40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.527+0000 7ffbe1ffb700 1 -- 172.21.15.49:0/2969029840 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ffbdc0ce050 con 0x7ffbe4106480 2023-10-08T09:13:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.527+0000 7ffbe8812700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffbd40a1480 0x7ffbd40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.527+0000 7ffbe8812700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffbd40a1480 0x7ffbd40a3940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8005950 tx=0x7ffbd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:16.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.531+0000 7ffbe1ffb700 1 -- 172.21.15.49:0/2969029840 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffbdc101100 con 0x7ffbe4106480 2023-10-08T09:13:16.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.733+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbe410bfe0 con 0x7ffbe4106480 2023-10-08T09:13:16.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.734+0000 7ffbe1ffb700 1 -- 172.21.15.49:0/2969029840 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbdc0974f0 con 0x7ffbe4106480 2023-10-08T09:13:16.734 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:13:16.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffbd40a1480 msgr2=0x7ffbd40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffbd40a1480 0x7ffbd40a3940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8005950 tx=0x7ffbd800b410 comp rx=0 tx=0).stop 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 msgr2=0x7ffbe419dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe419dd60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc007ab0 tx=0x7ffbdc004c80 comp rx=0 tx=0).stop 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 shutdown_connections 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffbd40a1480 0x7ffbd40a3940 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbe4101220 0x7ffbe4196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffbe4106480 0x7ffbe419dd60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 --2- 172.21.15.49:0/2969029840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffbe4103b50 0x7ffbe4196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 >> 172.21.15.49:0/2969029840 conn(0x7ffbe40fac10 msgr2=0x7ffbe4101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:16.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 shutdown_connections 2023-10-08T09:13:16.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.736+0000 7ffbeaa76700 1 -- 172.21.15.49:0/2969029840 wait complete. 2023-10-08T09:13:16.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:16 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2419266374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:16.747 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:16 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2419266374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:16.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:13:16.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:16 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2419266374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:16.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 -- 172.21.15.49:0/467912067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c80ff6a0 msgr2=0x7f26c80ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 --2- 172.21.15.49:0/467912067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c80ff6a0 0x7f26c80ffac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f26b80099c0 tx=0x7f26b8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 -- 172.21.15.49:0/467912067 shutdown_connections 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 --2- 172.21.15.49:0/467912067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26c8101230 0x7f26c8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 --2- 172.21.15.49:0/467912067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c80ff6a0 0x7f26c80ffac0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 --2- 172.21.15.49:0/467912067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c8100890 0x7f26c8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 -- 172.21.15.49:0/467912067 >> 172.21.15.49:0/467912067 conn(0x7f26c80fac10 msgr2=0x7f26c80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 -- 172.21.15.49:0/467912067 shutdown_connections 2023-10-08T09:13:16.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.877+0000 7f26ce98a700 1 -- 172.21.15.49:0/467912067 wait complete. 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 Processor -- start 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 -- start start 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c80ff6a0 0x7f26c8196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26c8100890 0x7f26c8196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c8101230 0x7f26c819dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c819e250 con 0x7f26c8101230 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c819e390 con 0x7f26c80ff6a0 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ce98a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c819e520 con 0x7f26c8100890 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c8101230 0x7f26c819dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c8101230 0x7f26c819dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48100/0 (socket says 172.21.15.49:48100) 2023-10-08T09:13:16.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 -- 172.21.15.49:0/2934434108 learned_addr learned my addr 172.21.15.49:0/2934434108 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26c77fe700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26c8100890 0x7f26c8196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26c7fff700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c80ff6a0 0x7f26c8196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 -- 172.21.15.49:0/2934434108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26c8100890 msgr2=0x7f26c8196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26c8100890 0x7f26c8196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 -- 172.21.15.49:0/2934434108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c80ff6a0 msgr2=0x7f26c8196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c80ff6a0 0x7f26c8196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.878+0000 7f26ccf27700 1 -- 172.21.15.49:0/2934434108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80095f0 con 0x7f26c8101230 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.879+0000 7f26ccf27700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c8101230 0x7f26c819dc80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007ee0 tx=0x7f26c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.879+0000 7f26c57fa700 1 -- 172.21.15.49:0/2934434108 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0017070 con 0x7f26c8101230 2023-10-08T09:13:16.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.879+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c819e800 con 0x7f26c8101230 2023-10-08T09:13:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.879+0000 7f26c57fa700 1 -- 172.21.15.49:0/2934434108 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c000f040 con 0x7f26c8101230 2023-10-08T09:13:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.879+0000 7f26c57fa700 1 -- 172.21.15.49:0/2934434108 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0005740 con 0x7f26c8101230 2023-10-08T09:13:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.879+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c819ecc0 con 0x7f26c8101230 2023-10-08T09:13:16.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.880+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c8067530 con 0x7f26c8101230 2023-10-08T09:13:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.881+0000 7f26c57fa700 1 -- 172.21.15.49:0/2934434108 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f26c000dc60 con 0x7f26c8101230 2023-10-08T09:13:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.881+0000 7f26c57fa700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26b40a1480 0x7f26b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.882+0000 7f26c57fa700 1 -- 172.21.15.49:0/2934434108 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f26c00ccde0 con 0x7f26c8101230 2023-10-08T09:13:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.882+0000 7f26c7fff700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26b40a1480 0x7f26b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:16.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.882+0000 7f26c7fff700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26b40a1480 0x7f26b40a3940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009990 tx=0x7f26b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:16.885 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:16.885+0000 7f26c57fa700 1 -- 172.21.15.49:0/2934434108 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f26c00961f0 con 0x7f26c8101230 2023-10-08T09:13:17.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.088+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26c810b040 con 0x7f26c8101230 2023-10-08T09:13:17.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.088+0000 7f26c57fa700 1 -- 172.21.15.49:0/2934434108 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26c0099810 con 0x7f26c8101230 2023-10-08T09:13:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:13:17.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.090+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26b40a1480 msgr2=0x7f26b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.090+0000 7f26ce98a700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26b40a1480 0x7f26b40a3940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f26b8009990 tx=0x7f26b801a040 comp rx=0 tx=0).stop 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.090+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c8101230 msgr2=0x7f26c819dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.090+0000 7f26ce98a700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c8101230 0x7f26c819dc80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007ee0 tx=0x7f26c000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 shutdown_connections 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26b40a1480 0x7f26b40a3940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26c8101230 0x7f26c819dc80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26c80ff6a0 0x7f26c8196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 --2- 172.21.15.49:0/2934434108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26c8100890 0x7f26c8196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 >> 172.21.15.49:0/2934434108 conn(0x7f26c80fac10 msgr2=0x7f26c80678e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:17.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 shutdown_connections 2023-10-08T09:13:17.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.091+0000 7f26ce98a700 1 -- 172.21.15.49:0/2934434108 wait complete. 2023-10-08T09:13:17.103 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.231+0000 7fa890f6f700 1 -- 172.21.15.49:0/113673267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1007c0 msgr2=0x7fa88c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.231+0000 7fa890f6f700 1 --2- 172.21.15.49:0/113673267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1007c0 0x7fa88c100be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0099c0 tx=0x7fa87c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 -- 172.21.15.49:0/113673267 shutdown_connections 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 --2- 172.21.15.49:0/113673267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c102350 0x7fa88c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 --2- 172.21.15.49:0/113673267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c1007c0 0x7fa88c100be0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 --2- 172.21.15.49:0/113673267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa88c1019b0 0x7fa88c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 -- 172.21.15.49:0/113673267 >> 172.21.15.49:0/113673267 conn(0x7fa88c0fbd30 msgr2=0x7fa88c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 -- 172.21.15.49:0/113673267 shutdown_connections 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 -- 172.21.15.49:0/113673267 wait complete. 2023-10-08T09:13:17.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 Processor -- start 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 -- start start 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.232+0000 7fa890f6f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c1007c0 0x7fa88c196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa890f6f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa88c1019b0 0x7fa88c196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa890f6f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c102350 0x7fa88c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa890f6f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c19e280 con 0x7fa88c102350 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa890f6f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c19e3c0 con 0x7fa88c1007c0 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa890f6f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c19e550 con 0x7fa88c1019b0 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c102350 0x7fa88c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c102350 0x7fa88c19dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48112/0 (socket says 172.21.15.49:48112) 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 -- 172.21.15.49:0/3728137142 learned_addr learned my addr 172.21.15.49:0/3728137142 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:17.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88a59c700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c1007c0 0x7fa88c196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa889d9b700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa88c1019b0 0x7fa88c196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 -- 172.21.15.49:0/3728137142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa88c1019b0 msgr2=0x7fa88c196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa88c1019b0 0x7fa88c196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 -- 172.21.15.49:0/3728137142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c1007c0 msgr2=0x7fa88c196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c1007c0 0x7fa88c196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 -- 172.21.15.49:0/3728137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0095f0 con 0x7fa88c102350 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa889d9b700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa88c1019b0 0x7fa88c196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa88ad9d700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c102350 0x7fa88c19dcb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ada0 tx=0x7fa88400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa87b7fe700 1 -- 172.21.15.49:0/3728137142 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400f070 con 0x7fa88c102350 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c19e830 con 0x7fa88c102350 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa87b7fe700 1 -- 172.21.15.49:0/3728137142 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa884007b30 con 0x7fa88c102350 2023-10-08T09:13:17.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa87b7fe700 1 -- 172.21.15.49:0/3728137142 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840053e0 con 0x7fa88c102350 2023-10-08T09:13:17.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.233+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c19ed40 con 0x7fa88c102350 2023-10-08T09:13:17.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.234+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c107c90 con 0x7fa88c102350 2023-10-08T09:13:17.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.235+0000 7fa87b7fe700 1 -- 172.21.15.49:0/3728137142 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa88400c980 con 0x7fa88c102350 2023-10-08T09:13:17.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.236+0000 7fa87b7fe700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8740a1480 0x7fa8740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.236+0000 7fa87b7fe700 1 -- 172.21.15.49:0/3728137142 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa8840ccb30 con 0x7fa88c102350 2023-10-08T09:13:17.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.236+0000 7fa88a59c700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8740a1480 0x7fa8740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.236+0000 7fa88a59c700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8740a1480 0x7fa8740a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa87c003820 tx=0x7fa87c005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:17.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.239+0000 7fa87b7fe700 1 -- 172.21.15.49:0/3728137142 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa884096030 con 0x7fa88c102350 2023-10-08T09:13:17.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.444+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa88c10c0e0 con 0x7fa88c102350 2023-10-08T09:13:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.445+0000 7fa87b7fe700 1 -- 172.21.15.49:0/3728137142 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa88401a310 con 0x7fa88c102350 2023-10-08T09:13:17.445 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:13:17.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8740a1480 msgr2=0x7fa8740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8740a1480 0x7fa8740a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa87c003820 tx=0x7fa87c005040 comp rx=0 tx=0).stop 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c102350 msgr2=0x7fa88c19dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c102350 0x7fa88c19dcb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ada0 tx=0x7fa88400c330 comp rx=0 tx=0).stop 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 shutdown_connections 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8740a1480 0x7fa8740a3940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa88c102350 0x7fa88c19dcb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa88c1007c0 0x7fa88c196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 --2- 172.21.15.49:0/3728137142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa88c1019b0 0x7fa88c196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 >> 172.21.15.49:0/3728137142 conn(0x7fa88c0fbd30 msgr2=0x7fa88c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:17.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 shutdown_connections 2023-10-08T09:13:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.447+0000 7fa890f6f700 1 -- 172.21.15.49:0/3728137142 wait complete. 2023-10-08T09:13:17.459 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:13:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 -- 172.21.15.49:0/567065681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781008c0 msgr2=0x7f4678100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 --2- 172.21.15.49:0/567065681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781008c0 0x7f4678100d20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f46680099c0 tx=0x7f4668009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 -- 172.21.15.49:0/567065681 shutdown_connections 2023-10-08T09:13:17.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 --2- 172.21.15.49:0/567065681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780ff6d0 0x7f46780ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 --2- 172.21.15.49:0/567065681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4678101260 0x7f4678107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 --2- 172.21.15.49:0/567065681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46781008c0 0x7f4678100d20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 -- 172.21.15.49:0/567065681 >> 172.21.15.49:0/567065681 conn(0x7f46780fac40 msgr2=0x7f46780fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 -- 172.21.15.49:0/567065681 shutdown_connections 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 -- 172.21.15.49:0/567065681 wait complete. 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 Processor -- start 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.588+0000 7f467fc65700 1 -- start start 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467fc65700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780ff6d0 0x7f4678103610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467fc65700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46781008c0 0x7f4678101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467fc65700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678101260 0x7f46781021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467fc65700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4678102980 con 0x7f4678101260 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467fc65700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4678102af0 con 0x7f46781008c0 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467fc65700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46781a0820 con 0x7f46780ff6d0 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678101260 0x7f46781021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678101260 0x7f46781021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48126/0 (socket says 172.21.15.49:48126) 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 -- 172.21.15.49:0/89545151 learned_addr learned my addr 172.21.15.49:0/89545151 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467da01700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780ff6d0 0x7f4678103610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467d200700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46781008c0 0x7f4678101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 -- 172.21.15.49:0/89545151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780ff6d0 msgr2=0x7f4678103610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780ff6d0 0x7f4678103610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 -- 172.21.15.49:0/89545151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46781008c0 msgr2=0x7f4678101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46781008c0 0x7f4678101ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 -- 172.21.15.49:0/89545151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46680095f0 con 0x7f4678101260 2023-10-08T09:13:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467e202700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678101260 0x7f46781021e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4670007ee0 tx=0x7f467000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f466effd700 1 -- 172.21.15.49:0/89545151 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670017070 con 0x7f4678101260 2023-10-08T09:13:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f466effd700 1 -- 172.21.15.49:0/89545151 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f467000f040 con 0x7f4678101260 2023-10-08T09:13:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46781a0ab0 con 0x7f4678101260 2023-10-08T09:13:17.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.589+0000 7f466effd700 1 -- 172.21.15.49:0/89545151 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46700054b0 con 0x7f4678101260 2023-10-08T09:13:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.590+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46781a0fc0 con 0x7f4678101260 2023-10-08T09:13:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.590+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4678066ca0 con 0x7f4678101260 2023-10-08T09:13:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.592+0000 7f466effd700 1 -- 172.21.15.49:0/89545151 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4670004ad0 con 0x7f4678101260 2023-10-08T09:13:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.593+0000 7f466effd700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46640a1500 0x7f46640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.593+0000 7f467da01700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46640a1500 0x7f46640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.593+0000 7f466effd700 1 -- 172.21.15.49:0/89545151 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f46700ccc40 con 0x7f4678101260 2023-10-08T09:13:17.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.593+0000 7f467da01700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46640a1500 0x7f46640a39c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4678100720 tx=0x7f467400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:17.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.596+0000 7f466effd700 1 -- 172.21.15.49:0/89545151 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f46700960b0 con 0x7f4678101260 2023-10-08T09:13:17.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:17 smithi184 ceph-mon[103675]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:17.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:17 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2969029840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:17.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:17 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2934434108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:13:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:17 smithi132 ceph-mon[103389]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:17 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2969029840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:17 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2934434108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:13:17.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.799+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f467810b070 con 0x7f4678101260 2023-10-08T09:13:17.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.799+0000 7f466effd700 1 -- 172.21.15.49:0/89545151 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f46700996d0 con 0x7f4678101260 2023-10-08T09:13:17.800 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:13:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46640a1500 msgr2=0x7f46640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46640a1500 0x7f46640a39c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4678100720 tx=0x7f467400b410 comp rx=0 tx=0).stop 2023-10-08T09:13:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678101260 msgr2=0x7f46781021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678101260 0x7f46781021e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4670007ee0 tx=0x7f467000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 shutdown_connections 2023-10-08T09:13:17.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46640a1500 0x7f46640a39c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780ff6d0 0x7f4678103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4678101260 0x7f46781021e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 --2- 172.21.15.49:0/89545151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46781008c0 0x7f4678101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 >> 172.21.15.49:0/89545151 conn(0x7f46780fac40 msgr2=0x7f4678068a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:17.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 shutdown_connections 2023-10-08T09:13:17.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.802+0000 7f467fc65700 1 -- 172.21.15.49:0/89545151 wait complete. 2023-10-08T09:13:17.813 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:17 smithi049 ceph-mon[99485]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:17.813 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:17 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2969029840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:17.813 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:17 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2934434108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:13:17.815 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T09:13:17.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.941+0000 7fceedcb7700 1 -- 172.21.15.49:0/2548744127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 msgr2=0x7fcee80ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.941+0000 7fceedcb7700 1 --2- 172.21.15.49:0/2548744127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee80ffac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fced80099c0 tx=0x7fced8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.941+0000 7fceedcb7700 1 -- 172.21.15.49:0/2548744127 shutdown_connections 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 --2- 172.21.15.49:0/2548744127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee8101230 0x7fcee8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 --2- 172.21.15.49:0/2548744127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee80ffac0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 --2- 172.21.15.49:0/2548744127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee8100890 0x7fcee8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 -- 172.21.15.49:0/2548744127 >> 172.21.15.49:0/2548744127 conn(0x7fcee80fac10 msgr2=0x7fcee80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 -- 172.21.15.49:0/2548744127 shutdown_connections 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 -- 172.21.15.49:0/2548744127 wait complete. 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 Processor -- start 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 -- start start 2023-10-08T09:13:17.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.942+0000 7fceedcb7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee8196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fceedcb7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee8100890 0x7fcee8196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fceedcb7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee8101230 0x7fcee819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fceedcb7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee819e2b0 con 0x7fcee80ff6a0 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fceedcb7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee819e3f0 con 0x7fcee8101230 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fceedcb7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee819e580 con 0x7fcee8100890 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee8196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee8196780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48134/0 (socket says 172.21.15.49:48134) 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 -- 172.21.15.49:0/3738558049 learned_addr learned my addr 172.21.15.49:0/3738558049 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:17.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee6ffd700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee8100890 0x7fcee8196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee7fff700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee8101230 0x7fcee819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 -- 172.21.15.49:0/3738558049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee8100890 msgr2=0x7fcee8196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee8100890 0x7fcee8196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 -- 172.21.15.49:0/3738558049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee8101230 msgr2=0x7fcee819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee8101230 0x7fcee819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 -- 172.21.15.49:0/3738558049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced80095f0 con 0x7fcee80ff6a0 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee7fff700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee8101230 0x7fcee819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee77fe700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee8196780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fced8006010 tx=0x7fced8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee4ff9700 1 -- 172.21.15.49:0/3738558049 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced801d070 con 0x7fcee80ff6a0 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee4ff9700 1 -- 172.21.15.49:0/3738558049 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced8004010 con 0x7fcee80ff6a0 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee819e800 con 0x7fcee80ff6a0 2023-10-08T09:13:17.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fcee4ff9700 1 -- 172.21.15.49:0/3738558049 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8022cd0 con 0x7fcee80ff6a0 2023-10-08T09:13:17.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.943+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee819ecb0 con 0x7fcee80ff6a0 2023-10-08T09:13:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.944+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee8067530 con 0x7fcee80ff6a0 2023-10-08T09:13:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.946+0000 7fcee4ff9700 1 -- 172.21.15.49:0/3738558049 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fced8022470 con 0x7fcee80ff6a0 2023-10-08T09:13:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.946+0000 7fcee4ff9700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fced00a1480 0x7fced00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.946+0000 7fcee6ffd700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fced00a1480 0x7fced00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.946+0000 7fcee4ff9700 1 -- 172.21.15.49:0/3738558049 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fced80ce9f0 con 0x7fcee80ff6a0 2023-10-08T09:13:17.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.947+0000 7fcee6ffd700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fced00a1480 0x7fced00a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcedc005950 tx=0x7fcedc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:17.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:17.949+0000 7fcee4ff9700 1 -- 172.21.15.49:0/3738558049 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fced8097ef0 con 0x7fcee80ff6a0 2023-10-08T09:13:18.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.167+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcee810aec0 con 0x7fcee80ff6a0 2023-10-08T09:13:18.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.167+0000 7fcee4ff9700 1 -- 172.21.15.49:0/3738558049 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+587 (secure 0 0 0) 0x7fced8027260 con 0x7fcee80ff6a0 2023-10-08T09:13:18.168 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2700533077 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:18.168 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/656654162 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:18.168 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/4176953235 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:18.168 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/907336252 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:18.168 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/890731290 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:18.169 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1281967666 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:18.169 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1812466550 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:18.169 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/907336252 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:18.169 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2211252163 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:18.169 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/890731290 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:18.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fced00a1480 msgr2=0x7fced00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fced00a1480 0x7fced00a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcedc005950 tx=0x7fcedc00b410 comp rx=0 tx=0).stop 2023-10-08T09:13:18.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 msgr2=0x7fcee8196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee8196780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fced8006010 tx=0x7fced8005740 comp rx=0 tx=0).stop 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 shutdown_connections 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fced00a1480 0x7fced00a3940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee8101230 0x7fcee819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcee80ff6a0 0x7fcee8196780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 --2- 172.21.15.49:0/3738558049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcee8100890 0x7fcee8196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 >> 172.21.15.49:0/3738558049 conn(0x7fcee80fac10 msgr2=0x7fcee80678e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 shutdown_connections 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.170+0000 7fceedcb7700 1 -- 172.21.15.49:0/3738558049 wait complete. 2023-10-08T09:13:18.171 INFO:teuthology.orchestra.run.smithi049.stderr:listed 10 entries 2023-10-08T09:13:18.183 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:13:18.183 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:13:18.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:18.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 -- 172.21.15.49:0/4052478573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100230 msgr2=0x7fbef0107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 --2- 172.21.15.49:0/4052478573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100230 0x7fbef0107af0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbee800b0a0 tx=0x7fbee800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 -- 172.21.15.49:0/4052478573 shutdown_connections 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 --2- 172.21.15.49:0/4052478573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0100b60 0x7fbef0100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 --2- 172.21.15.49:0/4052478573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100230 0x7fbef0107af0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 --2- 172.21.15.49:0/4052478573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef00ff890 0x7fbef00ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 -- 172.21.15.49:0/4052478573 >> 172.21.15.49:0/4052478573 conn(0x7fbef00fa2c0 msgr2=0x7fbef00fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 -- 172.21.15.49:0/4052478573 shutdown_connections 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.366+0000 7fbef59b9700 1 -- 172.21.15.49:0/4052478573 wait complete. 2023-10-08T09:13:18.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 Processor -- start 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 -- start start 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef00ff890 0x7fbef0196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0100230 0x7fbef0196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100b60 0x7fbef019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef019e240 con 0x7fbef0100b60 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef019e380 con 0x7fbef00ff890 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbef59b9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef019e510 con 0x7fbef0100230 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbeef7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100b60 0x7fbef019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbeef7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100b60 0x7fbef019dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48152/0 (socket says 172.21.15.49:48152) 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbeef7fe700 1 -- 172.21.15.49:0/600971311 learned_addr learned my addr 172.21.15.49:0/600971311 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbeee7fc700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0100230 0x7fbef0196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.367+0000 7fbeeeffd700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef00ff890 0x7fbef0196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbeef7fe700 1 -- 172.21.15.49:0/600971311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0100230 msgr2=0x7fbef0196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbeef7fe700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0100230 0x7fbef0196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbeef7fe700 1 -- 172.21.15.49:0/600971311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef00ff890 msgr2=0x7fbef0196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbeef7fe700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef00ff890 0x7fbef0196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbeef7fe700 1 -- 172.21.15.49:0/600971311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8009d00 con 0x7fbef0100b60 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbeef7fe700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100b60 0x7fbef019dc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbee80078c0 tx=0x7fbee80078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbef49b7700 1 -- 172.21.15.49:0/600971311 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee800e040 con 0x7fbef0100b60 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbef49b7700 1 -- 172.21.15.49:0/600971311 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbee800be40 con 0x7fbef0100b60 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef019e790 con 0x7fbef0100b60 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbef49b7700 1 -- 172.21.15.49:0/600971311 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee80052a0 con 0x7fbef0100b60 2023-10-08T09:13:18.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.368+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef019ebc0 con 0x7fbef0100b60 2023-10-08T09:13:18.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.369+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef00678a0 con 0x7fbef0100b60 2023-10-08T09:13:18.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.371+0000 7fbef49b7700 1 -- 172.21.15.49:0/600971311 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbee801d470 con 0x7fbef0100b60 2023-10-08T09:13:18.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.372+0000 7fbef49b7700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbed80a1480 0x7fbed80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.372+0000 7fbeeeffd700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbed80a1480 0x7fbed80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.372+0000 7fbef49b7700 1 -- 172.21.15.49:0/600971311 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbee80ce9e0 con 0x7fbef0100b60 2023-10-08T09:13:18.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.372+0000 7fbeeeffd700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbed80a1480 0x7fbed80a3940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbef00689d0 tx=0x7fbee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:18.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.375+0000 7fbef49b7700 1 -- 172.21.15.49:0/600971311 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbee8097ee0 con 0x7fbef0100b60 2023-10-08T09:13:18.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.579+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbef010add0 con 0x7fbef0100b60 2023-10-08T09:13:18.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.581+0000 7fbef49b7700 1 -- 172.21.15.49:0/600971311 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7fbee809b500 con 0x7fbef0100b60 2023-10-08T09:13:18.581 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:18.582 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:18.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbed80a1480 msgr2=0x7fbed80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbed80a1480 0x7fbed80a3940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbef00689d0 tx=0x7fbee0009260 comp rx=0 tx=0).stop 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100b60 msgr2=0x7fbef019dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100b60 0x7fbef019dc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbee80078c0 tx=0x7fbee80078f0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 shutdown_connections 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbed80a1480 0x7fbed80a3940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbef0100b60 0x7fbef019dc70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbef00ff890 0x7fbef0196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 --2- 172.21.15.49:0/600971311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbef0100230 0x7fbef0196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 >> 172.21.15.49:0/600971311 conn(0x7fbef00fa2c0 msgr2=0x7fbef00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 shutdown_connections 2023-10-08T09:13:18.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.583+0000 7fbef59b9700 1 -- 172.21.15.49:0/600971311 wait complete. 2023-10-08T09:13:18.594 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:18 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3728137142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:13:18.594 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:18 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/89545151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:13:18.595 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:18 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3738558049' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:13:18.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:13:18.596 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 -- 172.21.15.49:0/3494885807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc4101230 msgr2=0x7f4cc4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 --2- 172.21.15.49:0/3494885807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc4101230 0x7f4cc4107ae0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0099c0 tx=0x7f4cbc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 -- 172.21.15.49:0/3494885807 shutdown_connections 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 --2- 172.21.15.49:0/3494885807 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc4101230 0x7f4cc4107ae0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 --2- 172.21.15.49:0/3494885807 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc40ff6a0 0x7f4cc40ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 --2- 172.21.15.49:0/3494885807 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cc4100890 0x7f4cc4100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 -- 172.21.15.49:0/3494885807 >> 172.21.15.49:0/3494885807 conn(0x7f4cc40fac10 msgr2=0x7f4cc40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 -- 172.21.15.49:0/3494885807 shutdown_connections 2023-10-08T09:13:18.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 -- 172.21.15.49:0/3494885807 wait complete. 2023-10-08T09:13:18.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.725+0000 7f4cc94db700 1 Processor -- start 2023-10-08T09:13:18.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc94db700 1 -- start start 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc94db700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc40ff6a0 0x7f4cc4068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc94db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc4100890 0x7f4cc419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc94db700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cc4101230 0x7f4cc419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc94db700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc419e180 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc94db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc419e2f0 con 0x7f4cc4100890 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc94db700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc419e4b0 con 0x7f4cc4101230 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc40ff6a0 0x7f4cc4068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc40ff6a0 0x7f4cc4068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48164/0 (socket says 172.21.15.49:48164) 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 -- 172.21.15.49:0/4078079785 learned_addr learned my addr 172.21.15.49:0/4078079785 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:18.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc37fe700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cc4101230 0x7f4cc419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc27fc700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc4100890 0x7f4cc419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 -- 172.21.15.49:0/4078079785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cc4101230 msgr2=0x7f4cc419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cc4101230 0x7f4cc419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 -- 172.21.15.49:0/4078079785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc4100890 msgr2=0x7f4cc419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc4100890 0x7f4cc419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 -- 172.21.15.49:0/4078079785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc0095f0 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.726+0000 7f4cc2ffd700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc40ff6a0 0x7f4cc4068370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400cfd0 tx=0x7f4cb400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:18.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.727+0000 7f4cabfff700 1 -- 172.21.15.49:0/4078079785 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400ada0 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.727+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc419e730 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.727+0000 7f4cabfff700 1 -- 172.21.15.49:0/4078079785 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb4007e50 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.727+0000 7f4cabfff700 1 -- 172.21.15.49:0/4078079785 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4011670 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.727+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc419ec90 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.728+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc41069f0 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.729+0000 7f4cabfff700 1 -- 172.21.15.49:0/4078079785 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4cb4012420 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.729+0000 7f4cabfff700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cac0a1480 0x7f4cac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.729+0000 7f4cabfff700 1 -- 172.21.15.49:0/4078079785 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4cb40cd5a0 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.729+0000 7f4cc27fc700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cac0a1480 0x7f4cac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:18.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.730+0000 7f4cc27fc700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cac0a1480 0x7f4cac0a3940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005950 tx=0x7f4cb80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:18.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.733+0000 7f4cabfff700 1 -- 172.21.15.49:0/4078079785 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4cb4096a10 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:18 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3728137142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:13:18.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:18 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/89545151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:13:18.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:18 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3738558049' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:13:18.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:18 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3728137142' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:13:18.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:18 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/89545151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:13:18.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:18 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3738558049' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:13:18.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.939+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cc410b100 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.939+0000 7f4cabfff700 1 -- 172.21.15.49:0/4078079785 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cb409a030 con 0x7f4cc40ff6a0 2023-10-08T09:13:18.940 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cac0a1480 msgr2=0x7f4cac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cac0a1480 0x7f4cac0a3940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005950 tx=0x7f4cb80058e0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc40ff6a0 msgr2=0x7f4cc4068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc40ff6a0 0x7f4cc4068370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400cfd0 tx=0x7f4cb400a340 comp rx=0 tx=0).stop 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 shutdown_connections 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cac0a1480 0x7f4cac0a3940 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cc4101230 0x7f4cc419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cc40ff6a0 0x7f4cc4068370 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 --2- 172.21.15.49:0/4078079785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc4100890 0x7f4cc419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:18.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 >> 172.21.15.49:0/4078079785 conn(0x7f4cc40fac10 msgr2=0x7f4cc4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:18.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 shutdown_connections 2023-10-08T09:13:18.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:18.941+0000 7f4cc94db700 1 -- 172.21.15.49:0/4078079785 wait complete. 2023-10-08T09:13:18.954 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:13:19.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 -- 172.21.15.49:0/1142534896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100c50 msgr2=0x7f7274101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 --2- 172.21.15.49:0/1142534896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100c50 0x7f7274101070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7264009990 tx=0x7f7264009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 -- 172.21.15.49:0/1142534896 shutdown_connections 2023-10-08T09:13:19.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 --2- 172.21.15.49:0/1142534896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7274100260 0x7f7274107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 --2- 172.21.15.49:0/1142534896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100c50 0x7f7274101070 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 --2- 172.21.15.49:0/1142534896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72740ff8a0 0x7f72740ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 -- 172.21.15.49:0/1142534896 >> 172.21.15.49:0/1142534896 conn(0x7f72740fa390 msgr2=0x7f72740fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:19.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 -- 172.21.15.49:0/1142534896 shutdown_connections 2023-10-08T09:13:19.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.082+0000 7f727a86d700 1 -- 172.21.15.49:0/1142534896 wait complete. 2023-10-08T09:13:19.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 Processor -- start 2023-10-08T09:13:19.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 -- start start 2023-10-08T09:13:19.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72740ff8a0 0x7f72741037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100260 0x7f7274101e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7274100c50 0x7f7274102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7274102b10 con 0x7f7274100260 2023-10-08T09:13:19.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7274102c80 con 0x7f72740ff8a0 2023-10-08T09:13:19.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f727a86d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72741a0780 con 0x7f7274100c50 2023-10-08T09:13:19.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f72737fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100260 0x7f7274101e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.091 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f72737fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100260 0x7f7274101e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48180/0 (socket says 172.21.15.49:48180) 2023-10-08T09:13:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.083+0000 7f72737fe700 1 -- 172.21.15.49:0/3602440011 learned_addr learned my addr 172.21.15.49:0/3602440011 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f7273fff700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72740ff8a0 0x7f72741037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72737fe700 1 -- 172.21.15.49:0/3602440011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7274100c50 msgr2=0x7f7274102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f7278e0a700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7274100c50 0x7f7274102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72737fe700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7274100c50 0x7f7274102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72737fe700 1 -- 172.21.15.49:0/3602440011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72740ff8a0 msgr2=0x7f72741037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72737fe700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72740ff8a0 0x7f72741037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72737fe700 1 -- 172.21.15.49:0/3602440011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640095f0 con 0x7f7274100260 2023-10-08T09:13:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f7273fff700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72740ff8a0 0x7f72741037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72737fe700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100260 0x7f7274101e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c920 tx=0x7f725c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:19.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72717fa700 1 -- 172.21.15.49:0/3602440011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c007aa0 con 0x7f7274100260 2023-10-08T09:13:19.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72741a0a10 con 0x7f7274100260 2023-10-08T09:13:19.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72717fa700 1 -- 172.21.15.49:0/3602440011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f725c004cf0 con 0x7f7274100260 2023-10-08T09:13:19.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f72717fa700 1 -- 172.21.15.49:0/3602440011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0054b0 con 0x7f7274100260 2023-10-08T09:13:19.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.084+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72741a0f70 con 0x7f7274100260 2023-10-08T09:13:19.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.085+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7274066ca0 con 0x7f7274100260 2023-10-08T09:13:19.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.086+0000 7f72717fa700 1 -- 172.21.15.49:0/3602440011 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f725c004750 con 0x7f7274100260 2023-10-08T09:13:19.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.087+0000 7f72717fa700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72600a1480 0x7f72600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.087+0000 7f7273fff700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72600a1480 0x7f72600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.087+0000 7f72717fa700 1 -- 172.21.15.49:0/3602440011 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f725c0ccf70 con 0x7f7274100260 2023-10-08T09:13:19.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.087+0000 7f7273fff700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72600a1480 0x7f72600a3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7264006010 tx=0x7f72640037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:19.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.090+0000 7f72717fa700 1 -- 172.21.15.49:0/3602440011 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f725c096380 con 0x7f7274100260 2023-10-08T09:13:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.293+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72741a0ba0 con 0x7f7274100260 2023-10-08T09:13:19.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.293+0000 7f72717fa700 1 -- 172.21.15.49:0/3602440011 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72741a0ba0 con 0x7f7274100260 2023-10-08T09:13:19.294 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72600a1480 msgr2=0x7f72600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72600a1480 0x7f72600a3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7264006010 tx=0x7f72640037b0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100260 msgr2=0x7f7274101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100260 0x7f7274101e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c920 tx=0x7f725c00cce0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 shutdown_connections 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72600a1480 0x7f72600a3940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7274100c50 0x7f7274102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72740ff8a0 0x7f72741037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 --2- 172.21.15.49:0/3602440011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7274100260 0x7f7274101e30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 >> 172.21.15.49:0/3602440011 conn(0x7f72740fa390 msgr2=0x7f72740fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:19.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 shutdown_connections 2023-10-08T09:13:19.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.295+0000 7f727a86d700 1 -- 172.21.15.49:0/3602440011 wait complete. 2023-10-08T09:13:19.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:13:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/2511511514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e80ff6a0 msgr2=0x7fd4e80ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/2511511514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e80ff6a0 0x7fd4e80ffac0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0099c0 tx=0x7fd4dc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/2511511514 shutdown_connections 2023-10-08T09:13:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/2511511514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e8101230 0x7fd4e8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/2511511514 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e80ff6a0 0x7fd4e80ffac0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/2511511514 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e8100890 0x7fd4e8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/2511511514 >> 172.21.15.49:0/2511511514 conn(0x7fd4e80fac10 msgr2=0x7fd4e80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/2511511514 shutdown_connections 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.436+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/2511511514 wait complete. 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 Processor -- start 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 -- start start 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80ff6a0 0x7fd4e8196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e8100890 0x7fd4e8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e8101230 0x7fd4e819dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e819e2c0 con 0x7fd4e8101230 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e819e400 con 0x7fd4e80ff6a0 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4ecd4e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e819e590 con 0x7fd4e8100890 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4e6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e8101230 0x7fd4e819dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4e5d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e8100890 0x7fd4e8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4e6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e8101230 0x7fd4e819dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48196/0 (socket says 172.21.15.49:48196) 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4e6d9d700 1 -- 172.21.15.49:0/903762404 learned_addr learned my addr 172.21.15.49:0/903762404 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.437+0000 7fd4e659c700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80ff6a0 0x7fd4e8196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4e6d9d700 1 -- 172.21.15.49:0/903762404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e8100890 msgr2=0x7fd4e8196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4e6d9d700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e8100890 0x7fd4e8196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4e6d9d700 1 -- 172.21.15.49:0/903762404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80ff6a0 msgr2=0x7fd4e8196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4e6d9d700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80ff6a0 0x7fd4e8196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4e6d9d700 1 -- 172.21.15.49:0/903762404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0095f0 con 0x7fd4e8101230 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4e6d9d700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e8101230 0x7fd4e819dcf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000ada0 tx=0x7fd4e000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4d77fe700 1 -- 172.21.15.49:0/903762404 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000f070 con 0x7fd4e8101230 2023-10-08T09:13:19.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e819e870 con 0x7fd4e8101230 2023-10-08T09:13:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4d77fe700 1 -- 172.21.15.49:0/903762404 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4e0007b30 con 0x7fd4e8101230 2023-10-08T09:13:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4d77fe700 1 -- 172.21.15.49:0/903762404 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005280 con 0x7fd4e8101230 2023-10-08T09:13:19.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.438+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e819ed30 con 0x7fd4e8101230 2023-10-08T09:13:19.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.439+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e8067530 con 0x7fd4e8101230 2023-10-08T09:13:19.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.440+0000 7fd4d77fe700 1 -- 172.21.15.49:0/903762404 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd4e000a090 con 0x7fd4e8101230 2023-10-08T09:13:19.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.441+0000 7fd4d77fe700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4d00a1480 0x7fd4d00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.441+0000 7fd4e659c700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4d00a1480 0x7fd4d00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.441+0000 7fd4d77fe700 1 -- 172.21.15.49:0/903762404 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd4e00cc950 con 0x7fd4e8101230 2023-10-08T09:13:19.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.442+0000 7fd4e659c700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4d00a1480 0x7fd4d00a3940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009f50 tx=0x7fd4dc005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:19.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.445+0000 7fd4d77fe700 1 -- 172.21.15.49:0/903762404 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd4e0095e50 con 0x7fd4e8101230 2023-10-08T09:13:19.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:19 smithi049 ceph-mon[99485]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2023-10-08T09:13:19.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:19 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/600971311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:19.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:19 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4078079785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:19.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:19 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3602440011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:13:19.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.649+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4e810b090 con 0x7fd4e8101230 2023-10-08T09:13:19.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.650+0000 7fd4d77fe700 1 -- 172.21.15.49:0/903762404 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4e0099470 con 0x7fd4e8101230 2023-10-08T09:13:19.650 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:13:19.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4d00a1480 msgr2=0x7fd4d00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4d00a1480 0x7fd4d00a3940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009f50 tx=0x7fd4dc005ea0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e8101230 msgr2=0x7fd4e819dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e8101230 0x7fd4e819dcf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000ada0 tx=0x7fd4e000c330 comp rx=0 tx=0).stop 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 shutdown_connections 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4d00a1480 0x7fd4d00a3940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4e8101230 0x7fd4e819dcf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4e80ff6a0 0x7fd4e8196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 --2- 172.21.15.49:0/903762404 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4e8100890 0x7fd4e8196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 >> 172.21.15.49:0/903762404 conn(0x7fd4e80fac10 msgr2=0x7fd4e8067860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:19.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 shutdown_connections 2023-10-08T09:13:19.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.652+0000 7fd4ecd4e700 1 -- 172.21.15.49:0/903762404 wait complete. 2023-10-08T09:13:19.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:13:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:19 smithi184 ceph-mon[103675]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2023-10-08T09:13:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:19 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/600971311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:19 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4078079785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:19 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3602440011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:13:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 -- 172.21.15.49:0/1908687018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe420102350 msgr2=0x7fe420108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 --2- 172.21.15.49:0/1908687018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe420102350 0x7fe420108c00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe41800b0d0 tx=0x7fe41800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 -- 172.21.15.49:0/1908687018 shutdown_connections 2023-10-08T09:13:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 --2- 172.21.15.49:0/1908687018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4201007c0 0x7fe420100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 --2- 172.21.15.49:0/1908687018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe420102350 0x7fe420108c00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 --2- 172.21.15.49:0/1908687018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4201019b0 0x7fe420101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 -- 172.21.15.49:0/1908687018 >> 172.21.15.49:0/1908687018 conn(0x7fe4200fbd30 msgr2=0x7fe4200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:19.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 -- 172.21.15.49:0/1908687018 shutdown_connections 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.793+0000 7fe42494a700 1 -- 172.21.15.49:0/1908687018 wait complete. 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 Processor -- start 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 -- start start 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4201007c0 0x7fe420074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4201019b0 0x7fe420072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe420102350 0x7fe420073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200738c0 con 0x7fe4201019b0 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420073a30 con 0x7fe4201007c0 2023-10-08T09:13:19.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe42494a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201a0850 con 0x7fe420102350 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe41dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4201019b0 0x7fe420072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe41dd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4201019b0 0x7fe420072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48208/0 (socket says 172.21.15.49:48208) 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe41dd9b700 1 -- 172.21.15.49:0/4066708349 learned_addr learned my addr 172.21.15.49:0/4066708349 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe41e59c700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4201007c0 0x7fe420074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe41ed9d700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe420102350 0x7fe420073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.794+0000 7fe41dd9b700 1 -- 172.21.15.49:0/4066708349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe420102350 msgr2=0x7fe420073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe41dd9b700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe420102350 0x7fe420073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe41dd9b700 1 -- 172.21.15.49:0/4066708349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4201007c0 msgr2=0x7fe420074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe41dd9b700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4201007c0 0x7fe420074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:19.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe41dd9b700 1 -- 172.21.15.49:0/4066708349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe418009d00 con 0x7fe4201019b0 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe41ed9d700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe420102350 0x7fe420073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe41dd9b700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4201019b0 0x7fe420072be0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe41000d950 tx=0x7fe41000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe40f7fe700 1 -- 172.21.15.49:0/4066708349 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100098e0 con 0x7fe4201019b0 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe40f7fe700 1 -- 172.21.15.49:0/4066708349 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe410010460 con 0x7fe4201019b0 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4201a0ae0 con 0x7fe4201019b0 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe40f7fe700 1 -- 172.21.15.49:0/4066708349 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000f620 con 0x7fe4201019b0 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.795+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4201a0ff0 con 0x7fe4201019b0 2023-10-08T09:13:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.796+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe420107b70 con 0x7fe4201019b0 2023-10-08T09:13:19.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:19 smithi132 ceph-mon[103389]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2023-10-08T09:13:19.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:19 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/600971311' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:19.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:19 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4078079785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:13:19.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:19 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3602440011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:13:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.797+0000 7fe40f7fe700 1 -- 172.21.15.49:0/4066708349 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe4100108d0 con 0x7fe4201019b0 2023-10-08T09:13:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.798+0000 7fe40f7fe700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4080a1480 0x7fe4080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.798+0000 7fe41e59c700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4080a1480 0x7fe4080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.798+0000 7fe40f7fe700 1 -- 172.21.15.49:0/4066708349 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe4100ccd00 con 0x7fe4201019b0 2023-10-08T09:13:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.798+0000 7fe41e59c700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4080a1480 0x7fe4080a3940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe420101810 tx=0x7fe41400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:19.801+0000 7fe40f7fe700 1 -- 172.21.15.49:0/4066708349 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe410096170 con 0x7fe4201019b0 2023-10-08T09:13:20.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.003+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe42010bfe0 con 0x7fe4201019b0 2023-10-08T09:13:20.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.004+0000 7fe40f7fe700 1 -- 172.21.15.49:0/4066708349 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe4100160a0 con 0x7fe4201019b0 2023-10-08T09:13:20.004 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:13:20.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4080a1480 msgr2=0x7fe4080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:20.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4080a1480 0x7fe4080a3940 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe420101810 tx=0x7fe41400b410 comp rx=0 tx=0).stop 2023-10-08T09:13:20.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4201019b0 msgr2=0x7fe420072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4201019b0 0x7fe420072be0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe41000d950 tx=0x7fe41000dc60 comp rx=0 tx=0).stop 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 shutdown_connections 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4080a1480 0x7fe4080a3940 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe420102350 0x7fe420073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4201007c0 0x7fe420074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 --2- 172.21.15.49:0/4066708349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4201019b0 0x7fe420072be0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 >> 172.21.15.49:0/4066708349 conn(0x7fe4200fbd30 msgr2=0x7fe420104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 shutdown_connections 2023-10-08T09:13:20.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.006+0000 7fe42494a700 1 -- 172.21.15.49:0/4066708349 wait complete. 2023-10-08T09:13:20.019 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T09:13:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 -- 172.21.15.49:0/1013854774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c102350 msgr2=0x7efc7c108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 --2- 172.21.15.49:0/1013854774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c102350 0x7efc7c108c00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc7400b0d0 tx=0x7efc7400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 -- 172.21.15.49:0/1013854774 shutdown_connections 2023-10-08T09:13:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 --2- 172.21.15.49:0/1013854774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c102350 0x7efc7c108c00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 --2- 172.21.15.49:0/1013854774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc7c1007c0 0x7efc7c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 --2- 172.21.15.49:0/1013854774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c1019b0 0x7efc7c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 -- 172.21.15.49:0/1013854774 >> 172.21.15.49:0/1013854774 conn(0x7efc7c0fbd30 msgr2=0x7efc7c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 -- 172.21.15.49:0/1013854774 shutdown_connections 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.144+0000 7efc8225c700 1 -- 172.21.15.49:0/1013854774 wait complete. 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.145+0000 7efc8225c700 1 Processor -- start 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.145+0000 7efc8225c700 1 -- start start 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c1007c0 0x7efc7c1967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c1019b0 0x7efc7c196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc7c102350 0x7efc7c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:20.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c19e320 con 0x7efc7c1007c0 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c19e460 con 0x7efc7c102350 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c19e5f0 con 0x7efc7c1019b0 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c1007c0 0x7efc7c1967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c1007c0 0x7efc7c1967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48220/0 (socket says 172.21.15.49:48220) 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 -- 172.21.15.49:0/787685622 learned_addr learned my addr 172.21.15.49:0/787685622 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7bfff700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc7c102350 0x7efc7c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7affd700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c1019b0 0x7efc7c196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:20.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 -- 172.21.15.49:0/787685622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c1019b0 msgr2=0x7efc7c196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c1019b0 0x7efc7c196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 -- 172.21.15.49:0/787685622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc7c102350 msgr2=0x7efc7c19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc7c102350 0x7efc7c19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 -- 172.21.15.49:0/787685622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc74009d00 con 0x7efc7c1007c0 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7bfff700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc7c102350 0x7efc7c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7affd700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c1019b0 0x7efc7c196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc7b7fe700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c1007c0 0x7efc7c1967f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00d980 tx=0x7efc6c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:20.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc78ff9700 1 -- 172.21.15.49:0/787685622 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00de60 con 0x7efc7c1007c0 2023-10-08T09:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc78ff9700 1 -- 172.21.15.49:0/787685622 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6c0044e0 con 0x7efc7c1007c0 2023-10-08T09:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7c19e8d0 con 0x7efc7c1007c0 2023-10-08T09:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc78ff9700 1 -- 172.21.15.49:0/787685622 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c003be0 con 0x7efc7c1007c0 2023-10-08T09:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.146+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c19ee30 con 0x7efc7c1007c0 2023-10-08T09:13:20.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.147+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7c107b70 con 0x7efc7c1007c0 2023-10-08T09:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.149+0000 7efc78ff9700 1 -- 172.21.15.49:0/787685622 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efc6c009bb0 con 0x7efc7c1007c0 2023-10-08T09:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.150+0000 7efc78ff9700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc640a1500 0x7efc640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.150+0000 7efc7affd700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc640a1500 0x7efc640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.150+0000 7efc78ff9700 1 -- 172.21.15.49:0/787685622 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc6c0ccf50 con 0x7efc7c1007c0 2023-10-08T09:13:20.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.150+0000 7efc7affd700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc640a1500 0x7efc640a39c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efc70005fd0 tx=0x7efc70005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:20.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.153+0000 7efc78ff9700 1 -- 172.21.15.49:0/787685622 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efc6c096440 con 0x7efc7c1007c0 2023-10-08T09:13:20.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.373+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efc7c067530 con 0x7efc7c1007c0 2023-10-08T09:13:20.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.374+0000 7efc78ff9700 1 -- 172.21.15.49:0/787685622 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) v1 ==== 81+0+587 (secure 0 0 0) 0x7efc6c017260 con 0x7efc7c1007c0 2023-10-08T09:13:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2700533077 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/656654162 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:20.374 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/4176953235 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/907336252 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/890731290 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1281967666 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1812466550 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/907336252 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2211252163 2023-10-09T09:06:08.179468+0000 2023-10-08T09:13:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/890731290 2023-10-09T09:06:35.771578+0000 2023-10-08T09:13:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.375+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc640a1500 msgr2=0x7efc640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.375+0000 7efc8225c700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc640a1500 0x7efc640a39c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efc70005fd0 tx=0x7efc70005ee0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.375+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c1007c0 msgr2=0x7efc7c1967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.375+0000 7efc8225c700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c1007c0 0x7efc7c1967f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00d980 tx=0x7efc6c00dd40 comp rx=0 tx=0).stop 2023-10-08T09:13:20.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 shutdown_connections 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efc640a1500 0x7efc640a39c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc7c102350 0x7efc7c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc7c1007c0 0x7efc7c1967f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 --2- 172.21.15.49:0/787685622 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc7c1019b0 0x7efc7c196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 >> 172.21.15.49:0/787685622 conn(0x7efc7c0fbd30 msgr2=0x7efc7c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 shutdown_connections 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:20.376+0000 7efc8225c700 1 -- 172.21.15.49:0/787685622 wait complete. 2023-10-08T09:13:20.377 INFO:teuthology.orchestra.run.smithi049.stderr:listed 10 entries 2023-10-08T09:13:20.388 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi049.front.sepia.ceph.com... 2023-10-08T09:13:20.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:20.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-08T09:13:20.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi049.stderr:ceph-fuse[157776]: fuse finished with error 0 and tester_r 0 2023-10-08T09:13:20.425 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T09:13:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:20 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/903762404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:13:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:20 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4066708349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:13:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:20 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/787685622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:13:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:20 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/903762404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:13:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:20 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4066708349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:13:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:20 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/787685622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:13:20.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:20 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/903762404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:13:20.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:20 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4066708349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:13:20.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:20 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/787685622' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:13:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:21 smithi184 ceph-mon[103675]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 279 B/s wr, 0 op/s 2023-10-08T09:13:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:21 smithi132 ceph-mon[103389]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 279 B/s wr, 0 op/s 2023-10-08T09:13:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:21 smithi049 ceph-mon[99485]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 279 B/s wr, 0 op/s 2023-10-08T09:13:23.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:23 smithi184 ceph-mon[103675]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:23.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:23 smithi132 ceph-mon[103389]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:23.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:23 smithi049 ceph-mon[99485]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:25.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:25 smithi184 ceph-mon[103675]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:25.807 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:25 smithi132 ceph-mon[103389]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:25.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:25 smithi049 ceph-mon[99485]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-08T09:13:26.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:13:26.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:13:26.527 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-08T09:13:26.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 -- 172.21.15.49:0/1081351853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 msgr2=0x7f8700101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 --2- 172.21.15.49:0/1081351853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700101d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f86e80099c0 tx=0x7f86e8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 -- 172.21.15.49:0/1081351853 shutdown_connections 2023-10-08T09:13:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 --2- 172.21.15.49:0/1081351853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8700102250 0x7f8700108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 --2- 172.21.15.49:0/1081351853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f87001006c0 0x7f8700100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 --2- 172.21.15.49:0/1081351853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700101d10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 -- 172.21.15.49:0/1081351853 >> 172.21.15.49:0/1081351853 conn(0x7f87000fbc50 msgr2=0x7f87000fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 -- 172.21.15.49:0/1081351853 shutdown_connections 2023-10-08T09:13:26.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.748+0000 7f8704ea7700 1 -- 172.21.15.49:0/1081351853 wait complete. 2023-10-08T09:13:26.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.749+0000 7f8704ea7700 1 Processor -- start 2023-10-08T09:13:26.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.749+0000 7f8704ea7700 1 -- start start 2023-10-08T09:13:26.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.749+0000 7f8704ea7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87001006c0 0x7f8700196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:26.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.749+0000 7f8704ea7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:26.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.749+0000 7f8704ea7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700102250 0x7f870019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:26.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f8704ea7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019e190 con 0x7f87001018b0 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f8704ea7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019e2d0 con 0x7f8700102250 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f8704ea7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019e460 con 0x7f87001006c0 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48236/0 (socket says 172.21.15.49:48236) 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 -- 172.21.15.49:0/2202126901 learned_addr learned my addr 172.21.15.49:0/2202126901 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fe59c700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87001006c0 0x7f8700196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fed9d700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700102250 0x7f870019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:26.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 -- 172.21.15.49:0/2202126901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87001006c0 msgr2=0x7f8700196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87001006c0 0x7f8700196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 -- 172.21.15.49:0/2202126901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700102250 msgr2=0x7f870019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700102250 0x7f870019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 -- 172.21.15.49:0/2202126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80095f0 con 0x7f87001018b0 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86fdd9b700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700196ba0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b580 tx=0x7f86e8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86f77fe700 1 -- 172.21.15.49:0/2202126901 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e801d070 con 0x7f87001018b0 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86f77fe700 1 -- 172.21.15.49:0/2202126901 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e8003f90 con 0x7f87001018b0 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870019e6e0 con 0x7f87001018b0 2023-10-08T09:13:26.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f86f77fe700 1 -- 172.21.15.49:0/2202126901 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8022c40 con 0x7f87001018b0 2023-10-08T09:13:26.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.750+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870019eb90 con 0x7f87001018b0 2023-10-08T09:13:26.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.751+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8700107ae0 con 0x7f87001018b0 2023-10-08T09:13:26.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.752+0000 7f86f77fe700 1 -- 172.21.15.49:0/2202126901 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f86e8022470 con 0x7f87001018b0 2023-10-08T09:13:26.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.753+0000 7f86f77fe700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86ec0a1480 0x7f86ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:26.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.753+0000 7f86f77fe700 1 -- 172.21.15.49:0/2202126901 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f86e80cea40 con 0x7f87001018b0 2023-10-08T09:13:26.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.753+0000 7f86fe59c700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86ec0a1480 0x7f86ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:26.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.754+0000 7f86fe59c700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86ec0a1480 0x7f86ec0a3940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8700101710 tx=0x7f86f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:26.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.757+0000 7f86f77fe700 1 -- 172.21.15.49:0/2202126901 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f86e8097f40 con 0x7f87001018b0 2023-10-08T09:13:26.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.966+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f870010bf50 con 0x7f87001018b0 2023-10-08T09:13:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.968+0000 7f86f77fe700 1 -- 172.21.15.49:0/2202126901 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7f86e809b560 con 0x7f87001018b0 2023-10-08T09:13:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:26.970 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:26.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86ec0a1480 msgr2=0x7f86ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:26.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86ec0a1480 0x7f86ec0a3940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8700101710 tx=0x7f86f000b410 comp rx=0 tx=0).stop 2023-10-08T09:13:26.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 msgr2=0x7f8700196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700196ba0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f86e800b580 tx=0x7f86e8009580 comp rx=0 tx=0).stop 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 shutdown_connections 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86ec0a1480 0x7f86ec0a3940 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87001006c0 0x7f8700196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8700102250 0x7f870019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 --2- 172.21.15.49:0/2202126901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87001018b0 0x7f8700196ba0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 >> 172.21.15.49:0/2202126901 conn(0x7f87000fbc50 msgr2=0x7f8700104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 shutdown_connections 2023-10-08T09:13:26.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:26.970+0000 7f8704ea7700 1 -- 172.21.15.49:0/2202126901 wait complete. 2023-10-08T09:13:26.984 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:27.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 -- 172.21.15.49:0/4092887816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3800ff5a0 msgr2=0x7fd3800ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.124 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 --2- 172.21.15.49:0/4092887816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3800ff5a0 0x7fd3800ff9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd3700099c0 tx=0x7fd370009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 -- 172.21.15.49:0/4092887816 shutdown_connections 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 --2- 172.21.15.49:0/4092887816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd380101130 0x7fd3801079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 --2- 172.21.15.49:0/4092887816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3800ff5a0 0x7fd3800ff9c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 --2- 172.21.15.49:0/4092887816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd380100790 0x7fd380100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 -- 172.21.15.49:0/4092887816 >> 172.21.15.49:0/4092887816 conn(0x7fd3800fab10 msgr2=0x7fd3800fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 -- 172.21.15.49:0/4092887816 shutdown_connections 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.120+0000 7fd387ac1700 1 -- 172.21.15.49:0/4092887816 wait complete. 2023-10-08T09:13:27.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 Processor -- start 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 -- start start 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3800ff5a0 0x7fd380068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd380100790 0x7fd38019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd380101130 0x7fd38019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38019e160 con 0x7fd380100790 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38019e2d0 con 0x7fd380101130 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd387ac1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38019e490 con 0x7fd3800ff5a0 2023-10-08T09:13:27.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd380100790 0x7fd38019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd380100790 0x7fd38019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48250/0 (socket says 172.21.15.49:48250) 2023-10-08T09:13:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 -- 172.21.15.49:0/1251394644 learned_addr learned my addr 172.21.15.49:0/1251394644 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38605e700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd380101130 0x7fd38019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38585d700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3800ff5a0 0x7fd380068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 -- 172.21.15.49:0/1251394644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3800ff5a0 msgr2=0x7fd380068350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3800ff5a0 0x7fd380068350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 -- 172.21.15.49:0/1251394644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd380101130 msgr2=0x7fd38019d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd380101130 0x7fd38019d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 -- 172.21.15.49:0/1251394644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3700095f0 con 0x7fd380100790 2023-10-08T09:13:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.121+0000 7fd38505c700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd380100790 0x7fd38019b3d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9a0 tx=0x7fd37c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.122+0000 7fd376ffd700 1 -- 172.21.15.49:0/1251394644 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c009790 con 0x7fd380100790 2023-10-08T09:13:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.122+0000 7fd376ffd700 1 -- 172.21.15.49:0/1251394644 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd37c00cbd0 con 0x7fd380100790 2023-10-08T09:13:27.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.122+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38019e770 con 0x7fd380100790 2023-10-08T09:13:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.122+0000 7fd376ffd700 1 -- 172.21.15.49:0/1251394644 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c018680 con 0x7fd380100790 2023-10-08T09:13:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.122+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38019ecd0 con 0x7fd380100790 2023-10-08T09:13:27.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.123+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd380106a70 con 0x7fd380100790 2023-10-08T09:13:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.124+0000 7fd376ffd700 1 -- 172.21.15.49:0/1251394644 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd37c00cd40 con 0x7fd380100790 2023-10-08T09:13:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.125+0000 7fd376ffd700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd36c0a1500 0x7fd36c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.125+0000 7fd38585d700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd36c0a1500 0x7fd36c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.125+0000 7fd376ffd700 1 -- 172.21.15.49:0/1251394644 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd37c014070 con 0x7fd380100790 2023-10-08T09:13:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.125+0000 7fd38585d700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd36c0a1500 0x7fd36c0a39c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd370009990 tx=0x7fd37001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:27.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.128+0000 7fd376ffd700 1 -- 172.21.15.49:0/1251394644 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd37c096890 con 0x7fd380100790 2023-10-08T09:13:27.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.333+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd380067450 con 0x7fd380100790 2023-10-08T09:13:27.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.334+0000 7fd376ffd700 1 -- 172.21.15.49:0/1251394644 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7fd37c026090 con 0x7fd380100790 2023-10-08T09:13:27.336 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:27.336 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:27.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.336+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd36c0a1500 msgr2=0x7fd36c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.336+0000 7fd387ac1700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd36c0a1500 0x7fd36c0a39c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd370009990 tx=0x7fd37001a040 comp rx=0 tx=0).stop 2023-10-08T09:13:27.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.336+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd380100790 msgr2=0x7fd38019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.336+0000 7fd387ac1700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd380100790 0x7fd38019b3d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9a0 tx=0x7fd37c00ecb0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 shutdown_connections 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd36c0a1500 0x7fd36c0a39c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3800ff5a0 0x7fd380068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd380101130 0x7fd38019d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 --2- 172.21.15.49:0/1251394644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd380100790 0x7fd38019b3d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 >> 172.21.15.49:0/1251394644 conn(0x7fd3800fab10 msgr2=0x7fd380103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 shutdown_connections 2023-10-08T09:13:27.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.337+0000 7fd387ac1700 1 -- 172.21.15.49:0/1251394644 wait complete. 2023-10-08T09:13:27.351 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T09:13:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 -- 172.21.15.49:0/483143002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04103b60 msgr2=0x7f4c04105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 --2- 172.21.15.49:0/483143002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04103b60 0x7f4c04105f50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009990 tx=0x7f4bf4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 -- 172.21.15.49:0/483143002 shutdown_connections 2023-10-08T09:13:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 --2- 172.21.15.49:0/483143002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c04106490 0x7f4c04108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 --2- 172.21.15.49:0/483143002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c04101230 0x7f4c04103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 --2- 172.21.15.49:0/483143002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04103b60 0x7f4c04105f50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 -- 172.21.15.49:0/483143002 >> 172.21.15.49:0/483143002 conn(0x7f4c040fac40 msgr2=0x7f4c040fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 -- 172.21.15.49:0/483143002 shutdown_connections 2023-10-08T09:13:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.479+0000 7f4c09607700 1 -- 172.21.15.49:0/483143002 wait complete. 2023-10-08T09:13:27.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 Processor -- start 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 -- start start 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04101230 0x7f4c04196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c04103b60 0x7f4c04196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c04106490 0x7f4c0419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419e1b0 con 0x7f4c04101230 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419e2f0 con 0x7f4c04103b60 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.480+0000 7f4c09607700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419e480 con 0x7f4c04106490 2023-10-08T09:13:27.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04101230 0x7f4c04196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04101230 0x7f4c04196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48262/0 (socket says 172.21.15.49:48262) 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 -- 172.21.15.49:0/2411387442 learned_addr learned my addr 172.21.15.49:0/2411387442 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c027fc700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c04103b60 0x7f4c04196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c037fe700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c04106490 0x7f4c0419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 -- 172.21.15.49:0/2411387442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c04106490 msgr2=0x7f4c0419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c04106490 0x7f4c0419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 -- 172.21.15.49:0/2411387442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c04103b60 msgr2=0x7f4c04196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c04103b60 0x7f4c04196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 -- 172.21.15.49:0/2411387442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40095f0 con 0x7f4c04101230 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c02ffd700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04101230 0x7f4c04196680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c9b0 tx=0x7f4bec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:27.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4bfbfff700 1 -- 172.21.15.49:0/2411387442 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec007b00 con 0x7f4c04101230 2023-10-08T09:13:27.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0419e760 con 0x7f4c04101230 2023-10-08T09:13:27.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4bfbfff700 1 -- 172.21.15.49:0/2411387442 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bec00ce50 con 0x7f4c04101230 2023-10-08T09:13:27.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4bfbfff700 1 -- 172.21.15.49:0/2411387442 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec018660 con 0x7f4c04101230 2023-10-08T09:13:27.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.481+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0419ecc0 con 0x7f4c04101230 2023-10-08T09:13:27.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.482+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c040fc7c0 con 0x7f4c04101230 2023-10-08T09:13:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.484+0000 7f4bfbfff700 1 -- 172.21.15.49:0/2411387442 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4bec00f450 con 0x7f4c04101230 2023-10-08T09:13:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.484+0000 7f4bfbfff700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bf00a1480 0x7f4bf00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.484+0000 7f4c027fc700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bf00a1480 0x7f4bf00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.484+0000 7f4bfbfff700 1 -- 172.21.15.49:0/2411387442 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4bec0cd070 con 0x7f4c04101230 2023-10-08T09:13:27.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.485+0000 7f4c027fc700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bf00a1480 0x7f4bf00a3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400b5c0 tx=0x7f4bf4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:27.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.488+0000 7f4bfbfff700 1 -- 172.21.15.49:0/2411387442 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4bec096570 con 0x7f4c04101230 2023-10-08T09:13:27.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:27 smithi049 ceph-mon[99485]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:27.645 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:27 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2202126901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:27.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:27 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1251394644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:27.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:27.739+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4c04066ca0 con 0x7f4c04101230 2023-10-08T09:13:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:27 smithi184 ceph-mon[103675]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:27 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2202126901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:27.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:27 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1251394644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:27 smithi132 ceph-mon[103389]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:27 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2202126901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:27 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1251394644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:28.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.435+0000 7f4bfbfff700 1 -- 172.21.15.49:0/2411387442 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4bec007e60 con 0x7f4c04101230 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bf00a1480 msgr2=0x7f4bf00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bf00a1480 0x7f4bf00a3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400b5c0 tx=0x7f4bf4005fd0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04101230 msgr2=0x7f4c04196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04101230 0x7f4c04196680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c9b0 tx=0x7f4bec00ccc0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 shutdown_connections 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bf00a1480 0x7f4bf00a3940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c04106490 0x7f4c0419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c04101230 0x7f4c04196680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 --2- 172.21.15.49:0/2411387442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c04103b60 0x7f4c04196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 >> 172.21.15.49:0/2411387442 conn(0x7f4c040fac40 msgr2=0x7f4c04101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 shutdown_connections 2023-10-08T09:13:28.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.438+0000 7f4c09607700 1 -- 172.21.15.49:0/2411387442 wait complete. 2023-10-08T09:13:28.455 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-10-08T09:13:28.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-08T09:13:28.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 -- 172.21.15.49:0/111424362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff034101240 msgr2=0x7ff034107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:28.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 --2- 172.21.15.49:0/111424362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff034101240 0x7ff034107af0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00b0d0 tx=0x7ff02c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 -- 172.21.15.49:0/111424362 shutdown_connections 2023-10-08T09:13:28.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 --2- 172.21.15.49:0/111424362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0340ff6b0 0x7ff0340ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 --2- 172.21.15.49:0/111424362 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff034101240 0x7ff034107af0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 --2- 172.21.15.49:0/111424362 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff0341008a0 0x7ff034100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 -- 172.21.15.49:0/111424362 >> 172.21.15.49:0/111424362 conn(0x7ff0340fac00 msgr2=0x7ff0340fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 -- 172.21.15.49:0/111424362 shutdown_connections 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.590+0000 7ff038b4d700 1 -- 172.21.15.49:0/111424362 wait complete. 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 Processor -- start 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 -- start start 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0340ff6b0 0x7ff034069870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0341008a0 0x7ff03419b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff034101240 0x7ff03419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419e310 con 0x7ff0341008a0 2023-10-08T09:13:28.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419e480 con 0x7ff034101240 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.591+0000 7ff038b4d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03419e640 con 0x7ff0340ff6b0 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0341008a0 0x7ff03419b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0341008a0 0x7ff03419b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48280/0 (socket says 172.21.15.49:48280) 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 -- 172.21.15.49:0/940974453 learned_addr learned my addr 172.21.15.49:0/940974453 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff032d9d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff034101240 0x7ff03419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff03259c700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0340ff6b0 0x7ff034069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 -- 172.21.15.49:0/940974453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0340ff6b0 msgr2=0x7ff034069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0340ff6b0 0x7ff034069870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 -- 172.21.15.49:0/940974453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff034101240 msgr2=0x7ff03419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff034101240 0x7ff03419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:28.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 -- 172.21.15.49:0/940974453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c009d00 con 0x7ff0341008a0 2023-10-08T09:13:28.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff032d9d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff034101240 0x7ff03419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:13:28.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff031d9b700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0341008a0 0x7ff03419b500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00e9a0 tx=0x7ff01c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:28.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff02b7fe700 1 -- 172.21.15.49:0/940974453 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c009790 con 0x7ff0341008a0 2023-10-08T09:13:28.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff02b7fe700 1 -- 172.21.15.49:0/940974453 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff01c010430 con 0x7ff0341008a0 2023-10-08T09:13:28.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff02b7fe700 1 -- 172.21.15.49:0/940974453 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c018680 con 0x7ff0341008a0 2023-10-08T09:13:28.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03419e8c0 con 0x7ff0341008a0 2023-10-08T09:13:28.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.592+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03419edd0 con 0x7ff0341008a0 2023-10-08T09:13:28.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.594+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034106a60 con 0x7ff0341008a0 2023-10-08T09:13:28.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.595+0000 7ff02b7fe700 1 -- 172.21.15.49:0/940974453 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff01c0105a0 con 0x7ff0341008a0 2023-10-08T09:13:28.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.595+0000 7ff02b7fe700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0200a1500 0x7ff0200a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:28.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.595+0000 7ff03259c700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0200a1500 0x7ff0200a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:28.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.595+0000 7ff02b7fe700 1 -- 172.21.15.49:0/940974453 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff01c014070 con 0x7ff0341008a0 2023-10-08T09:13:28.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.596+0000 7ff03259c700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0200a1500 0x7ff0200a39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff034100700 tx=0x7ff02400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:28.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.599+0000 7ff02b7fe700 1 -- 172.21.15.49:0/940974453 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff01c0964b0 con 0x7ff0341008a0 2023-10-08T09:13:28.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:28 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2411387442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T09:13:28.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:28 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2411387442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T09:13:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:28.846+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7ff034067980 con 0x7ff0341008a0 2023-10-08T09:13:28.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:28 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2411387442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-08T09:13:29.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.441+0000 7ff02b7fe700 1 -- 172.21.15.49:0/940974453 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff034067980 con 0x7ff0341008a0 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0200a1500 msgr2=0x7ff0200a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0200a1500 0x7ff0200a39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff034100700 tx=0x7ff02400b410 comp rx=0 tx=0).stop 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0341008a0 msgr2=0x7ff03419b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0341008a0 0x7ff03419b500 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00e9a0 tx=0x7ff01c00ecb0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 shutdown_connections 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff0200a1500 0x7ff0200a39c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0340ff6b0 0x7ff034069870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff034101240 0x7ff03419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 --2- 172.21.15.49:0/940974453 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff0341008a0 0x7ff03419b500 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 >> 172.21.15.49:0/940974453 conn(0x7ff0340fac00 msgr2=0x7ff034103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 shutdown_connections 2023-10-08T09:13:29.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.444+0000 7ff038b4d700 1 -- 172.21.15.49:0/940974453 wait complete. 2023-10-08T09:13:29.464 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 -- 172.21.15.49:0/1186953083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb441008c0 msgr2=0x7fbb44100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/1186953083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb441008c0 0x7fbb44100d20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbb380099c0 tx=0x7fbb38009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 -- 172.21.15.49:0/1186953083 shutdown_connections 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/1186953083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb44101260 0x7fbb44107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/1186953083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb440ff6d0 0x7fbb440ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/1186953083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb441008c0 0x7fbb44100d20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 -- 172.21.15.49:0/1186953083 >> 172.21.15.49:0/1186953083 conn(0x7fbb440fac40 msgr2=0x7fbb440fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:29.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 -- 172.21.15.49:0/1186953083 shutdown_connections 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.597+0000 7fbb4bac5700 1 -- 172.21.15.49:0/1186953083 wait complete. 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 Processor -- start 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 -- start start 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb440ff6d0 0x7fbb44069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb441008c0 0x7fbb4419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44101260 0x7fbb4419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4419e240 con 0x7fbb44101260 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4419e3b0 con 0x7fbb440ff6d0 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4bac5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4419e570 con 0x7fbb441008c0 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4a062700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44101260 0x7fbb4419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4a062700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44101260 0x7fbb4419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48294/0 (socket says 172.21.15.49:48294) 2023-10-08T09:13:29.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb49861700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb440ff6d0 0x7fbb44069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb4a062700 1 -- 172.21.15.49:0/3052359736 learned_addr learned my addr 172.21.15.49:0/3052359736 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.598+0000 7fbb49060700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb441008c0 0x7fbb4419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4a062700 1 -- 172.21.15.49:0/3052359736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb441008c0 msgr2=0x7fbb4419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4a062700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb441008c0 0x7fbb4419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4a062700 1 -- 172.21.15.49:0/3052359736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb440ff6d0 msgr2=0x7fbb44069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4a062700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb440ff6d0 0x7fbb44069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4a062700 1 -- 172.21.15.49:0/3052359736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380095f0 con 0x7fbb44101260 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb49060700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb441008c0 0x7fbb4419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4a062700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44101260 0x7fbb4419da50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c007ee0 tx=0x7fbb3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb36ffd700 1 -- 172.21.15.49:0/3052359736 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c017070 con 0x7fbb44101260 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb36ffd700 1 -- 172.21.15.49:0/3052359736 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb3c00f040 con 0x7fbb44101260 2023-10-08T09:13:29.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4419e7f0 con 0x7fbb44101260 2023-10-08T09:13:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb36ffd700 1 -- 172.21.15.49:0/3052359736 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c005740 con 0x7fbb44101260 2023-10-08T09:13:29.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.599+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4419ec80 con 0x7fbb44101260 2023-10-08T09:13:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.600+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb44106ba0 con 0x7fbb44101260 2023-10-08T09:13:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.601+0000 7fbb36ffd700 1 -- 172.21.15.49:0/3052359736 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbb3c00dc60 con 0x7fbb44101260 2023-10-08T09:13:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.602+0000 7fbb36ffd700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb300a1480 0x7fbb300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.602+0000 7fbb49861700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb300a1480 0x7fbb300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.602+0000 7fbb36ffd700 1 -- 172.21.15.49:0/3052359736 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbb3c0ccda0 con 0x7fbb44101260 2023-10-08T09:13:29.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.602+0000 7fbb49861700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb300a1480 0x7fbb300a3940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb44100720 tx=0x7fbb4000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:29.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.605+0000 7fbb36ffd700 1 -- 172.21.15.49:0/3052359736 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb3c096210 con 0x7fbb44101260 2023-10-08T09:13:29.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:29 smithi184 ceph-mon[103675]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:29.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:29 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2411387442' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T09:13:29.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:29 smithi184 ceph-mon[103675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:29.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:29 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/940974453' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-08T09:13:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:29 smithi132 ceph-mon[103389]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:29 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2411387442' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T09:13:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:29 smithi132 ceph-mon[103389]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:29 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/940974453' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-08T09:13:29.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:29.855+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fbb4410aff0 con 0x7fbb44101260 2023-10-08T09:13:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:29 smithi049 ceph-mon[99485]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:29 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2411387442' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-08T09:13:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:29 smithi049 ceph-mon[99485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:29 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/940974453' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-08T09:13:30.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.451+0000 7fbb36ffd700 1 -- 172.21.15.49:0/3052359736 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fbb3c099830 con 0x7fbb44101260 2023-10-08T09:13:30.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb300a1480 msgr2=0x7fbb300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb300a1480 0x7fbb300a3940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb44100720 tx=0x7fbb4000b410 comp rx=0 tx=0).stop 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44101260 msgr2=0x7fbb4419da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44101260 0x7fbb4419da50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c007ee0 tx=0x7fbb3c00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 shutdown_connections 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb300a1480 0x7fbb300a3940 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb44101260 0x7fbb4419da50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb440ff6d0 0x7fbb44069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 --2- 172.21.15.49:0/3052359736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb441008c0 0x7fbb4419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 >> 172.21.15.49:0/3052359736 conn(0x7fbb440fac40 msgr2=0x7fbb44103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 shutdown_connections 2023-10-08T09:13:30.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.454+0000 7fbb4bac5700 1 -- 172.21.15.49:0/3052359736 wait complete. 2023-10-08T09:13:30.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-08T09:13:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.610+0000 7f5b65dae700 1 -- 172.21.15.49:0/3178670549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 msgr2=0x7f5b60100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.610+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3178670549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60100cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5b540099c0 tx=0x7f5b54009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.610+0000 7f5b65dae700 1 -- 172.21.15.49:0/3178670549 shutdown_connections 2023-10-08T09:13:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.610+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3178670549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b60101230 0x7f5b60107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.610+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3178670549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b600ff6a0 0x7f5b600ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.610+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3178670549 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60100cf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.611+0000 7f5b65dae700 1 -- 172.21.15.49:0/3178670549 >> 172.21.15.49:0/3178670549 conn(0x7f5b600fac10 msgr2=0x7f5b600fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.611+0000 7f5b65dae700 1 -- 172.21.15.49:0/3178670549 shutdown_connections 2023-10-08T09:13:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.611+0000 7f5b65dae700 1 -- 172.21.15.49:0/3178670549 wait complete. 2023-10-08T09:13:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.611+0000 7f5b65dae700 1 Processor -- start 2023-10-08T09:13:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b65dae700 1 -- start start 2023-10-08T09:13:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b65dae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b600ff6a0 0x7f5b60195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b65dae700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b65dae700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b60101230 0x7f5b6019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b65dae700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019d150 con 0x7f5b60100890 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b65dae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019d290 con 0x7f5b600ff6a0 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b65dae700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019d420 con 0x7f5b60101230 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48306/0 (socket says 172.21.15.49:48306) 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 -- 172.21.15.49:0/3759242379 learned_addr learned my addr 172.21.15.49:0/3759242379 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5f7fe700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b600ff6a0 0x7f5b60195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5ffff700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b60101230 0x7f5b6019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 -- 172.21.15.49:0/3759242379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b60101230 msgr2=0x7f5b6019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b60101230 0x7f5b6019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 -- 172.21.15.49:0/3759242379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b600ff6a0 msgr2=0x7f5b60195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b600ff6a0 0x7f5b60195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 -- 172.21.15.49:0/3759242379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b540095f0 con 0x7f5b60100890 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5f7fe700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b600ff6a0 0x7f5b60195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.612+0000 7f5b5effd700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60195b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b540036f0 tx=0x7f5b540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.613+0000 7f5b5cff9700 1 -- 172.21.15.49:0/3759242379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5401d070 con 0x7f5b60100890 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.613+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6019d6a0 con 0x7f5b60100890 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.613+0000 7f5b5cff9700 1 -- 172.21.15.49:0/3759242379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b54004110 con 0x7f5b60100890 2023-10-08T09:13:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.613+0000 7f5b5cff9700 1 -- 172.21.15.49:0/3759242379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54022e10 con 0x7f5b60100890 2023-10-08T09:13:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.613+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6019db50 con 0x7f5b60100890 2023-10-08T09:13:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.614+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b60106b20 con 0x7f5b60100890 2023-10-08T09:13:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.615+0000 7f5b5cff9700 1 -- 172.21.15.49:0/3759242379 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5b54022470 con 0x7f5b60100890 2023-10-08T09:13:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.615+0000 7f5b5cff9700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b480a1480 0x7f5b480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.616+0000 7f5b5f7fe700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b480a1480 0x7f5b480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.616+0000 7f5b5cff9700 1 -- 172.21.15.49:0/3759242379 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5b540ce490 con 0x7f5b60100890 2023-10-08T09:13:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.616+0000 7f5b5f7fe700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b480a1480 0x7f5b480a3940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5b50005950 tx=0x7f5b500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.619+0000 7f5b5cff9700 1 -- 172.21.15.49:0/3759242379 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5b54097930 con 0x7f5b60100890 2023-10-08T09:13:30.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:30 smithi184 ceph-mon[103675]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T09:13:30.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:30 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/940974453' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-08T09:13:30.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:30 smithi184 ceph-mon[103675]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T09:13:30.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:30 smithi184 ceph-mon[103675]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:30.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:30 smithi184 ceph-mon[103675]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T09:13:30.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:30 smithi184 ceph-mon[103675]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T09:13:30.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:30 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3052359736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T09:13:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:30 smithi132 ceph-mon[103389]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T09:13:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:30 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/940974453' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-08T09:13:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:30 smithi132 ceph-mon[103389]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T09:13:30.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:30 smithi132 ceph-mon[103389]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:30.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:30 smithi132 ceph-mon[103389]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T09:13:30.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:30 smithi132 ceph-mon[103389]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T09:13:30.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:30 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3052359736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T09:13:30.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:30.868+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f5b60067530 con 0x7f5b60100890 2023-10-08T09:13:30.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:30 smithi049 ceph-mon[99485]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-08T09:13:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:30 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/940974453' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-08T09:13:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:30 smithi049 ceph-mon[99485]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-08T09:13:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:30 smithi049 ceph-mon[99485]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-08T09:13:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:30 smithi049 ceph-mon[99485]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-08T09:13:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:30 smithi049 ceph-mon[99485]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-08T09:13:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:30 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3052359736' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-08T09:13:31.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.399+0000 7f5b5cff9700 1 -- 172.21.15.49:0/3759242379 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5b5409af50 con 0x7f5b60100890 2023-10-08T09:13:31.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b480a1480 msgr2=0x7f5b480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b480a1480 0x7f5b480a3940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5b50005950 tx=0x7f5b500058e0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 msgr2=0x7f5b60195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60195b60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5b540036f0 tx=0x7f5b540047b0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 shutdown_connections 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b480a1480 0x7f5b480a3940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b60101230 0x7f5b6019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b600ff6a0 0x7f5b60195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 --2- 172.21.15.49:0/3759242379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b60100890 0x7f5b60195b60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 >> 172.21.15.49:0/3759242379 conn(0x7f5b600fac10 msgr2=0x7f5b60103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 shutdown_connections 2023-10-08T09:13:31.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.401+0000 7f5b65dae700 1 -- 172.21.15.49:0/3759242379 wait complete. 2023-10-08T09:13:31.421 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-10-08T09:13:31.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-08T09:13:31.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 -- 172.21.15.49:0/754002426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5398102350 msgr2=0x7f5398108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 --2- 172.21.15.49:0/754002426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5398102350 0x7f5398108c00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f539000b0d0 tx=0x7f539000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 -- 172.21.15.49:0/754002426 shutdown_connections 2023-10-08T09:13:31.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 --2- 172.21.15.49:0/754002426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5398102350 0x7f5398108c00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 --2- 172.21.15.49:0/754002426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53981007c0 0x7f5398100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 --2- 172.21.15.49:0/754002426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53981019b0 0x7f5398101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 -- 172.21.15.49:0/754002426 >> 172.21.15.49:0/754002426 conn(0x7f53980fbd30 msgr2=0x7f53980fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 -- 172.21.15.49:0/754002426 shutdown_connections 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.554+0000 7f53a0558700 1 -- 172.21.15.49:0/754002426 wait complete. 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 Processor -- start 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 -- start start 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53981007c0 0x7f5398196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53981019b0 0x7f5398196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398102350 0x7f539819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:31.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819e340 con 0x7f53981019b0 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819e480 con 0x7f5398102350 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f53a0558700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819e610 con 0x7f53981007c0 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53981019b0 0x7f5398196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53981019b0 0x7f5398196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48320/0 (socket says 172.21.15.49:48320) 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 -- 172.21.15.49:0/3221085387 learned_addr learned my addr 172.21.15.49:0/3221085387 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539eaf5700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398102350 0x7f539819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539e2f4700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53981007c0 0x7f5398196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 -- 172.21.15.49:0/3221085387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53981007c0 msgr2=0x7f5398196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:31.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53981007c0 0x7f5398196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 -- 172.21.15.49:0/3221085387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398102350 msgr2=0x7f539819dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398102350 0x7f539819dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.555+0000 7f539daf3700 1 -- 172.21.15.49:0/3221085387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5390009d00 con 0x7f53981019b0 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.556+0000 7f539daf3700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53981019b0 0x7f5398196d50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9a0 tx=0x7f539400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.556+0000 7f538f7fe700 1 -- 172.21.15.49:0/3221085387 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394009790 con 0x7f53981019b0 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.556+0000 7f538f7fe700 1 -- 172.21.15.49:0/3221085387 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539400cbd0 con 0x7f53981019b0 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.556+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539819e8f0 con 0x7f53981019b0 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.556+0000 7f538f7fe700 1 -- 172.21.15.49:0/3221085387 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394018680 con 0x7f53981019b0 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.556+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539819ee50 con 0x7f53981019b0 2023-10-08T09:13:31.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.557+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5398107bf0 con 0x7f53981019b0 2023-10-08T09:13:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.558+0000 7f538f7fe700 1 -- 172.21.15.49:0/3221085387 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f539400cd40 con 0x7f53981019b0 2023-10-08T09:13:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.558+0000 7f538f7fe700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53840a1500 0x7f53840a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.558+0000 7f538f7fe700 1 -- 172.21.15.49:0/3221085387 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5394014070 con 0x7f53981019b0 2023-10-08T09:13:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.558+0000 7f539e2f4700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53840a1500 0x7f53840a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:31.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.559+0000 7f539e2f4700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53840a1500 0x7f53840a39c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5388005950 tx=0x7f53880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:31.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.563+0000 7f538f7fe700 1 -- 172.21.15.49:0/3221085387 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5394052100 con 0x7f53981019b0 2023-10-08T09:13:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3052359736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] up:active 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3759242379' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3759242379' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T09:13:31.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] up:active 2023-10-08T09:13:31.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-08T09:13:31.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:13:31.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2023-10-08T09:13:31.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:31 smithi184 ceph-mon[103675]: daemon mds.l is now active in filesystem cephfs as rank 3 2023-10-08T09:13:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3052359736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T09:13:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] up:active 2023-10-08T09:13:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T09:13:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T09:13:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-08T09:13:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-08T09:13:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3759242379' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T09:13:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3759242379' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T09:13:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] up:active 2023-10-08T09:13:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-08T09:13:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:13:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2023-10-08T09:13:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-mon[103389]: daemon mds.l is now active in filesystem cephfs as rank 3 2023-10-08T09:13:31.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:31.805+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f5398060c30 con 0x7f53840a1500 2023-10-08T09:13:31.814 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:13:31.810+0000 7fe3a9011700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T09:13:31.814 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:13:31.810+0000 7fe3a9011700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T09:13:31.814 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:13:31.810+0000 7fe3a9011700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T09:13:31.814 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:13:31.810+0000 7fe3a9011700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T09:13:31.814 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:13:31.810+0000 7fe3a9011700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T09:13:31.814 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-08T09:13:31.814 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3052359736' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] up:active 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3759242379' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3759242379' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] up:active 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2023-10-08T09:13:31.815 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:31 smithi049 ceph-mon[99485]: daemon mds.l is now active in filesystem cephfs as rank 3 2023-10-08T09:13:32.297 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: 2023-10-08T09:13:31.940+0000 7fb4a80a8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-08T09:13:32.297 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:13:31 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: 2023-10-08T09:13:31.940+0000 7fb4a80a8700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-08T09:13:33.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.452+0000 7f538f7fe700 1 -- 172.21.15.49:0/3221085387 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5398060c30 con 0x7f53840a1500 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53840a1500 msgr2=0x7f53840a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53840a1500 0x7f53840a39c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5388005950 tx=0x7f53880058e0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53981019b0 msgr2=0x7f5398196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53981019b0 0x7f5398196d50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9a0 tx=0x7f539400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 shutdown_connections 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53840a1500 0x7f53840a39c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53981007c0 0x7f5398196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5398102350 0x7f539819dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 --2- 172.21.15.49:0/3221085387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53981019b0 0x7f5398196d50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 >> 172.21.15.49:0/3221085387 conn(0x7f53980fbd30 msgr2=0x7f5398104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:33.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 shutdown_connections 2023-10-08T09:13:33.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.454+0000 7f53a0558700 1 -- 172.21.15.49:0/3221085387 wait complete. 2023-10-08T09:13:33.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-08T09:13:33.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 -- 172.21.15.49:0/3271601713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa894101240 msgr2=0x7fa894107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 --2- 172.21.15.49:0/3271601713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa894101240 0x7fa894107af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00b0d0 tx=0x7fa88c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 -- 172.21.15.49:0/3271601713 shutdown_connections 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 --2- 172.21.15.49:0/3271601713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa894101240 0x7fa894107af0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 --2- 172.21.15.49:0/3271601713 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa8940ff6b0 0x7fa8940ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 --2- 172.21.15.49:0/3271601713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8941008a0 0x7fa894100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 -- 172.21.15.49:0/3271601713 >> 172.21.15.49:0/3271601713 conn(0x7fa8940fac40 msgr2=0x7fa8940fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 -- 172.21.15.49:0/3271601713 shutdown_connections 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.604+0000 7fa89aced700 1 -- 172.21.15.49:0/3271601713 wait complete. 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 Processor -- start 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 -- start start 2023-10-08T09:13:33.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8940ff6b0 0x7fa8941966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8941008a0 0x7fa894196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa894101240 0x7fa89419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419e1d0 con 0x7fa8940ff6b0 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419e310 con 0x7fa894101240 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89aced700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419e4a0 con 0x7fa8941008a0 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8940ff6b0 0x7fa8941966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8940ff6b0 0x7fa8941966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48374/0 (socket says 172.21.15.49:48374) 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 -- 172.21.15.49:0/1476375475 learned_addr learned my addr 172.21.15.49:0/1476375475 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89928a700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa894101240 0x7fa89419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa893fff700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8941008a0 0x7fa894196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 -- 172.21.15.49:0/1476375475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8941008a0 msgr2=0x7fa894196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:33.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8941008a0 0x7fa894196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 -- 172.21.15.49:0/1476375475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa894101240 msgr2=0x7fa89419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa894101240 0x7fa89419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 -- 172.21.15.49:0/1476375475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c009d00 con 0x7fa8940ff6b0 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa893fff700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8941008a0 0x7fa894196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa89928a700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa894101240 0x7fa89419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa898a89700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8940ff6b0 0x7fa8941966a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c990 tx=0x7fa88800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.605+0000 7fa891ffb700 1 -- 172.21.15.49:0/1476375475 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888007b00 con 0x7fa8940ff6b0 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.606+0000 7fa891ffb700 1 -- 172.21.15.49:0/1476375475 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa888007c60 con 0x7fa8940ff6b0 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.606+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89419e780 con 0x7fa8940ff6b0 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.606+0000 7fa891ffb700 1 -- 172.21.15.49:0/1476375475 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888018660 con 0x7fa8940ff6b0 2023-10-08T09:13:33.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.606+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89419ec90 con 0x7fa8940ff6b0 2023-10-08T09:13:33.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.607+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa894067980 con 0x7fa8940ff6b0 2023-10-08T09:13:33.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.608+0000 7fa891ffb700 1 -- 172.21.15.49:0/1476375475 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa88800f450 con 0x7fa8940ff6b0 2023-10-08T09:13:33.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.608+0000 7fa891ffb700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8840a1480 0x7fa8840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:33.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.608+0000 7fa893fff700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8840a1480 0x7fa8840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:33.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.608+0000 7fa891ffb700 1 -- 172.21.15.49:0/1476375475 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa8880ccee0 con 0x7fa8940ff6b0 2023-10-08T09:13:33.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.609+0000 7fa893fff700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8840a1480 0x7fa8840a3940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa880005fd0 tx=0x7fa880005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:33.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.613+0000 7fa891ffb700 1 -- 172.21.15.49:0/1476375475 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa888096400 con 0x7fa8940ff6b0 2023-10-08T09:13:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: pgmap v284: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 2 op/s 2023-10-08T09:13:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] up:active 2023-10-08T09:13:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-08T09:13:33.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T09:13:33.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: Cluster is now healthy 2023-10-08T09:13:33.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2023-10-08T09:13:33.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-10-08T09:13:33.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:33 smithi184 ceph-mon[103675]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-10-08T09:13:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: pgmap v284: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 2 op/s 2023-10-08T09:13:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] up:active 2023-10-08T09:13:33.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-08T09:13:33.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T09:13:33.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: Cluster is now healthy 2023-10-08T09:13:33.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2023-10-08T09:13:33.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-10-08T09:13:33.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:33 smithi132 ceph-mon[103389]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-10-08T09:13:33.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.851+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8940fccc0 con 0x7fa8840a1480 2023-10-08T09:13:33.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.855+0000 7fa891ffb700 1 -- 172.21.15.49:0/1476375475 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fa8940fccc0 con 0x7fa8840a1480 2023-10-08T09:13:33.857 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/_nogroup/sv_0/13c63aef-59da-4382-b7e4-023c571af7ba 2023-10-08T09:13:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8840a1480 msgr2=0x7fa8840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8840a1480 0x7fa8840a3940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa880005fd0 tx=0x7fa880005e20 comp rx=0 tx=0).stop 2023-10-08T09:13:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8940ff6b0 msgr2=0x7fa8941966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8940ff6b0 0x7fa8941966a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c990 tx=0x7fa88800cd50 comp rx=0 tx=0).stop 2023-10-08T09:13:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 shutdown_connections 2023-10-08T09:13:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa8840a1480 0x7fa8840a3940 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa894101240 0x7fa89419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa8940ff6b0 0x7fa8941966a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 --2- 172.21.15.49:0/1476375475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa8941008a0 0x7fa894196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:33.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.857+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 >> 172.21.15.49:0/1476375475 conn(0x7fa8940fac40 msgr2=0x7fa8940fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:33.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.858+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 shutdown_connections 2023-10-08T09:13:33.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:33.858+0000 7fa89aced700 1 -- 172.21.15.49:0/1476375475 wait complete. 2023-10-08T09:13:33.870 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:33.870 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: pgmap v284: 129 pgs: 129 active+clean; 582 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 2 op/s 2023-10-08T09:13:33.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] up:active 2023-10-08T09:13:33.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-08T09:13:33.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-08T09:13:33.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: Cluster is now healthy 2023-10-08T09:13:33.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2023-10-08T09:13:33.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-10-08T09:13:33.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:33 smithi049 ceph-mon[99485]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-10-08T09:13:33.872 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-08T09:13:34.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 -- 172.21.15.49:0/1773329109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c101240 msgr2=0x7fbf2c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 --2- 172.21.15.49:0/1773329109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c101240 0x7fbf2c107af0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400b0d0 tx=0x7fbf2400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 -- 172.21.15.49:0/1773329109 shutdown_connections 2023-10-08T09:13:34.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 --2- 172.21.15.49:0/1773329109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c101240 0x7fbf2c107af0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 --2- 172.21.15.49:0/1773329109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf2c0ff6b0 0x7fbf2c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 --2- 172.21.15.49:0/1773329109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf2c1008a0 0x7fbf2c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 -- 172.21.15.49:0/1773329109 >> 172.21.15.49:0/1773329109 conn(0x7fbf2c0fac40 msgr2=0x7fbf2c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 -- 172.21.15.49:0/1773329109 shutdown_connections 2023-10-08T09:13:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.000+0000 7fbf323ed700 1 -- 172.21.15.49:0/1773329109 wait complete. 2023-10-08T09:13:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 Processor -- start 2023-10-08T09:13:34.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 -- start start 2023-10-08T09:13:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf2c0ff6b0 0x7fbf2c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c1008a0 0x7fbf2c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf2c101240 0x7fbf2c19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c1a04b0 con 0x7fbf2c1008a0 2023-10-08T09:13:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c1a05f0 con 0x7fbf2c101240 2023-10-08T09:13:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf323ed700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c1a0780 con 0x7fbf2c0ff6b0 2023-10-08T09:13:34.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf2b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c1008a0 0x7fbf2c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf2b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c1008a0 0x7fbf2c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48388/0 (socket says 172.21.15.49:48388) 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.001+0000 7fbf2b7fe700 1 -- 172.21.15.49:0/3070249573 learned_addr learned my addr 172.21.15.49:0/3070249573 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf3098a700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf2c101240 0x7fbf2c19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2bfff700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf2c0ff6b0 0x7fbf2c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2b7fe700 1 -- 172.21.15.49:0/3070249573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf2c0ff6b0 msgr2=0x7fbf2c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2b7fe700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf2c0ff6b0 0x7fbf2c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2b7fe700 1 -- 172.21.15.49:0/3070249573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf2c101240 msgr2=0x7fbf2c19fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2b7fe700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf2c101240 0x7fbf2c19fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2b7fe700 1 -- 172.21.15.49:0/3070249573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009d00 con 0x7fbf2c1008a0 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2bfff700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf2c0ff6b0 0x7fbf2c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:34.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf2b7fe700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c1008a0 0x7fbf2c198ec0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d980 tx=0x7fbf2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf297fa700 1 -- 172.21.15.49:0/3070249573 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20004220 con 0x7fbf2c1008a0 2023-10-08T09:13:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf297fa700 1 -- 172.21.15.49:0/3070249573 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf20007640 con 0x7fbf2c1008a0 2023-10-08T09:13:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2c1a0a60 con 0x7fbf2c1008a0 2023-10-08T09:13:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf297fa700 1 -- 172.21.15.49:0/3070249573 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20005610 con 0x7fbf2c1008a0 2023-10-08T09:13:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.002+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2c1a0f70 con 0x7fbf2c1008a0 2023-10-08T09:13:34.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.003+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2c105270 con 0x7fbf2c1008a0 2023-10-08T09:13:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.004+0000 7fbf297fa700 1 -- 172.21.15.49:0/3070249573 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbf20004750 con 0x7fbf2c1008a0 2023-10-08T09:13:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.005+0000 7fbf297fa700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf140a1480 0x7fbf140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.005+0000 7fbf2bfff700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf140a1480 0x7fbf140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.005+0000 7fbf297fa700 1 -- 172.21.15.49:0/3070249573 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbf200cd1a0 con 0x7fbf2c1008a0 2023-10-08T09:13:34.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.005+0000 7fbf2bfff700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf140a1480 0x7fbf140a3940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c100700 tx=0x7fbf1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:34.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.009+0000 7fbf297fa700 1 -- 172.21.15.49:0/3070249573 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf200966b0 con 0x7fbf2c1008a0 2023-10-08T09:13:34.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.244+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf2c02cf80 con 0x7fbf140a1480 2023-10-08T09:13:34.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.271+0000 7fbf297fa700 1 -- 172.21.15.49:0/3070249573 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbf2c02cf80 con 0x7fbf140a1480 2023-10-08T09:13:34.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf140a1480 msgr2=0x7fbf140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf140a1480 0x7fbf140a3940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c100700 tx=0x7fbf1c009260 comp rx=0 tx=0).stop 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c1008a0 msgr2=0x7fbf2c198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c1008a0 0x7fbf2c198ec0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d980 tx=0x7fbf2000dc90 comp rx=0 tx=0).stop 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 shutdown_connections 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf140a1480 0x7fbf140a3940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf2c0ff6b0 0x7fbf2c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf2c101240 0x7fbf2c19fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 --2- 172.21.15.49:0/3070249573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf2c1008a0 0x7fbf2c198ec0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 >> 172.21.15.49:0/3070249573 conn(0x7fbf2c0fac40 msgr2=0x7fbf2c0fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:34.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 shutdown_connections 2023-10-08T09:13:34.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.273+0000 7fbf323ed700 1 -- 172.21.15.49:0/3070249573 wait complete. 2023-10-08T09:13:34.287 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 -- 172.21.15.49:0/3875169965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84101260 msgr2=0x7f3d84107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 --2- 172.21.15.49:0/3875169965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84101260 0x7f3d84107b10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00b0d0 tx=0x7f3d7c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 -- 172.21.15.49:0/3875169965 shutdown_connections 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 --2- 172.21.15.49:0/3875169965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d840ff6d0 0x7f3d840ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 --2- 172.21.15.49:0/3875169965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d84101260 0x7f3d84107b10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 --2- 172.21.15.49:0/3875169965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d841008c0 0x7f3d84100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 -- 172.21.15.49:0/3875169965 >> 172.21.15.49:0/3875169965 conn(0x7f3d840fac40 msgr2=0x7f3d840fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 -- 172.21.15.49:0/3875169965 shutdown_connections 2023-10-08T09:13:34.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.419+0000 7f3d89e18700 1 -- 172.21.15.49:0/3875169965 wait complete. 2023-10-08T09:13:34.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 Processor -- start 2023-10-08T09:13:34.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 -- start start 2023-10-08T09:13:34.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d840ff6d0 0x7f3d84069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d841008c0 0x7f3d8419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d84101260 0x7f3d8419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8419e2e0 con 0x7f3d841008c0 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8419e450 con 0x7f3d840ff6d0 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d89e18700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8419e610 con 0x7f3d84101260 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d841008c0 0x7f3d8419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d82ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d841008c0 0x7f3d8419b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48404/0 (socket says 172.21.15.49:48404) 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d82ffd700 1 -- 172.21.15.49:0/876790548 learned_addr learned my addr 172.21.15.49:0/876790548 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d837fe700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d840ff6d0 0x7f3d84069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d83fff700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d84101260 0x7f3d8419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.420+0000 7f3d82ffd700 1 -- 172.21.15.49:0/876790548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d84101260 msgr2=0x7f3d8419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.421+0000 7f3d82ffd700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d84101260 0x7f3d8419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.421+0000 7f3d82ffd700 1 -- 172.21.15.49:0/876790548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d840ff6d0 msgr2=0x7f3d84069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.421+0000 7f3d82ffd700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d840ff6d0 0x7f3d84069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.421+0000 7f3d82ffd700 1 -- 172.21.15.49:0/876790548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c009d00 con 0x7f3d841008c0 2023-10-08T09:13:34.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.421+0000 7f3d83fff700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d84101260 0x7f3d8419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:34.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.421+0000 7f3d82ffd700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d841008c0 0x7f3d8419b550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00d980 tx=0x7f3d6c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.423+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c004220 con 0x7f3d841008c0 2023-10-08T09:13:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.423+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d6c007640 con 0x7f3d841008c0 2023-10-08T09:13:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.423+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8419e8f0 con 0x7f3d841008c0 2023-10-08T09:13:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.423+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c005610 con 0x7f3d841008c0 2023-10-08T09:13:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.424+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8419ee50 con 0x7f3d841008c0 2023-10-08T09:13:34.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.424+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d84106a80 con 0x7f3d841008c0 2023-10-08T09:13:34.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.426+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3d6c004750 con 0x7f3d841008c0 2023-10-08T09:13:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.426+0000 7f3d80ff9700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d700a1500 0x7f3d700a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.426+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3d6c0ccd00 con 0x7f3d841008c0 2023-10-08T09:13:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.426+0000 7f3d837fe700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d700a1500 0x7f3d700a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.427+0000 7f3d837fe700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d700a1500 0x7f3d700a39c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d84100720 tx=0x7f3d7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:34.434 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.433+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3d6c096540 con 0x7f3d841008c0 2023-10-08T09:13:34.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.440+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3d6c094150 con 0x7f3d841008c0 2023-10-08T09:13:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:34 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] up:active 2023-10-08T09:13:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:34 smithi049 ceph-mon[99485]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-10-08T09:13:34.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:34 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:13:34.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.682+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d84060c30 con 0x7f3d700a1500 2023-10-08T09:13:34.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.686+0000 7f3d80ff9700 1 -- 172.21.15.49:0/876790548 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f3d84060c30 con 0x7f3d700a1500 2023-10-08T09:13:34.687 INFO:teuthology.orchestra.run.smithi049.stdout:/volumes/_nogroup/sv_1/485a435c-3760-47ed-ad82-bcf59bb1ae2b 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d700a1500 msgr2=0x7f3d700a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d700a1500 0x7f3d700a39c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d84100720 tx=0x7f3d7400b410 comp rx=0 tx=0).stop 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d841008c0 msgr2=0x7f3d8419b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d841008c0 0x7f3d8419b550 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00d980 tx=0x7f3d6c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 shutdown_connections 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d700a1500 0x7f3d700a39c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d84101260 0x7f3d8419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d840ff6d0 0x7f3d84069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 --2- 172.21.15.49:0/876790548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d841008c0 0x7f3d8419b550 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.688+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 >> 172.21.15.49:0/876790548 conn(0x7f3d840fac40 msgr2=0x7f3d84103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.689+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 shutdown_connections 2023-10-08T09:13:34.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.689+0000 7f3d89e18700 1 -- 172.21.15.49:0/876790548 wait complete. 2023-10-08T09:13:34.702 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:13:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:34 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] up:active 2023-10-08T09:13:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:34 smithi184 ceph-mon[103675]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-10-08T09:13:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:34 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:13:34.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:34 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] up:active 2023-10-08T09:13:34.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:34 smithi132 ceph-mon[103389]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-10-08T09:13:34.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:34 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 -- 172.21.15.49:0/1824063338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0101130 msgr2=0x7f99f01079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 --2- 172.21.15.49:0/1824063338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0101130 0x7f99f01079e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f99e800b0d0 tx=0x7f99e800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 -- 172.21.15.49:0/1824063338 shutdown_connections 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 --2- 172.21.15.49:0/1824063338 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f00ff5a0 0x7f99f00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 --2- 172.21.15.49:0/1824063338 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f0101130 0x7f99f01079e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 --2- 172.21.15.49:0/1824063338 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f0100790 0x7f99f0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 -- 172.21.15.49:0/1824063338 >> 172.21.15.49:0/1824063338 conn(0x7f99f00fab10 msgr2=0x7f99f00fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 -- 172.21.15.49:0/1824063338 shutdown_connections 2023-10-08T09:13:34.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.827+0000 7f99f50fa700 1 -- 172.21.15.49:0/1824063338 wait complete. 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 Processor -- start 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 -- start start 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f00ff5a0 0x7f99f0068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f0100790 0x7f99f019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0101130 0x7f99f019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019e200 con 0x7f99f00ff5a0 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019e370 con 0x7f99f0100790 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99f50fa700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f019e530 con 0x7f99f0101130 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f00ff5a0 0x7f99f0068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f00ff5a0 0x7f99f0068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48416/0 (socket says 172.21.15.49:48416) 2023-10-08T09:13:34.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.828+0000 7f99eed9d700 1 -- 172.21.15.49:0/4084469350 learned_addr learned my addr 172.21.15.49:0/4084469350 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99ef59e700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0101130 0x7f99f019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99ee59c700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f0100790 0x7f99f019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99eed9d700 1 -- 172.21.15.49:0/4084469350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0101130 msgr2=0x7f99f019da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99eed9d700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0101130 0x7f99f019da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99eed9d700 1 -- 172.21.15.49:0/4084469350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f0100790 msgr2=0x7f99f019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99eed9d700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f0100790 0x7f99f019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99eed9d700 1 -- 172.21.15.49:0/4084469350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8009d00 con 0x7f99f00ff5a0 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99ef59e700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0101130 0x7f99f019da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99eed9d700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f00ff5a0 0x7f99f0068370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f99e000e970 tx=0x7f99e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:34.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99d7fff700 1 -- 172.21.15.49:0/4084469350 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0009790 con 0x7f99f00ff5a0 2023-10-08T09:13:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99d7fff700 1 -- 172.21.15.49:0/4084469350 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99e00044e0 con 0x7f99f00ff5a0 2023-10-08T09:13:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f0195280 con 0x7f99f00ff5a0 2023-10-08T09:13:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99d7fff700 1 -- 172.21.15.49:0/4084469350 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0003e30 con 0x7f99f00ff5a0 2023-10-08T09:13:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.829+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f0195790 con 0x7f99f00ff5a0 2023-10-08T09:13:34.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.831+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0106950 con 0x7f99f00ff5a0 2023-10-08T09:13:34.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.832+0000 7f99d7fff700 1 -- 172.21.15.49:0/4084469350 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f99e0003f90 con 0x7f99f00ff5a0 2023-10-08T09:13:34.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.832+0000 7f99d7fff700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f99d80a1660 0x7f99d80a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:34.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.833+0000 7f99d7fff700 1 -- 172.21.15.49:0/4084469350 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f99e0014070 con 0x7f99f00ff5a0 2023-10-08T09:13:34.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.833+0000 7f99ee59c700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f99d80a1660 0x7f99d80a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:34.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.833+0000 7f99ee59c700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f99d80a1660 0x7f99d80a3b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99f0068910 tx=0x7f99e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:34.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:34.838+0000 7f99d7fff700 1 -- 172.21.15.49:0/4084469350 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f99e0096530 con 0x7f99f00ff5a0 2023-10-08T09:13:35.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.088+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99f0067450 con 0x7f99f00ff5a0 2023-10-08T09:13:35.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.089+0000 7f99d7fff700 1 -- 172.21.15.49:0/4084469350 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f99e0099b50 con 0x7f99f00ff5a0 2023-10-08T09:13:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:35.091 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:13:35.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f99d80a1660 msgr2=0x7f99d80a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f99d80a1660 0x7f99d80a3b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99f0068910 tx=0x7f99e4009260 comp rx=0 tx=0).stop 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f00ff5a0 msgr2=0x7f99f0068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f00ff5a0 0x7f99f0068370 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f99e000e970 tx=0x7f99e000ed30 comp rx=0 tx=0).stop 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 shutdown_connections 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f99d80a1660 0x7f99d80a3b20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99f0101130 0x7f99f019da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99f00ff5a0 0x7f99f0068370 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 --2- 172.21.15.49:0/4084469350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f99f0100790 0x7f99f019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 >> 172.21.15.49:0/4084469350 conn(0x7f99f00fab10 msgr2=0x7f99f0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 shutdown_connections 2023-10-08T09:13:35.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.091+0000 7f99f50fa700 1 -- 172.21.15.49:0/4084469350 wait complete. 2023-10-08T09:13:35.094 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:13:35.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:13:35.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 -- 172.21.15.49:0/2413866523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 msgr2=0x7f5484100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 --2- 172.21.15.49:0/2413866523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f5484100c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54740099c0 tx=0x7f5474009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 -- 172.21.15.49:0/2413866523 shutdown_connections 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 --2- 172.21.15.49:0/2413866523 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5484101160 0x7f5484107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 --2- 172.21.15.49:0/2413866523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54840ff5d0 0x7f54840ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 --2- 172.21.15.49:0/2413866523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f5484100c20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 -- 172.21.15.49:0/2413866523 >> 172.21.15.49:0/2413866523 conn(0x7f54840fab40 msgr2=0x7f54840fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 -- 172.21.15.49:0/2413866523 shutdown_connections 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 -- 172.21.15.49:0/2413866523 wait complete. 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 Processor -- start 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.231+0000 7f548b325700 1 -- start start 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f548b325700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54840ff5d0 0x7f5484069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f548b325700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f548419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f548b325700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5484101160 0x7f548419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f548b325700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548419e0e0 con 0x7f54841007c0 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f548b325700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548419e250 con 0x7f5484101160 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f548b325700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548419e410 con 0x7f54840ff5d0 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f548419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f548419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48432/0 (socket says 172.21.15.49:48432) 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 -- 172.21.15.49:0/1878820473 learned_addr learned my addr 172.21.15.49:0/1878820473 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54890c1700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54840ff5d0 0x7f5484069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54898c2700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5484101160 0x7f548419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 -- 172.21.15.49:0/1878820473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54840ff5d0 msgr2=0x7f5484069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54840ff5d0 0x7f5484069730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 -- 172.21.15.49:0/1878820473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5484101160 msgr2=0x7f548419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5484101160 0x7f548419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 -- 172.21.15.49:0/1878820473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54740095f0 con 0x7f54841007c0 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54898c2700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5484101160 0x7f548419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f54888c0700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f548419b350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54740036f0 tx=0x7f54740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.232+0000 7f547a7fc700 1 -- 172.21.15.49:0/1878820473 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547401d070 con 0x7f54841007c0 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.233+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548419e690 con 0x7f54841007c0 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.233+0000 7f547a7fc700 1 -- 172.21.15.49:0/1878820473 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5474004090 con 0x7f54841007c0 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.233+0000 7f547a7fc700 1 -- 172.21.15.49:0/1878820473 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474022d40 con 0x7f54841007c0 2023-10-08T09:13:35.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.233+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548419eb40 con 0x7f54841007c0 2023-10-08T09:13:35.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.233+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5484106aa0 con 0x7f54841007c0 2023-10-08T09:13:35.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.235+0000 7f547a7fc700 1 -- 172.21.15.49:0/1878820473 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5474022470 con 0x7f54841007c0 2023-10-08T09:13:35.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.235+0000 7f547a7fc700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54700a1510 0x7f54700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.235+0000 7f54890c1700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54700a1510 0x7f54700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.235+0000 7f547a7fc700 1 -- 172.21.15.49:0/1878820473 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f54740ce530 con 0x7f54841007c0 2023-10-08T09:13:35.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.236+0000 7f54890c1700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54700a1510 0x7f54700a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5484100620 tx=0x7f5480007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:35.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.238+0000 7f547a7fc700 1 -- 172.21.15.49:0/1878820473 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f54740978d0 con 0x7f54841007c0 2023-10-08T09:13:35.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.444+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54840678a0 con 0x7f54841007c0 2023-10-08T09:13:35.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.445+0000 7f547a7fc700 1 -- 172.21.15.49:0/1878820473 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+23969 (secure 0 0 0) 0x7f5474027090 con 0x7f54841007c0 2023-10-08T09:13:35.447 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:35.447 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":86,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:08.274410+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:35.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54700a1510 msgr2=0x7f54700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54700a1510 0x7f54700a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5484100620 tx=0x7f5480007400 comp rx=0 tx=0).stop 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 msgr2=0x7f548419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f548419b350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54740036f0 tx=0x7f54740047b0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 shutdown_connections 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54700a1510 0x7f54700a39d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54840ff5d0 0x7f5484069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5484101160 0x7f548419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 --2- 172.21.15.49:0/1878820473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f54841007c0 0x7f548419b350 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 >> 172.21.15.49:0/1878820473 conn(0x7f54840fab40 msgr2=0x7f5484103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 shutdown_connections 2023-10-08T09:13:35.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.448+0000 7f548b325700 1 -- 172.21.15.49:0/1878820473 wait complete. 2023-10-08T09:13:35.462 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T09:13:35.462 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:13:35.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 -- 172.21.15.49:0/21787795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970101130 msgr2=0x7fb9701079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 --2- 172.21.15.49:0/21787795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970101130 0x7fb9701079e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb96800b0d0 tx=0x7fb96800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 -- 172.21.15.49:0/21787795 shutdown_connections 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 --2- 172.21.15.49:0/21787795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb970101130 0x7fb9701079e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 --2- 172.21.15.49:0/21787795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9700ff5a0 0x7fb9700ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 --2- 172.21.15.49:0/21787795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb970100790 0x7fb970100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 -- 172.21.15.49:0/21787795 >> 172.21.15.49:0/21787795 conn(0x7fb9700fab10 msgr2=0x7fb9700fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 -- 172.21.15.49:0/21787795 shutdown_connections 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.586+0000 7fb975bb2700 1 -- 172.21.15.49:0/21787795 wait complete. 2023-10-08T09:13:35.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 Processor -- start 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 -- start start 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9700ff5a0 0x7fb970103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb970100790 0x7fb970101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb970101130 0x7fb9701020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9701027b0 con 0x7fb9700ff5a0 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb970102920 con 0x7fb970100790 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb975bb2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9701a0800 con 0x7fb970101130 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb96f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9700ff5a0 0x7fb970103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb96f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9700ff5a0 0x7fb970103490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48444/0 (socket says 172.21.15.49:48444) 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb96f7fe700 1 -- 172.21.15.49:0/606491962 learned_addr learned my addr 172.21.15.49:0/606491962 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:35.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb96ffff700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb970101130 0x7fb9701020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.587+0000 7fb96effd700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb970100790 0x7fb970101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96f7fe700 1 -- 172.21.15.49:0/606491962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb970101130 msgr2=0x7fb9701020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96f7fe700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb970101130 0x7fb9701020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96f7fe700 1 -- 172.21.15.49:0/606491962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb970100790 msgr2=0x7fb970101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96f7fe700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb970100790 0x7fb970101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96f7fe700 1 -- 172.21.15.49:0/606491962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb968009d00 con 0x7fb9700ff5a0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96f7fe700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9700ff5a0 0x7fb970103490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb96000c970 tx=0x7fb96000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96cff9700 1 -- 172.21.15.49:0/606491962 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960004220 con 0x7fb9700ff5a0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96cff9700 1 -- 172.21.15.49:0/606491962 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb960004380 con 0x7fb9700ff5a0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9701a0a60 con 0x7fb9700ff5a0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb96cff9700 1 -- 172.21.15.49:0/606491962 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960003b50 con 0x7fb9700ff5a0 2023-10-08T09:13:35.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.588+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9701a0fc0 con 0x7fb9700ff5a0 2023-10-08T09:13:35.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.589+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb970067450 con 0x7fb9700ff5a0 2023-10-08T09:13:35.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.590+0000 7fb96cff9700 1 -- 172.21.15.49:0/606491962 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fb960003cf0 con 0x7fb9700ff5a0 2023-10-08T09:13:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.591+0000 7fb96cff9700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb9580a1590 0x7fb9580a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.591+0000 7fb96effd700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb9580a1590 0x7fb9580a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.591+0000 7fb96cff9700 1 -- 172.21.15.49:0/606491962 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb9600ccb00 con 0x7fb9700ff5a0 2023-10-08T09:13:35.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.591+0000 7fb96effd700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb9580a1590 0x7fb9580a3a50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb964004630 tx=0x7fb96400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:35.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.595+0000 7fb96cff9700 1 -- 172.21.15.49:0/606491962 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb9600cee60 con 0x7fb9700ff5a0 2023-10-08T09:13:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:35 smithi184 ceph-mon[103675]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:35 smithi184 ceph-mon[103675]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2023-10-08T09:13:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:35 smithi184 ceph-mon[103675]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:35 smithi184 ceph-mon[103675]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-08T09:13:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:35 smithi184 ceph-mon[103675]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:35 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4084469350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.792+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9700fccf0 con 0x7fb9580a1590 2023-10-08T09:13:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:35 smithi132 ceph-mon[103389]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:35 smithi132 ceph-mon[103389]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2023-10-08T09:13:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:35 smithi132 ceph-mon[103389]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:35 smithi132 ceph-mon[103389]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-08T09:13:35.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:35 smithi132 ceph-mon[103389]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:35 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4084469350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:35.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.808+0000 7fb96cff9700 1 -- 172.21.15.49:0/606491962 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422511 (secure 0 0 0) 0x7fb9700fccf0 con 0x7fb9580a1590 2023-10-08T09:13:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:35.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb9580a1590 msgr2=0x7fb9580a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb9580a1590 0x7fb9580a3a50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb964004630 tx=0x7fb96400b3c0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9700ff5a0 msgr2=0x7fb970103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:35.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9700ff5a0 0x7fb970103490 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb96000c970 tx=0x7fb96000cc80 comp rx=0 tx=0).stop 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 shutdown_connections 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb9580a1590 0x7fb9580a3a50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb970101130 0x7fb9701020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb9700ff5a0 0x7fb970103490 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 --2- 172.21.15.49:0/606491962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb970100790 0x7fb970101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 >> 172.21.15.49:0/606491962 conn(0x7fb9700fab10 msgr2=0x7fb9700fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 shutdown_connections 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:35.813+0000 7fb975bb2700 1 -- 172.21.15.49:0/606491962 wait complete. 2023-10-08T09:13:35.832 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-10-08T09:13:35.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:35 smithi049 ceph-mon[99485]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:35 smithi049 ceph-mon[99485]: pgmap v285: 129 pgs: 129 active+clean; 584 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2023-10-08T09:13:35.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:35 smithi049 ceph-mon[99485]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:35 smithi049 ceph-mon[99485]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-10-08T09:13:35.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:35 smithi049 ceph-mon[99485]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:13:35.836 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:35 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4084469350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:13:35.838 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2023-10-08T09:13:33.859044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598368,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":149,"num_read_kb":127,"num_write":305,"num_write_kb":2989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335440,"kb_used_data":11240,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606384,"statfs":{"total":1151940427776,"available":1151596937216,"internally_reserved":0,"allocated":11509760,"data_stored":7355195,"data_compressed":248499,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3501,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":77,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004024"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.853653+0000","last_change":"2023-10-08T09:13:02.831436+0000","last_active":"2023-10-08T09:13:07.853653+0000","last_peered":"2023-10-08T09:13:07.853653+0000","last_clean":"2023-10-08T09:13:07.853653+0000","last_became_active":"2023-10-08T09:13:02.831123+0000","last_became_peered":"2023-10-08T09:13:02.831123+0000","last_unstale":"2023-10-08T09:13:07.853653+0000","last_undegraded":"2023-10-08T09:13:07.853653+0000","last_fullsized":"2023-10-08T09:13:07.853653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:33.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856370+0000","last_change":"2023-10-08T09:13:02.832286+0000","last_active":"2023-10-08T09:13:07.856370+0000","last_peered":"2023-10-08T09:13:07.856370+0000","last_clean":"2023-10-08T09:13:07.856370+0000","last_became_active":"2023-10-08T09:13:02.831881+0000","last_became_peered":"2023-10-08T09:13:02.831881+0000","last_unstale":"2023-10-08T09:13:07.856370+0000","last_undegraded":"2023-10-08T09:13:07.856370+0000","last_fullsized":"2023-10-08T09:13:07.856370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:31.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856154+0000","last_change":"2023-10-08T09:13:02.831454+0000","last_active":"2023-10-08T09:13:07.856154+0000","last_peered":"2023-10-08T09:13:07.856154+0000","last_clean":"2023-10-08T09:13:07.856154+0000","last_became_active":"2023-10-08T09:13:02.831040+0000","last_became_peered":"2023-10-08T09:13:02.831040+0000","last_unstale":"2023-10-08T09:13:07.856154+0000","last_undegraded":"2023-10-08T09:13:07.856154+0000","last_fullsized":"2023-10-08T09:13:07.856154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:16:02.788279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852354+0000","last_change":"2023-10-08T09:13:02.832699+0000","last_active":"2023-10-08T09:13:07.852354+0000","last_peered":"2023-10-08T09:13:07.852354+0000","last_clean":"2023-10-08T09:13:07.852354+0000","last_became_active":"2023-10-08T09:13:02.832345+0000","last_became_peered":"2023-10-08T09:13:02.832345+0000","last_unstale":"2023-10-08T09:13:07.852354+0000","last_undegraded":"2023-10-08T09:13:07.852354+0000","last_fullsized":"2023-10-08T09:13:07.852354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:01.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851384+0000","last_change":"2023-10-08T09:13:02.827767+0000","last_active":"2023-10-08T09:13:07.851384+0000","last_peered":"2023-10-08T09:13:07.851384+0000","last_clean":"2023-10-08T09:13:07.851384+0000","last_became_active":"2023-10-08T09:13:02.827454+0000","last_became_peered":"2023-10-08T09:13:02.827454+0000","last_unstale":"2023-10-08T09:13:07.851384+0000","last_undegraded":"2023-10-08T09:13:07.851384+0000","last_fullsized":"2023-10-08T09:13:07.851384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:43.122347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.094802+0000","last_change":"2023-10-08T09:13:02.828989+0000","last_active":"2023-10-08T09:13:08.094802+0000","last_peered":"2023-10-08T09:13:08.094802+0000","last_clean":"2023-10-08T09:13:08.094802+0000","last_became_active":"2023-10-08T09:13:02.828648+0000","last_became_peered":"2023-10-08T09:13:02.828648+0000","last_unstale":"2023-10-08T09:13:08.094802+0000","last_undegraded":"2023-10-08T09:13:08.094802+0000","last_fullsized":"2023-10-08T09:13:08.094802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:15:33.751851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852032+0000","last_change":"2023-10-08T09:13:02.831765+0000","last_active":"2023-10-08T09:13:07.852032+0000","last_peered":"2023-10-08T09:13:07.852032+0000","last_clean":"2023-10-08T09:13:07.852032+0000","last_became_active":"2023-10-08T09:13:02.831446+0000","last_became_peered":"2023-10-08T09:13:02.831446+0000","last_unstale":"2023-10-08T09:13:07.852032+0000","last_undegraded":"2023-10-08T09:13:07.852032+0000","last_fullsized":"2023-10-08T09:13:07.852032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:41.463161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867118+0000","last_change":"2023-10-08T09:13:02.817586+0000","last_active":"2023-10-08T09:13:07.867118+0000","last_peered":"2023-10-08T09:13:07.867118+0000","last_clean":"2023-10-08T09:13:07.867118+0000","last_became_active":"2023-10-08T09:13:02.817231+0000","last_became_peered":"2023-10-08T09:13:02.817231+0000","last_unstale":"2023-10-08T09:13:07.867118+0000","last_undegraded":"2023-10-08T09:13:07.867118+0000","last_fullsized":"2023-10-08T09:13:07.867118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:23.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.846396+0000","last_change":"2023-10-08T09:13:02.831925+0000","last_active":"2023-10-08T09:13:07.846396+0000","last_peered":"2023-10-08T09:13:07.846396+0000","last_clean":"2023-10-08T09:13:07.846396+0000","last_became_active":"2023-10-08T09:13:02.831625+0000","last_became_peered":"2023-10-08T09:13:02.831625+0000","last_unstale":"2023-10-08T09:13:07.846396+0000","last_undegraded":"2023-10-08T09:13:07.846396+0000","last_fullsized":"2023-10-08T09:13:07.846396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:39.115740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.095335+0000","last_change":"2023-10-08T09:13:02.831752+0000","last_active":"2023-10-08T09:13:08.095335+0000","last_peered":"2023-10-08T09:13:08.095335+0000","last_clean":"2023-10-08T09:13:08.095335+0000","last_became_active":"2023-10-08T09:13:02.831468+0000","last_became_peered":"2023-10-08T09:13:02.831468+0000","last_unstale":"2023-10-08T09:13:08.095335+0000","last_undegraded":"2023-10-08T09:13:08.095335+0000","last_fullsized":"2023-10-08T09:13:08.095335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:05.758017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852576+0000","last_change":"2023-10-08T09:13:02.828838+0000","last_active":"2023-10-08T09:13:07.852576+0000","last_peered":"2023-10-08T09:13:07.852576+0000","last_clean":"2023-10-08T09:13:07.852576+0000","last_became_active":"2023-10-08T09:13:02.828598+0000","last_became_peered":"2023-10-08T09:13:02.828598+0000","last_unstale":"2023-10-08T09:13:07.852576+0000","last_undegraded":"2023-10-08T09:13:07.852576+0000","last_fullsized":"2023-10-08T09:13:07.852576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:04.317649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852193+0000","last_change":"2023-10-08T09:13:02.831730+0000","last_active":"2023-10-08T09:13:07.852193+0000","last_peered":"2023-10-08T09:13:07.852193+0000","last_clean":"2023-10-08T09:13:07.852193+0000","last_became_active":"2023-10-08T09:13:02.831404+0000","last_became_peered":"2023-10-08T09:13:02.831404+0000","last_unstale":"2023-10-08T09:13:07.852193+0000","last_undegraded":"2023-10-08T09:13:07.852193+0000","last_fullsized":"2023-10-08T09:13:07.852193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:22.067908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845131+0000","last_change":"2023-10-08T09:13:02.832869+0000","last_active":"2023-10-08T09:13:07.845131+0000","last_peered":"2023-10-08T09:13:07.845131+0000","last_clean":"2023-10-08T09:13:07.845131+0000","last_became_active":"2023-10-08T09:13:02.832332+0000","last_became_peered":"2023-10-08T09:13:02.832332+0000","last_unstale":"2023-10-08T09:13:07.845131+0000","last_undegraded":"2023-10-08T09:13:07.845131+0000","last_fullsized":"2023-10-08T09:13:07.845131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:45:54.609041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856060+0000","last_change":"2023-10-08T09:13:02.829027+0000","last_active":"2023-10-08T09:13:07.856060+0000","last_peered":"2023-10-08T09:13:07.856060+0000","last_clean":"2023-10-08T09:13:07.856060+0000","last_became_active":"2023-10-08T09:13:02.828714+0000","last_became_peered":"2023-10-08T09:13:02.828714+0000","last_unstale":"2023-10-08T09:13:07.856060+0000","last_undegraded":"2023-10-08T09:13:07.856060+0000","last_fullsized":"2023-10-08T09:13:07.856060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:13.121274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856252+0000","last_change":"2023-10-08T09:13:02.838640+0000","last_active":"2023-10-08T09:13:07.856252+0000","last_peered":"2023-10-08T09:13:07.856252+0000","last_clean":"2023-10-08T09:13:07.856252+0000","last_became_active":"2023-10-08T09:13:02.838302+0000","last_became_peered":"2023-10-08T09:13:02.838302+0000","last_unstale":"2023-10-08T09:13:07.856252+0000","last_undegraded":"2023-10-08T09:13:07.856252+0000","last_fullsized":"2023-10-08T09:13:07.856252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:54.268315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845967+0000","last_change":"2023-10-08T09:13:02.834027+0000","last_active":"2023-10-08T09:13:07.845967+0000","last_peered":"2023-10-08T09:13:07.845967+0000","last_clean":"2023-10-08T09:13:07.845967+0000","last_became_active":"2023-10-08T09:13:02.833733+0000","last_became_peered":"2023-10-08T09:13:02.833733+0000","last_unstale":"2023-10-08T09:13:07.845967+0000","last_undegraded":"2023-10-08T09:13:07.845967+0000","last_fullsized":"2023-10-08T09:13:07.845967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:03.644640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.853559+0000","last_change":"2023-10-08T09:13:02.832040+0000","last_active":"2023-10-08T09:13:07.853559+0000","last_peered":"2023-10-08T09:13:07.853559+0000","last_clean":"2023-10-08T09:13:07.853559+0000","last_became_active":"2023-10-08T09:13:02.831752+0000","last_became_peered":"2023-10-08T09:13:02.831752+0000","last_unstale":"2023-10-08T09:13:07.853559+0000","last_undegraded":"2023-10-08T09:13:07.853559+0000","last_fullsized":"2023-10-08T09:13:07.853559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:38.938703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852839+0000","last_change":"2023-10-08T09:13:02.830535+0000","last_active":"2023-10-08T09:13:07.852839+0000","last_peered":"2023-10-08T09:13:07.852839+0000","last_clean":"2023-10-08T09:13:07.852839+0000","last_became_active":"2023-10-08T09:13:02.830144+0000","last_became_peered":"2023-10-08T09:13:02.830144+0000","last_unstale":"2023-10-08T09:13:07.852839+0000","last_undegraded":"2023-10-08T09:13:07.852839+0000","last_fullsized":"2023-10-08T09:13:07.852839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:33.850815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847951+0000","last_change":"2023-10-08T09:13:02.830938+0000","last_active":"2023-10-08T09:13:07.847951+0000","last_peered":"2023-10-08T09:13:07.847951+0000","last_clean":"2023-10-08T09:13:07.847951+0000","last_became_active":"2023-10-08T09:13:02.830551+0000","last_became_peered":"2023-10-08T09:13:02.830551+0000","last_unstale":"2023-10-08T09:13:07.847951+0000","last_undegraded":"2023-10-08T09:13:07.847951+0000","last_fullsized":"2023-10-08T09:13:07.847951+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:26.894614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066152+0000","last_change":"2023-10-08T09:13:07.871322+0000","last_active":"2023-10-08T09:13:09.066152+0000","last_peered":"2023-10-08T09:13:09.066152+0000","last_clean":"2023-10-08T09:13:09.066152+0000","last_became_active":"2023-10-08T09:13:07.870828+0000","last_became_peered":"2023-10-08T09:13:07.870828+0000","last_unstale":"2023-10-08T09:13:09.066152+0000","last_undegraded":"2023-10-08T09:13:09.066152+0000","last_fullsized":"2023-10-08T09:13:09.066152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:37.145705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.853372+0000","last_change":"2023-10-08T09:13:02.373510+0000","last_active":"2023-10-08T09:13:07.853372+0000","last_peered":"2023-10-08T09:13:07.853372+0000","last_clean":"2023-10-08T09:13:07.853372+0000","last_became_active":"2023-10-08T09:13:02.373174+0000","last_became_peered":"2023-10-08T09:13:02.373174+0000","last_unstale":"2023-10-08T09:13:07.853372+0000","last_undegraded":"2023-10-08T09:13:07.853372+0000","last_fullsized":"2023-10-08T09:13:07.853372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867265+0000","last_change":"2023-10-08T09:13:02.835195+0000","last_active":"2023-10-08T09:13:07.867265+0000","last_peered":"2023-10-08T09:13:07.867265+0000","last_clean":"2023-10-08T09:13:07.867265+0000","last_became_active":"2023-10-08T09:13:02.834893+0000","last_became_peered":"2023-10-08T09:13:02.834893+0000","last_unstale":"2023-10-08T09:13:07.867265+0000","last_undegraded":"2023-10-08T09:13:07.867265+0000","last_fullsized":"2023-10-08T09:13:07.867265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:35.835017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856567+0000","last_change":"2023-10-08T09:13:07.871418+0000","last_active":"2023-10-08T09:13:08.856567+0000","last_peered":"2023-10-08T09:13:08.856567+0000","last_clean":"2023-10-08T09:13:08.856567+0000","last_became_active":"2023-10-08T09:13:07.870961+0000","last_became_peered":"2023-10-08T09:13:07.870961+0000","last_unstale":"2023-10-08T09:13:08.856567+0000","last_undegraded":"2023-10-08T09:13:08.856567+0000","last_fullsized":"2023-10-08T09:13:08.856567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:31.723498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857106+0000","last_change":"2023-10-08T09:13:02.370728+0000","last_active":"2023-10-08T09:13:08.857106+0000","last_peered":"2023-10-08T09:13:08.857106+0000","last_clean":"2023-10-08T09:13:08.857106+0000","last_became_active":"2023-10-08T09:13:02.370396+0000","last_became_peered":"2023-10-08T09:13:02.370396+0000","last_unstale":"2023-10-08T09:13:08.857106+0000","last_undegraded":"2023-10-08T09:13:08.857106+0000","last_fullsized":"2023-10-08T09:13:08.857106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:32.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858747+0000","last_change":"2023-10-08T09:13:02.827823+0000","last_active":"2023-10-08T09:13:07.858747+0000","last_peered":"2023-10-08T09:13:07.858747+0000","last_clean":"2023-10-08T09:13:07.858747+0000","last_became_active":"2023-10-08T09:13:02.827417+0000","last_became_peered":"2023-10-08T09:13:02.827417+0000","last_unstale":"2023-10-08T09:13:07.858747+0000","last_undegraded":"2023-10-08T09:13:07.858747+0000","last_fullsized":"2023-10-08T09:13:07.858747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:38.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856926+0000","last_change":"2023-10-08T09:13:07.868912+0000","last_active":"2023-10-08T09:13:08.856926+0000","last_peered":"2023-10-08T09:13:08.856926+0000","last_clean":"2023-10-08T09:13:08.856926+0000","last_became_active":"2023-10-08T09:13:07.868504+0000","last_became_peered":"2023-10-08T09:13:07.868504+0000","last_unstale":"2023-10-08T09:13:08.856926+0000","last_undegraded":"2023-10-08T09:13:08.856926+0000","last_fullsized":"2023-10-08T09:13:08.856926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:04.686985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.420183+0000","last_change":"2023-10-08T09:13:02.375244+0000","last_active":"2023-10-08T09:13:31.420183+0000","last_peered":"2023-10-08T09:13:31.420183+0000","last_clean":"2023-10-08T09:13:31.420183+0000","last_became_active":"2023-10-08T09:13:02.374526+0000","last_became_peered":"2023-10-08T09:13:02.374526+0000","last_unstale":"2023-10-08T09:13:31.420183+0000","last_undegraded":"2023-10-08T09:13:31.420183+0000","last_fullsized":"2023-10-08T09:13:31.420183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:36.380491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845834+0000","last_change":"2023-10-08T09:13:02.838373+0000","last_active":"2023-10-08T09:13:07.845834+0000","last_peered":"2023-10-08T09:13:07.845834+0000","last_clean":"2023-10-08T09:13:07.845834+0000","last_became_active":"2023-10-08T09:13:02.838100+0000","last_became_peered":"2023-10-08T09:13:02.838100+0000","last_unstale":"2023-10-08T09:13:07.845834+0000","last_undegraded":"2023-10-08T09:13:07.845834+0000","last_fullsized":"2023-10-08T09:13:07.845834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:59.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856678+0000","last_change":"2023-10-08T09:13:07.868929+0000","last_active":"2023-10-08T09:13:08.856678+0000","last_peered":"2023-10-08T09:13:08.856678+0000","last_clean":"2023-10-08T09:13:08.856678+0000","last_became_active":"2023-10-08T09:13:07.868451+0000","last_became_peered":"2023-10-08T09:13:07.868451+0000","last_unstale":"2023-10-08T09:13:08.856678+0000","last_undegraded":"2023-10-08T09:13:08.856678+0000","last_fullsized":"2023-10-08T09:13:08.856678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:36.540319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.855228+0000","last_change":"2023-10-08T09:13:02.366404+0000","last_active":"2023-10-08T09:13:07.855228+0000","last_peered":"2023-10-08T09:13:07.855228+0000","last_clean":"2023-10-08T09:13:07.855228+0000","last_became_active":"2023-10-08T09:13:02.366025+0000","last_became_peered":"2023-10-08T09:13:02.366025+0000","last_unstale":"2023-10-08T09:13:07.855228+0000","last_undegraded":"2023-10-08T09:13:07.855228+0000","last_fullsized":"2023-10-08T09:13:07.855228+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:51.916565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858268+0000","last_change":"2023-10-08T09:13:02.832852+0000","last_active":"2023-10-08T09:13:07.858268+0000","last_peered":"2023-10-08T09:13:07.858268+0000","last_clean":"2023-10-08T09:13:07.858268+0000","last_became_active":"2023-10-08T09:13:02.832530+0000","last_became_peered":"2023-10-08T09:13:02.832530+0000","last_unstale":"2023-10-08T09:13:07.858268+0000","last_undegraded":"2023-10-08T09:13:07.858268+0000","last_fullsized":"2023-10-08T09:13:07.858268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:20.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066518+0000","last_change":"2023-10-08T09:13:07.863864+0000","last_active":"2023-10-08T09:13:09.066518+0000","last_peered":"2023-10-08T09:13:09.066518+0000","last_clean":"2023-10-08T09:13:09.066518+0000","last_became_active":"2023-10-08T09:13:07.863509+0000","last_became_peered":"2023-10-08T09:13:07.863509+0000","last_unstale":"2023-10-08T09:13:09.066518+0000","last_undegraded":"2023-10-08T09:13:09.066518+0000","last_fullsized":"2023-10-08T09:13:09.066518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:22.933327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.413622+0000","last_change":"2023-10-08T09:13:02.374215+0000","last_active":"2023-10-08T09:13:32.413622+0000","last_peered":"2023-10-08T09:13:32.413622+0000","last_clean":"2023-10-08T09:13:32.413622+0000","last_became_active":"2023-10-08T09:13:02.373912+0000","last_became_peered":"2023-10-08T09:13:02.373912+0000","last_unstale":"2023-10-08T09:13:32.413622+0000","last_undegraded":"2023-10-08T09:13:32.413622+0000","last_fullsized":"2023-10-08T09:13:32.413622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:18.315244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852561+0000","last_change":"2023-10-08T09:13:02.830651+0000","last_active":"2023-10-08T09:13:07.852561+0000","last_peered":"2023-10-08T09:13:07.852561+0000","last_clean":"2023-10-08T09:13:07.852561+0000","last_became_active":"2023-10-08T09:13:02.830324+0000","last_became_peered":"2023-10-08T09:13:02.830324+0000","last_unstale":"2023-10-08T09:13:07.852561+0000","last_undegraded":"2023-10-08T09:13:07.852561+0000","last_fullsized":"2023-10-08T09:13:07.852561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:02.143794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066375+0000","last_change":"2023-10-08T09:13:07.862270+0000","last_active":"2023-10-08T09:13:09.066375+0000","last_peered":"2023-10-08T09:13:09.066375+0000","last_clean":"2023-10-08T09:13:09.066375+0000","last_became_active":"2023-10-08T09:13:07.861847+0000","last_became_peered":"2023-10-08T09:13:07.861847+0000","last_unstale":"2023-10-08T09:13:09.066375+0000","last_undegraded":"2023-10-08T09:13:09.066375+0000","last_fullsized":"2023-10-08T09:13:09.066375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:33.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"80'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.094786+0000","last_change":"2023-10-08T09:13:02.370334+0000","last_active":"2023-10-08T09:13:08.094786+0000","last_peered":"2023-10-08T09:13:08.094786+0000","last_clean":"2023-10-08T09:13:08.094786+0000","last_became_active":"2023-10-08T09:13:02.370068+0000","last_became_peered":"2023-10-08T09:13:02.370068+0000","last_unstale":"2023-10-08T09:13:08.094786+0000","last_undegraded":"2023-10-08T09:13:08.094786+0000","last_fullsized":"2023-10-08T09:13:08.094786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.962779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.855103+0000","last_change":"2023-10-08T09:13:02.832346+0000","last_active":"2023-10-08T09:13:07.855103+0000","last_peered":"2023-10-08T09:13:07.855103+0000","last_clean":"2023-10-08T09:13:07.855103+0000","last_became_active":"2023-10-08T09:13:02.832059+0000","last_became_peered":"2023-10-08T09:13:02.832059+0000","last_unstale":"2023-10-08T09:13:07.855103+0000","last_undegraded":"2023-10-08T09:13:07.855103+0000","last_fullsized":"2023-10-08T09:13:07.855103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:30.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197219+0000","last_change":"2023-10-08T09:13:07.863466+0000","last_active":"2023-10-08T09:13:09.197219+0000","last_peered":"2023-10-08T09:13:09.197219+0000","last_clean":"2023-10-08T09:13:09.197219+0000","last_became_active":"2023-10-08T09:13:07.863015+0000","last_became_peered":"2023-10-08T09:13:07.863015+0000","last_unstale":"2023-10-08T09:13:09.197219+0000","last_undegraded":"2023-10-08T09:13:09.197219+0000","last_fullsized":"2023-10-08T09:13:09.197219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:30.316625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844076+0000","last_change":"2023-10-08T09:13:02.369444+0000","last_active":"2023-10-08T09:13:07.844076+0000","last_peered":"2023-10-08T09:13:07.844076+0000","last_clean":"2023-10-08T09:13:07.844076+0000","last_became_active":"2023-10-08T09:13:02.369142+0000","last_became_peered":"2023-10-08T09:13:02.369142+0000","last_unstale":"2023-10-08T09:13:07.844076+0000","last_undegraded":"2023-10-08T09:13:07.844076+0000","last_fullsized":"2023-10-08T09:13:07.844076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:28.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867230+0000","last_change":"2023-10-08T09:13:02.836125+0000","last_active":"2023-10-08T09:13:07.867230+0000","last_peered":"2023-10-08T09:13:07.867230+0000","last_clean":"2023-10-08T09:13:07.867230+0000","last_became_active":"2023-10-08T09:13:02.835809+0000","last_became_peered":"2023-10-08T09:13:02.835809+0000","last_unstale":"2023-10-08T09:13:07.867230+0000","last_undegraded":"2023-10-08T09:13:07.867230+0000","last_fullsized":"2023-10-08T09:13:07.867230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:15.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.198700+0000","last_change":"2023-10-08T09:13:07.861633+0000","last_active":"2023-10-08T09:13:09.198700+0000","last_peered":"2023-10-08T09:13:09.198700+0000","last_clean":"2023-10-08T09:13:09.198700+0000","last_became_active":"2023-10-08T09:13:07.861168+0000","last_became_peered":"2023-10-08T09:13:07.861168+0000","last_unstale":"2023-10-08T09:13:09.198700+0000","last_undegraded":"2023-10-08T09:13:09.198700+0000","last_fullsized":"2023-10-08T09:13:09.198700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:47.283892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.424293+0000","last_change":"2023-10-08T09:13:02.365703+0000","last_active":"2023-10-08T09:13:32.424293+0000","last_peered":"2023-10-08T09:13:32.424293+0000","last_clean":"2023-10-08T09:13:32.424293+0000","last_became_active":"2023-10-08T09:13:02.365421+0000","last_became_peered":"2023-10-08T09:13:02.365421+0000","last_unstale":"2023-10-08T09:13:32.424293+0000","last_undegraded":"2023-10-08T09:13:32.424293+0000","last_fullsized":"2023-10-08T09:13:32.424293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:43.124511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.850748+0000","last_change":"2023-10-08T09:13:02.840941+0000","last_active":"2023-10-08T09:13:07.850748+0000","last_peered":"2023-10-08T09:13:07.850748+0000","last_clean":"2023-10-08T09:13:07.850748+0000","last_became_active":"2023-10-08T09:13:02.840605+0000","last_became_peered":"2023-10-08T09:13:02.840605+0000","last_unstale":"2023-10-08T09:13:07.850748+0000","last_undegraded":"2023-10-08T09:13:07.850748+0000","last_fullsized":"2023-10-08T09:13:07.850748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:24:13.417028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856661+0000","last_change":"2023-10-08T09:13:07.868950+0000","last_active":"2023-10-08T09:13:08.856661+0000","last_peered":"2023-10-08T09:13:08.856661+0000","last_clean":"2023-10-08T09:13:08.856661+0000","last_became_active":"2023-10-08T09:13:07.868542+0000","last_became_peered":"2023-10-08T09:13:07.868542+0000","last_unstale":"2023-10-08T09:13:08.856661+0000","last_undegraded":"2023-10-08T09:13:08.856661+0000","last_fullsized":"2023-10-08T09:13:08.856661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:11.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.095282+0000","last_change":"2023-10-08T09:13:02.376188+0000","last_active":"2023-10-08T09:13:08.095282+0000","last_peered":"2023-10-08T09:13:08.095282+0000","last_clean":"2023-10-08T09:13:08.095282+0000","last_became_active":"2023-10-08T09:13:02.375914+0000","last_became_peered":"2023-10-08T09:13:02.375914+0000","last_unstale":"2023-10-08T09:13:08.095282+0000","last_undegraded":"2023-10-08T09:13:08.095282+0000","last_fullsized":"2023-10-08T09:13:08.095282+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:10.744085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.846172+0000","last_change":"2023-10-08T09:13:02.833429+0000","last_active":"2023-10-08T09:13:07.846172+0000","last_peered":"2023-10-08T09:13:07.846172+0000","last_clean":"2023-10-08T09:13:07.846172+0000","last_became_active":"2023-10-08T09:13:02.832792+0000","last_became_peered":"2023-10-08T09:13:02.832792+0000","last_unstale":"2023-10-08T09:13:07.846172+0000","last_undegraded":"2023-10-08T09:13:07.846172+0000","last_fullsized":"2023-10-08T09:13:07.846172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:24.563447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196856+0000","last_change":"2023-10-08T09:13:07.862688+0000","last_active":"2023-10-08T09:13:09.196856+0000","last_peered":"2023-10-08T09:13:09.196856+0000","last_clean":"2023-10-08T09:13:09.196856+0000","last_became_active":"2023-10-08T09:13:07.862197+0000","last_became_peered":"2023-10-08T09:13:07.862197+0000","last_unstale":"2023-10-08T09:13:09.196856+0000","last_undegraded":"2023-10-08T09:13:09.196856+0000","last_fullsized":"2023-10-08T09:13:09.196856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:56.792045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:29.458142+0000","last_change":"2023-10-08T09:13:02.380283+0000","last_active":"2023-10-08T09:13:29.458142+0000","last_peered":"2023-10-08T09:13:29.458142+0000","last_clean":"2023-10-08T09:13:29.458142+0000","last_became_active":"2023-10-08T09:13:02.379844+0000","last_became_peered":"2023-10-08T09:13:02.379844+0000","last_unstale":"2023-10-08T09:13:29.458142+0000","last_undegraded":"2023-10-08T09:13:29.458142+0000","last_fullsized":"2023-10-08T09:13:29.458142+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:18.162437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858856+0000","last_change":"2023-10-08T09:13:02.839211+0000","last_active":"2023-10-08T09:13:07.858856+0000","last_peered":"2023-10-08T09:13:07.858856+0000","last_clean":"2023-10-08T09:13:07.858856+0000","last_became_active":"2023-10-08T09:13:02.838968+0000","last_became_peered":"2023-10-08T09:13:02.838968+0000","last_unstale":"2023-10-08T09:13:07.858856+0000","last_undegraded":"2023-10-08T09:13:07.858856+0000","last_fullsized":"2023-10-08T09:13:07.858856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:24:02.541749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857159+0000","last_change":"2023-10-08T09:13:07.857016+0000","last_active":"2023-10-08T09:13:08.857159+0000","last_peered":"2023-10-08T09:13:08.857159+0000","last_clean":"2023-10-08T09:13:08.857159+0000","last_became_active":"2023-10-08T09:13:07.856562+0000","last_became_peered":"2023-10-08T09:13:07.856562+0000","last_unstale":"2023-10-08T09:13:08.857159+0000","last_undegraded":"2023-10-08T09:13:08.857159+0000","last_fullsized":"2023-10-08T09:13:08.857159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:27.073675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.411103+0000","last_change":"2023-10-08T09:13:02.377027+0000","last_active":"2023-10-08T09:13:31.411103+0000","last_peered":"2023-10-08T09:13:31.411103+0000","last_clean":"2023-10-08T09:13:31.411103+0000","last_became_active":"2023-10-08T09:13:02.376746+0000","last_became_peered":"2023-10-08T09:13:02.376746+0000","last_unstale":"2023-10-08T09:13:31.411103+0000","last_undegraded":"2023-10-08T09:13:31.411103+0000","last_fullsized":"2023-10-08T09:13:31.411103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:19.062296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845910+0000","last_change":"2023-10-08T09:13:02.837279+0000","last_active":"2023-10-08T09:13:07.845910+0000","last_peered":"2023-10-08T09:13:07.845910+0000","last_clean":"2023-10-08T09:13:07.845910+0000","last_became_active":"2023-10-08T09:13:02.836966+0000","last_became_peered":"2023-10-08T09:13:02.836966+0000","last_unstale":"2023-10-08T09:13:07.845910+0000","last_undegraded":"2023-10-08T09:13:07.845910+0000","last_fullsized":"2023-10-08T09:13:07.845910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:25.462835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197967+0000","last_change":"2023-10-08T09:13:07.865538+0000","last_active":"2023-10-08T09:13:09.197967+0000","last_peered":"2023-10-08T09:13:09.197967+0000","last_clean":"2023-10-08T09:13:09.197967+0000","last_became_active":"2023-10-08T09:13:07.865118+0000","last_became_peered":"2023-10-08T09:13:07.865118+0000","last_unstale":"2023-10-08T09:13:09.197967+0000","last_undegraded":"2023-10-08T09:13:09.197967+0000","last_fullsized":"2023-10-08T09:13:09.197967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:32.167439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:20.426978+0000","last_change":"2023-10-08T09:13:02.378173+0000","last_active":"2023-10-08T09:13:20.426978+0000","last_peered":"2023-10-08T09:13:20.426978+0000","last_clean":"2023-10-08T09:13:20.426978+0000","last_became_active":"2023-10-08T09:13:02.377880+0000","last_became_peered":"2023-10-08T09:13:02.377880+0000","last_unstale":"2023-10-08T09:13:20.426978+0000","last_undegraded":"2023-10-08T09:13:20.426978+0000","last_fullsized":"2023-10-08T09:13:20.426978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851336+0000","last_change":"2023-10-08T09:13:02.829874+0000","last_active":"2023-10-08T09:13:07.851336+0000","last_peered":"2023-10-08T09:13:07.851336+0000","last_clean":"2023-10-08T09:13:07.851336+0000","last_became_active":"2023-10-08T09:13:02.829552+0000","last_became_peered":"2023-10-08T09:13:02.829552+0000","last_unstale":"2023-10-08T09:13:07.851336+0000","last_undegraded":"2023-10-08T09:13:07.851336+0000","last_fullsized":"2023-10-08T09:13:07.851336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:29.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857511+0000","last_change":"2023-10-08T09:13:07.869661+0000","last_active":"2023-10-08T09:13:08.857511+0000","last_peered":"2023-10-08T09:13:08.857511+0000","last_clean":"2023-10-08T09:13:08.857511+0000","last_became_active":"2023-10-08T09:13:07.869236+0000","last_became_peered":"2023-10-08T09:13:07.869236+0000","last_unstale":"2023-10-08T09:13:08.857511+0000","last_undegraded":"2023-10-08T09:13:08.857511+0000","last_fullsized":"2023-10-08T09:13:08.857511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:26.354461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:30.468325+0000","last_change":"2023-10-08T09:13:02.376941+0000","last_active":"2023-10-08T09:13:30.468325+0000","last_peered":"2023-10-08T09:13:30.468325+0000","last_clean":"2023-10-08T09:13:30.468325+0000","last_became_active":"2023-10-08T09:13:02.375810+0000","last_became_peered":"2023-10-08T09:13:02.375810+0000","last_unstale":"2023-10-08T09:13:30.468325+0000","last_undegraded":"2023-10-08T09:13:30.468325+0000","last_fullsized":"2023-10-08T09:13:30.468325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:17.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844323+0000","last_change":"2023-10-08T09:13:02.836781+0000","last_active":"2023-10-08T09:13:07.844323+0000","last_peered":"2023-10-08T09:13:07.844323+0000","last_clean":"2023-10-08T09:13:07.844323+0000","last_became_active":"2023-10-08T09:13:02.836493+0000","last_became_peered":"2023-10-08T09:13:02.836493+0000","last_unstale":"2023-10-08T09:13:07.844323+0000","last_undegraded":"2023-10-08T09:13:07.844323+0000","last_fullsized":"2023-10-08T09:13:07.844323+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:03.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196963+0000","last_change":"2023-10-08T09:13:07.859104+0000","last_active":"2023-10-08T09:13:09.196963+0000","last_peered":"2023-10-08T09:13:09.196963+0000","last_clean":"2023-10-08T09:13:09.196963+0000","last_became_active":"2023-10-08T09:13:07.858691+0000","last_became_peered":"2023-10-08T09:13:07.858691+0000","last_unstale":"2023-10-08T09:13:09.196963+0000","last_undegraded":"2023-10-08T09:13:09.196963+0000","last_fullsized":"2023-10-08T09:13:09.196963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:33.256044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.424682+0000","last_change":"2023-10-08T09:13:02.364472+0000","last_active":"2023-10-08T09:13:32.424682+0000","last_peered":"2023-10-08T09:13:32.424682+0000","last_clean":"2023-10-08T09:13:32.424682+0000","last_became_active":"2023-10-08T09:13:02.364130+0000","last_became_peered":"2023-10-08T09:13:02.364130+0000","last_unstale":"2023-10-08T09:13:32.424682+0000","last_undegraded":"2023-10-08T09:13:32.424682+0000","last_fullsized":"2023-10-08T09:13:32.424682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:18.156186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851785+0000","last_change":"2023-10-08T09:13:02.836119+0000","last_active":"2023-10-08T09:13:07.851785+0000","last_peered":"2023-10-08T09:13:07.851785+0000","last_clean":"2023-10-08T09:13:07.851785+0000","last_became_active":"2023-10-08T09:13:02.835548+0000","last_became_peered":"2023-10-08T09:13:02.835548+0000","last_unstale":"2023-10-08T09:13:07.851785+0000","last_undegraded":"2023-10-08T09:13:07.851785+0000","last_fullsized":"2023-10-08T09:13:07.851785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:34.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196717+0000","last_change":"2023-10-08T09:13:07.871447+0000","last_active":"2023-10-08T09:13:09.196717+0000","last_peered":"2023-10-08T09:13:09.196717+0000","last_clean":"2023-10-08T09:13:09.196717+0000","last_became_active":"2023-10-08T09:13:07.871018+0000","last_became_peered":"2023-10-08T09:13:07.871018+0000","last_unstale":"2023-10-08T09:13:09.196717+0000","last_undegraded":"2023-10-08T09:13:09.196717+0000","last_fullsized":"2023-10-08T09:13:09.196717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:34.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419512+0000","last_change":"2023-10-08T09:13:02.378954+0000","last_active":"2023-10-08T09:13:31.419512+0000","last_peered":"2023-10-08T09:13:31.419512+0000","last_clean":"2023-10-08T09:13:31.419512+0000","last_became_active":"2023-10-08T09:13:02.378635+0000","last_became_peered":"2023-10-08T09:13:02.378635+0000","last_unstale":"2023-10-08T09:13:31.419512+0000","last_undegraded":"2023-10-08T09:13:31.419512+0000","last_fullsized":"2023-10-08T09:13:31.419512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:09.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851405+0000","last_change":"2023-10-08T09:13:02.828732+0000","last_active":"2023-10-08T09:13:07.851405+0000","last_peered":"2023-10-08T09:13:07.851405+0000","last_clean":"2023-10-08T09:13:07.851405+0000","last_became_active":"2023-10-08T09:13:02.828449+0000","last_became_peered":"2023-10-08T09:13:02.828449+0000","last_unstale":"2023-10-08T09:13:07.851405+0000","last_undegraded":"2023-10-08T09:13:07.851405+0000","last_fullsized":"2023-10-08T09:13:07.851405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:09.883908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197375+0000","last_change":"2023-10-08T09:13:07.869610+0000","last_active":"2023-10-08T09:13:09.197375+0000","last_peered":"2023-10-08T09:13:09.197375+0000","last_clean":"2023-10-08T09:13:09.197375+0000","last_became_active":"2023-10-08T09:13:07.869225+0000","last_became_peered":"2023-10-08T09:13:07.869225+0000","last_unstale":"2023-10-08T09:13:09.197375+0000","last_undegraded":"2023-10-08T09:13:09.197375+0000","last_fullsized":"2023-10-08T09:13:09.197375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:35.225452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.407181+0000","last_change":"2023-10-08T09:13:02.375658+0000","last_active":"2023-10-08T09:13:31.407181+0000","last_peered":"2023-10-08T09:13:31.407181+0000","last_clean":"2023-10-08T09:13:31.407181+0000","last_became_active":"2023-10-08T09:13:02.375383+0000","last_became_peered":"2023-10-08T09:13:02.375383+0000","last_unstale":"2023-10-08T09:13:31.407181+0000","last_undegraded":"2023-10-08T09:13:31.407181+0000","last_fullsized":"2023-10-08T09:13:31.407181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:52.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851572+0000","last_change":"2023-10-08T09:13:02.827684+0000","last_active":"2023-10-08T09:13:07.851572+0000","last_peered":"2023-10-08T09:13:07.851572+0000","last_clean":"2023-10-08T09:13:07.851572+0000","last_became_active":"2023-10-08T09:13:02.827337+0000","last_became_peered":"2023-10-08T09:13:02.827337+0000","last_unstale":"2023-10-08T09:13:07.851572+0000","last_undegraded":"2023-10-08T09:13:07.851572+0000","last_fullsized":"2023-10-08T09:13:07.851572+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:37.435065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856874+0000","last_change":"2023-10-08T09:13:07.871441+0000","last_active":"2023-10-08T09:13:08.856874+0000","last_peered":"2023-10-08T09:13:08.856874+0000","last_clean":"2023-10-08T09:13:08.856874+0000","last_became_active":"2023-10-08T09:13:07.870968+0000","last_became_peered":"2023-10-08T09:13:07.870968+0000","last_unstale":"2023-10-08T09:13:08.856874+0000","last_undegraded":"2023-10-08T09:13:08.856874+0000","last_fullsized":"2023-10-08T09:13:08.856874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:48.746147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.934155+0000","last_change":"2023-10-08T09:13:02.372514+0000","last_active":"2023-10-08T09:13:08.934155+0000","last_peered":"2023-10-08T09:13:08.934155+0000","last_clean":"2023-10-08T09:13:08.934155+0000","last_became_active":"2023-10-08T09:13:02.372181+0000","last_became_peered":"2023-10-08T09:13:02.372181+0000","last_unstale":"2023-10-08T09:13:08.934155+0000","last_undegraded":"2023-10-08T09:13:08.934155+0000","last_fullsized":"2023-10-08T09:13:08.934155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:28.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851785+0000","last_change":"2023-10-08T09:13:02.835220+0000","last_active":"2023-10-08T09:13:07.851785+0000","last_peered":"2023-10-08T09:13:07.851785+0000","last_clean":"2023-10-08T09:13:07.851785+0000","last_became_active":"2023-10-08T09:13:02.834914+0000","last_became_peered":"2023-10-08T09:13:02.834914+0000","last_unstale":"2023-10-08T09:13:07.851785+0000","last_undegraded":"2023-10-08T09:13:07.851785+0000","last_fullsized":"2023-10-08T09:13:07.851785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:37.447418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.933840+0000","last_change":"2023-10-08T09:13:08.096146+0000","last_active":"2023-10-08T09:13:08.933840+0000","last_peered":"2023-10-08T09:13:08.933840+0000","last_clean":"2023-10-08T09:13:08.933840+0000","last_became_active":"2023-10-08T09:13:08.095703+0000","last_became_peered":"2023-10-08T09:13:08.095703+0000","last_unstale":"2023-10-08T09:13:08.933840+0000","last_undegraded":"2023-10-08T09:13:08.933840+0000","last_fullsized":"2023-10-08T09:13:08.933840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:37.511094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852390+0000","last_change":"2023-10-08T09:13:02.372836+0000","last_active":"2023-10-08T09:13:07.852390+0000","last_peered":"2023-10-08T09:13:07.852390+0000","last_clean":"2023-10-08T09:13:07.852390+0000","last_became_active":"2023-10-08T09:13:02.372488+0000","last_became_peered":"2023-10-08T09:13:02.372488+0000","last_unstale":"2023-10-08T09:13:07.852390+0000","last_undegraded":"2023-10-08T09:13:07.852390+0000","last_fullsized":"2023-10-08T09:13:07.852390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:19.269539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.095649+0000","last_change":"2023-10-08T09:13:02.827306+0000","last_active":"2023-10-08T09:13:08.095649+0000","last_peered":"2023-10-08T09:13:08.095649+0000","last_clean":"2023-10-08T09:13:08.095649+0000","last_became_active":"2023-10-08T09:13:02.826965+0000","last_became_peered":"2023-10-08T09:13:02.826965+0000","last_unstale":"2023-10-08T09:13:08.095649+0000","last_undegraded":"2023-10-08T09:13:08.095649+0000","last_fullsized":"2023-10-08T09:13:08.095649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:27.513860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197872+0000","last_change":"2023-10-08T09:13:07.862637+0000","last_active":"2023-10-08T09:13:09.197872+0000","last_peered":"2023-10-08T09:13:09.197872+0000","last_clean":"2023-10-08T09:13:09.197872+0000","last_became_active":"2023-10-08T09:13:07.862117+0000","last_became_peered":"2023-10-08T09:13:07.862117+0000","last_unstale":"2023-10-08T09:13:09.197872+0000","last_undegraded":"2023-10-08T09:13:09.197872+0000","last_fullsized":"2023-10-08T09:13:09.197872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:14.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197677+0000","last_change":"2023-10-08T09:13:02.379585+0000","last_active":"2023-10-08T09:13:09.197677+0000","last_peered":"2023-10-08T09:13:09.197677+0000","last_clean":"2023-10-08T09:13:09.197677+0000","last_became_active":"2023-10-08T09:13:02.379324+0000","last_became_peered":"2023-10-08T09:13:02.379324+0000","last_unstale":"2023-10-08T09:13:09.197677+0000","last_undegraded":"2023-10-08T09:13:09.197677+0000","last_fullsized":"2023-10-08T09:13:09.197677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:31.609558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.094946+0000","last_change":"2023-10-08T09:13:02.833416+0000","last_active":"2023-10-08T09:13:08.094946+0000","last_peered":"2023-10-08T09:13:08.094946+0000","last_clean":"2023-10-08T09:13:08.094946+0000","last_became_active":"2023-10-08T09:13:02.833040+0000","last_became_peered":"2023-10-08T09:13:02.833040+0000","last_unstale":"2023-10-08T09:13:08.094946+0000","last_undegraded":"2023-10-08T09:13:08.094946+0000","last_fullsized":"2023-10-08T09:13:08.094946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:44.480445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196474+0000","last_change":"2023-10-08T09:13:08.095723+0000","last_active":"2023-10-08T09:13:09.196474+0000","last_peered":"2023-10-08T09:13:09.196474+0000","last_clean":"2023-10-08T09:13:09.196474+0000","last_became_active":"2023-10-08T09:13:08.095271+0000","last_became_peered":"2023-10-08T09:13:08.095271+0000","last_unstale":"2023-10-08T09:13:09.196474+0000","last_undegraded":"2023-10-08T09:13:09.196474+0000","last_fullsized":"2023-10-08T09:13:09.196474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:04.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858052+0000","last_change":"2023-10-08T09:13:02.829663+0000","last_active":"2023-10-08T09:13:07.858052+0000","last_peered":"2023-10-08T09:13:07.858052+0000","last_clean":"2023-10-08T09:13:07.858052+0000","last_became_active":"2023-10-08T09:13:02.829354+0000","last_became_peered":"2023-10-08T09:13:02.829354+0000","last_unstale":"2023-10-08T09:13:07.858052+0000","last_undegraded":"2023-10-08T09:13:07.858052+0000","last_fullsized":"2023-10-08T09:13:07.858052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:55.332346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196485+0000","last_change":"2023-10-08T09:13:02.372114+0000","last_active":"2023-10-08T09:13:09.196485+0000","last_peered":"2023-10-08T09:13:09.196485+0000","last_clean":"2023-10-08T09:13:09.196485+0000","last_became_active":"2023-10-08T09:13:02.371603+0000","last_became_peered":"2023-10-08T09:13:02.371603+0000","last_unstale":"2023-10-08T09:13:09.196485+0000","last_undegraded":"2023-10-08T09:13:09.196485+0000","last_fullsized":"2023-10-08T09:13:09.196485+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:39.078307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.198401+0000","last_change":"2023-10-08T09:13:07.863492+0000","last_active":"2023-10-08T09:13:09.198401+0000","last_peered":"2023-10-08T09:13:09.198401+0000","last_clean":"2023-10-08T09:13:09.198401+0000","last_became_active":"2023-10-08T09:13:07.862803+0000","last_became_peered":"2023-10-08T09:13:07.862803+0000","last_unstale":"2023-10-08T09:13:09.198401+0000","last_undegraded":"2023-10-08T09:13:09.198401+0000","last_fullsized":"2023-10-08T09:13:09.198401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:39.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867318+0000","last_change":"2023-10-08T09:13:02.836596+0000","last_active":"2023-10-08T09:13:07.867318+0000","last_peered":"2023-10-08T09:13:07.867318+0000","last_clean":"2023-10-08T09:13:07.867318+0000","last_became_active":"2023-10-08T09:13:02.836283+0000","last_became_peered":"2023-10-08T09:13:02.836283+0000","last_unstale":"2023-10-08T09:13:07.867318+0000","last_undegraded":"2023-10-08T09:13:07.867318+0000","last_fullsized":"2023-10-08T09:13:07.867318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:02.393575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419022+0000","last_change":"2023-10-08T09:13:02.379516+0000","last_active":"2023-10-08T09:13:31.419022+0000","last_peered":"2023-10-08T09:13:31.419022+0000","last_clean":"2023-10-08T09:13:31.419022+0000","last_became_active":"2023-10-08T09:13:02.379229+0000","last_became_peered":"2023-10-08T09:13:02.379229+0000","last_unstale":"2023-10-08T09:13:31.419022+0000","last_undegraded":"2023-10-08T09:13:31.419022+0000","last_fullsized":"2023-10-08T09:13:31.419022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:55.294651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":310,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851134+0000","last_change":"2023-10-08T09:12:02.620769+0000","last_active":"2023-10-08T09:13:07.851134+0000","last_peered":"2023-10-08T09:13:07.851134+0000","last_clean":"2023-10-08T09:13:07.851134+0000","last_became_active":"2023-10-08T09:12:02.620388+0000","last_became_peered":"2023-10-08T09:12:02.620388+0000","last_unstale":"2023-10-08T09:13:07.851134+0000","last_undegraded":"2023-10-08T09:13:07.851134+0000","last_fullsized":"2023-10-08T09:13:07.851134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_clean_scrub_stamp":"2023-10-08T09:09:10.746288+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:55:35.713060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857458+0000","last_change":"2023-10-08T09:13:08.096318+0000","last_active":"2023-10-08T09:13:08.857458+0000","last_peered":"2023-10-08T09:13:08.857458+0000","last_clean":"2023-10-08T09:13:08.857458+0000","last_became_active":"2023-10-08T09:13:08.095774+0000","last_became_peered":"2023-10-08T09:13:08.095774+0000","last_unstale":"2023-10-08T09:13:08.857458+0000","last_undegraded":"2023-10-08T09:13:08.857458+0000","last_fullsized":"2023-10-08T09:13:08.857458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:53.089468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.843966+0000","last_change":"2023-10-08T09:13:02.835167+0000","last_active":"2023-10-08T09:13:07.843966+0000","last_peered":"2023-10-08T09:13:07.843966+0000","last_clean":"2023-10-08T09:13:07.843966+0000","last_became_active":"2023-10-08T09:13:02.834850+0000","last_became_peered":"2023-10-08T09:13:02.834850+0000","last_unstale":"2023-10-08T09:13:07.843966+0000","last_undegraded":"2023-10-08T09:13:07.843966+0000","last_fullsized":"2023-10-08T09:13:07.843966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:19.422372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:30.466887+0000","last_change":"2023-10-08T09:13:02.377746+0000","last_active":"2023-10-08T09:13:30.466887+0000","last_peered":"2023-10-08T09:13:30.466887+0000","last_clean":"2023-10-08T09:13:30.466887+0000","last_became_active":"2023-10-08T09:13:02.377422+0000","last_became_peered":"2023-10-08T09:13:02.377422+0000","last_unstale":"2023-10-08T09:13:30.466887+0000","last_undegraded":"2023-10-08T09:13:30.466887+0000","last_fullsized":"2023-10-08T09:13:30.466887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:13.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196921+0000","last_change":"2023-10-08T09:13:07.864678+0000","last_active":"2023-10-08T09:13:09.196921+0000","last_peered":"2023-10-08T09:13:09.196921+0000","last_clean":"2023-10-08T09:13:09.196921+0000","last_became_active":"2023-10-08T09:13:07.864226+0000","last_became_peered":"2023-10-08T09:13:07.864226+0000","last_unstale":"2023-10-08T09:13:09.196921+0000","last_undegraded":"2023-10-08T09:13:09.196921+0000","last_fullsized":"2023-10-08T09:13:09.196921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:43:51.179853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.420310+0000","last_change":"2023-10-08T09:13:02.370014+0000","last_active":"2023-10-08T09:13:31.420310+0000","last_peered":"2023-10-08T09:13:31.420310+0000","last_clean":"2023-10-08T09:13:31.420310+0000","last_became_active":"2023-10-08T09:13:02.369704+0000","last_became_peered":"2023-10-08T09:13:02.369704+0000","last_unstale":"2023-10-08T09:13:31.420310+0000","last_undegraded":"2023-10-08T09:13:31.420310+0000","last_fullsized":"2023-10-08T09:13:31.420310+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:28.902274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844826+0000","last_change":"2023-10-08T09:13:02.833859+0000","last_active":"2023-10-08T09:13:07.844826+0000","last_peered":"2023-10-08T09:13:07.844826+0000","last_clean":"2023-10-08T09:13:07.844826+0000","last_became_active":"2023-10-08T09:13:02.833546+0000","last_became_peered":"2023-10-08T09:13:02.833546+0000","last_unstale":"2023-10-08T09:13:07.844826+0000","last_undegraded":"2023-10-08T09:13:07.844826+0000","last_fullsized":"2023-10-08T09:13:07.844826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:35.534588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856554+0000","last_change":"2023-10-08T09:13:08.095622+0000","last_active":"2023-10-08T09:13:08.856554+0000","last_peered":"2023-10-08T09:13:08.856554+0000","last_clean":"2023-10-08T09:13:08.856554+0000","last_became_active":"2023-10-08T09:13:08.095143+0000","last_became_peered":"2023-10-08T09:13:08.095143+0000","last_unstale":"2023-10-08T09:13:08.856554+0000","last_undegraded":"2023-10-08T09:13:08.856554+0000","last_fullsized":"2023-10-08T09:13:08.856554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:46.738102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852731+0000","last_change":"2023-10-08T09:13:02.366393+0000","last_active":"2023-10-08T09:13:07.852731+0000","last_peered":"2023-10-08T09:13:07.852731+0000","last_clean":"2023-10-08T09:13:07.852731+0000","last_became_active":"2023-10-08T09:13:02.366050+0000","last_became_peered":"2023-10-08T09:13:02.366050+0000","last_unstale":"2023-10-08T09:13:07.852731+0000","last_undegraded":"2023-10-08T09:13:07.852731+0000","last_fullsized":"2023-10-08T09:13:07.852731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:16.607422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867284+0000","last_change":"2023-10-08T09:13:02.834053+0000","last_active":"2023-10-08T09:13:07.867284+0000","last_peered":"2023-10-08T09:13:07.867284+0000","last_clean":"2023-10-08T09:13:07.867284+0000","last_became_active":"2023-10-08T09:13:02.833707+0000","last_became_peered":"2023-10-08T09:13:02.833707+0000","last_unstale":"2023-10-08T09:13:07.867284+0000","last_undegraded":"2023-10-08T09:13:07.867284+0000","last_fullsized":"2023-10-08T09:13:07.867284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:31.372003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066008+0000","last_change":"2023-10-08T09:13:07.855734+0000","last_active":"2023-10-08T09:13:09.066008+0000","last_peered":"2023-10-08T09:13:09.066008+0000","last_clean":"2023-10-08T09:13:09.066008+0000","last_became_active":"2023-10-08T09:13:07.855313+0000","last_became_peered":"2023-10-08T09:13:07.855313+0000","last_unstale":"2023-10-08T09:13:09.066008+0000","last_undegraded":"2023-10-08T09:13:09.066008+0000","last_fullsized":"2023-10-08T09:13:09.066008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:15.207337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.421788+0000","last_change":"2023-10-08T09:13:02.371319+0000","last_active":"2023-10-08T09:13:32.421788+0000","last_peered":"2023-10-08T09:13:32.421788+0000","last_clean":"2023-10-08T09:13:32.421788+0000","last_became_active":"2023-10-08T09:13:02.371049+0000","last_became_peered":"2023-10-08T09:13:02.371049+0000","last_unstale":"2023-10-08T09:13:32.421788+0000","last_undegraded":"2023-10-08T09:13:32.421788+0000","last_fullsized":"2023-10-08T09:13:32.421788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:28.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856489+0000","last_change":"2023-10-08T09:13:02.838682+0000","last_active":"2023-10-08T09:13:07.856489+0000","last_peered":"2023-10-08T09:13:07.856489+0000","last_clean":"2023-10-08T09:13:07.856489+0000","last_became_active":"2023-10-08T09:13:02.838389+0000","last_became_peered":"2023-10-08T09:13:02.838389+0000","last_unstale":"2023-10-08T09:13:07.856489+0000","last_undegraded":"2023-10-08T09:13:07.856489+0000","last_fullsized":"2023-10-08T09:13:07.856489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:52.937723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857116+0000","last_change":"2023-10-08T09:13:08.098131+0000","last_active":"2023-10-08T09:13:08.857116+0000","last_peered":"2023-10-08T09:13:08.857116+0000","last_clean":"2023-10-08T09:13:08.857116+0000","last_became_active":"2023-10-08T09:13:08.097352+0000","last_became_peered":"2023-10-08T09:13:08.097352+0000","last_unstale":"2023-10-08T09:13:08.857116+0000","last_undegraded":"2023-10-08T09:13:08.857116+0000","last_fullsized":"2023-10-08T09:13:08.857116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:30.388875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845811+0000","last_change":"2023-10-08T09:13:02.372129+0000","last_active":"2023-10-08T09:13:07.845811+0000","last_peered":"2023-10-08T09:13:07.845811+0000","last_clean":"2023-10-08T09:13:07.845811+0000","last_became_active":"2023-10-08T09:13:02.371806+0000","last_became_peered":"2023-10-08T09:13:02.371806+0000","last_unstale":"2023-10-08T09:13:07.845811+0000","last_undegraded":"2023-10-08T09:13:07.845811+0000","last_fullsized":"2023-10-08T09:13:07.845811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:14:33.930261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851260+0000","last_change":"2023-10-08T09:13:02.830017+0000","last_active":"2023-10-08T09:13:07.851260+0000","last_peered":"2023-10-08T09:13:07.851260+0000","last_clean":"2023-10-08T09:13:07.851260+0000","last_became_active":"2023-10-08T09:13:02.829700+0000","last_became_peered":"2023-10-08T09:13:02.829700+0000","last_unstale":"2023-10-08T09:13:07.851260+0000","last_undegraded":"2023-10-08T09:13:07.851260+0000","last_fullsized":"2023-10-08T09:13:07.851260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:31:31.387049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.198221+0000","last_change":"2023-10-08T09:13:07.869097+0000","last_active":"2023-10-08T09:13:09.198221+0000","last_peered":"2023-10-08T09:13:09.198221+0000","last_clean":"2023-10-08T09:13:09.198221+0000","last_became_active":"2023-10-08T09:13:07.868710+0000","last_became_peered":"2023-10-08T09:13:07.868710+0000","last_unstale":"2023-10-08T09:13:09.198221+0000","last_undegraded":"2023-10-08T09:13:09.198221+0000","last_fullsized":"2023-10-08T09:13:09.198221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:37:27.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419277+0000","last_change":"2023-10-08T09:13:02.370694+0000","last_active":"2023-10-08T09:13:31.419277+0000","last_peered":"2023-10-08T09:13:31.419277+0000","last_clean":"2023-10-08T09:13:31.419277+0000","last_became_active":"2023-10-08T09:13:02.370161+0000","last_became_peered":"2023-10-08T09:13:02.370161+0000","last_unstale":"2023-10-08T09:13:31.419277+0000","last_undegraded":"2023-10-08T09:13:31.419277+0000","last_fullsized":"2023-10-08T09:13:31.419277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:04.809703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852727+0000","last_change":"2023-10-08T09:13:02.831026+0000","last_active":"2023-10-08T09:13:07.852727+0000","last_peered":"2023-10-08T09:13:07.852727+0000","last_clean":"2023-10-08T09:13:07.852727+0000","last_became_active":"2023-10-08T09:13:02.830715+0000","last_became_peered":"2023-10-08T09:13:02.830715+0000","last_unstale":"2023-10-08T09:13:07.852727+0000","last_undegraded":"2023-10-08T09:13:07.852727+0000","last_fullsized":"2023-10-08T09:13:07.852727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:47.709820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857049+0000","last_change":"2023-10-08T09:13:07.860464+0000","last_active":"2023-10-08T09:13:08.857049+0000","last_peered":"2023-10-08T09:13:08.857049+0000","last_clean":"2023-10-08T09:13:08.857049+0000","last_became_active":"2023-10-08T09:13:07.859962+0000","last_became_peered":"2023-10-08T09:13:07.859962+0000","last_unstale":"2023-10-08T09:13:08.857049+0000","last_undegraded":"2023-10-08T09:13:08.857049+0000","last_fullsized":"2023-10-08T09:13:08.857049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:04.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847141+0000","last_change":"2023-10-08T09:13:02.371308+0000","last_active":"2023-10-08T09:13:07.847141+0000","last_peered":"2023-10-08T09:13:07.847141+0000","last_clean":"2023-10-08T09:13:07.847141+0000","last_became_active":"2023-10-08T09:13:02.370968+0000","last_became_peered":"2023-10-08T09:13:02.370968+0000","last_unstale":"2023-10-08T09:13:07.847141+0000","last_undegraded":"2023-10-08T09:13:07.847141+0000","last_fullsized":"2023-10-08T09:13:07.847141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:58.491112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867614+0000","last_change":"2023-10-08T09:13:02.826511+0000","last_active":"2023-10-08T09:13:07.867614+0000","last_peered":"2023-10-08T09:13:07.867614+0000","last_clean":"2023-10-08T09:13:07.867614+0000","last_became_active":"2023-10-08T09:13:02.826205+0000","last_became_peered":"2023-10-08T09:13:02.826205+0000","last_unstale":"2023-10-08T09:13:07.867614+0000","last_undegraded":"2023-10-08T09:13:07.867614+0000","last_fullsized":"2023-10-08T09:13:07.867614+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:12.270176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857078+0000","last_change":"2023-10-08T09:13:08.096360+0000","last_active":"2023-10-08T09:13:08.857078+0000","last_peered":"2023-10-08T09:13:08.857078+0000","last_clean":"2023-10-08T09:13:08.857078+0000","last_became_active":"2023-10-08T09:13:08.095915+0000","last_became_peered":"2023-10-08T09:13:08.095915+0000","last_unstale":"2023-10-08T09:13:08.857078+0000","last_undegraded":"2023-10-08T09:13:08.857078+0000","last_fullsized":"2023-10-08T09:13:08.857078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:59.412471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:29.460947+0000","last_change":"2023-10-08T09:13:02.375287+0000","last_active":"2023-10-08T09:13:29.460947+0000","last_peered":"2023-10-08T09:13:29.460947+0000","last_clean":"2023-10-08T09:13:29.460947+0000","last_became_active":"2023-10-08T09:13:02.374668+0000","last_became_peered":"2023-10-08T09:13:02.374668+0000","last_unstale":"2023-10-08T09:13:29.460947+0000","last_undegraded":"2023-10-08T09:13:29.460947+0000","last_fullsized":"2023-10-08T09:13:29.460947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:22.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852423+0000","last_change":"2023-10-08T09:13:02.828240+0000","last_active":"2023-10-08T09:13:07.852423+0000","last_peered":"2023-10-08T09:13:07.852423+0000","last_clean":"2023-10-08T09:13:07.852423+0000","last_became_active":"2023-10-08T09:13:02.827951+0000","last_became_peered":"2023-10-08T09:13:02.827951+0000","last_unstale":"2023-10-08T09:13:07.852423+0000","last_undegraded":"2023-10-08T09:13:07.852423+0000","last_fullsized":"2023-10-08T09:13:07.852423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:37.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197019+0000","last_change":"2023-10-08T09:13:07.865846+0000","last_active":"2023-10-08T09:13:09.197019+0000","last_peered":"2023-10-08T09:13:09.197019+0000","last_clean":"2023-10-08T09:13:09.197019+0000","last_became_active":"2023-10-08T09:13:07.865529+0000","last_became_peered":"2023-10-08T09:13:07.865529+0000","last_unstale":"2023-10-08T09:13:09.197019+0000","last_undegraded":"2023-10-08T09:13:09.197019+0000","last_fullsized":"2023-10-08T09:13:09.197019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:34:03.408036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:30.467728+0000","last_change":"2023-10-08T09:13:02.380128+0000","last_active":"2023-10-08T09:13:30.467728+0000","last_peered":"2023-10-08T09:13:30.467728+0000","last_clean":"2023-10-08T09:13:30.467728+0000","last_became_active":"2023-10-08T09:13:02.379800+0000","last_became_peered":"2023-10-08T09:13:02.379800+0000","last_unstale":"2023-10-08T09:13:30.467728+0000","last_undegraded":"2023-10-08T09:13:30.467728+0000","last_fullsized":"2023-10-08T09:13:30.467728+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:01.546289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852605+0000","last_change":"2023-10-08T09:13:02.829474+0000","last_active":"2023-10-08T09:13:07.852605+0000","last_peered":"2023-10-08T09:13:07.852605+0000","last_clean":"2023-10-08T09:13:07.852605+0000","last_became_active":"2023-10-08T09:13:02.829173+0000","last_became_peered":"2023-10-08T09:13:02.829173+0000","last_unstale":"2023-10-08T09:13:07.852605+0000","last_undegraded":"2023-10-08T09:13:07.852605+0000","last_fullsized":"2023-10-08T09:13:07.852605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:02.469158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196717+0000","last_change":"2023-10-08T09:13:07.863163+0000","last_active":"2023-10-08T09:13:09.196717+0000","last_peered":"2023-10-08T09:13:09.196717+0000","last_clean":"2023-10-08T09:13:09.196717+0000","last_became_active":"2023-10-08T09:13:07.862755+0000","last_became_peered":"2023-10-08T09:13:07.862755+0000","last_unstale":"2023-10-08T09:13:09.196717+0000","last_undegraded":"2023-10-08T09:13:09.196717+0000","last_fullsized":"2023-10-08T09:13:09.196717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:14:25.878241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.420326+0000","last_change":"2023-10-08T09:13:02.376641+0000","last_active":"2023-10-08T09:13:31.420326+0000","last_peered":"2023-10-08T09:13:31.420326+0000","last_clean":"2023-10-08T09:13:31.420326+0000","last_became_active":"2023-10-08T09:13:02.376348+0000","last_became_peered":"2023-10-08T09:13:02.376348+0000","last_unstale":"2023-10-08T09:13:31.420326+0000","last_undegraded":"2023-10-08T09:13:31.420326+0000","last_fullsized":"2023-10-08T09:13:31.420326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:42.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851741+0000","last_change":"2023-10-08T09:13:02.836201+0000","last_active":"2023-10-08T09:13:07.851741+0000","last_peered":"2023-10-08T09:13:07.851741+0000","last_clean":"2023-10-08T09:13:07.851741+0000","last_became_active":"2023-10-08T09:13:02.835885+0000","last_became_peered":"2023-10-08T09:13:02.835885+0000","last_unstale":"2023-10-08T09:13:07.851741+0000","last_undegraded":"2023-10-08T09:13:07.851741+0000","last_fullsized":"2023-10-08T09:13:07.851741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:34.310854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856610+0000","last_change":"2023-10-08T09:13:07.863854+0000","last_active":"2023-10-08T09:13:08.856610+0000","last_peered":"2023-10-08T09:13:08.856610+0000","last_clean":"2023-10-08T09:13:08.856610+0000","last_became_active":"2023-10-08T09:13:07.863460+0000","last_became_peered":"2023-10-08T09:13:07.863460+0000","last_unstale":"2023-10-08T09:13:08.856610+0000","last_undegraded":"2023-10-08T09:13:08.856610+0000","last_fullsized":"2023-10-08T09:13:08.856610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:03.093858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844389+0000","last_change":"2023-10-08T09:13:02.372336+0000","last_active":"2023-10-08T09:13:07.844389+0000","last_peered":"2023-10-08T09:13:07.844389+0000","last_clean":"2023-10-08T09:13:07.844389+0000","last_became_active":"2023-10-08T09:13:02.372022+0000","last_became_peered":"2023-10-08T09:13:02.372022+0000","last_unstale":"2023-10-08T09:13:07.844389+0000","last_undegraded":"2023-10-08T09:13:07.844389+0000","last_fullsized":"2023-10-08T09:13:07.844389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:10.610332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852815+0000","last_change":"2023-10-08T09:13:02.832430+0000","last_active":"2023-10-08T09:13:07.852815+0000","last_peered":"2023-10-08T09:13:07.852815+0000","last_clean":"2023-10-08T09:13:07.852815+0000","last_became_active":"2023-10-08T09:13:02.832092+0000","last_became_peered":"2023-10-08T09:13:02.832092+0000","last_unstale":"2023-10-08T09:13:07.852815+0000","last_undegraded":"2023-10-08T09:13:07.852815+0000","last_fullsized":"2023-10-08T09:13:07.852815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:00.798900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852833+0000","last_change":"2023-10-08T09:13:02.833114+0000","last_active":"2023-10-08T09:13:07.852833+0000","last_peered":"2023-10-08T09:13:07.852833+0000","last_clean":"2023-10-08T09:13:07.852833+0000","last_became_active":"2023-10-08T09:13:02.830959+0000","last_became_peered":"2023-10-08T09:13:02.830959+0000","last_unstale":"2023-10-08T09:13:07.852833+0000","last_undegraded":"2023-10-08T09:13:07.852833+0000","last_fullsized":"2023-10-08T09:13:07.852833+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:13.853585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847750+0000","last_change":"2023-10-08T09:13:02.836046+0000","last_active":"2023-10-08T09:13:07.847750+0000","last_peered":"2023-10-08T09:13:07.847750+0000","last_clean":"2023-10-08T09:13:07.847750+0000","last_became_active":"2023-10-08T09:13:02.835722+0000","last_became_peered":"2023-10-08T09:13:02.835722+0000","last_unstale":"2023-10-08T09:13:07.847750+0000","last_undegraded":"2023-10-08T09:13:07.847750+0000","last_fullsized":"2023-10-08T09:13:07.847750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:00.316259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847033+0000","last_change":"2023-10-08T09:13:02.830021+0000","last_active":"2023-10-08T09:13:07.847033+0000","last_peered":"2023-10-08T09:13:07.847033+0000","last_clean":"2023-10-08T09:13:07.847033+0000","last_became_active":"2023-10-08T09:13:02.829397+0000","last_became_peered":"2023-10-08T09:13:02.829397+0000","last_unstale":"2023-10-08T09:13:07.847033+0000","last_undegraded":"2023-10-08T09:13:07.847033+0000","last_fullsized":"2023-10-08T09:13:07.847033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:29.153457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858028+0000","last_change":"2023-10-08T09:13:02.832002+0000","last_active":"2023-10-08T09:13:07.858028+0000","last_peered":"2023-10-08T09:13:07.858028+0000","last_clean":"2023-10-08T09:13:07.858028+0000","last_became_active":"2023-10-08T09:13:02.831666+0000","last_became_peered":"2023-10-08T09:13:02.831666+0000","last_unstale":"2023-10-08T09:13:07.858028+0000","last_undegraded":"2023-10-08T09:13:07.858028+0000","last_fullsized":"2023-10-08T09:13:07.858028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:16.414127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844926+0000","last_change":"2023-10-08T09:13:02.835881+0000","last_active":"2023-10-08T09:13:07.844926+0000","last_peered":"2023-10-08T09:13:07.844926+0000","last_clean":"2023-10-08T09:13:07.844926+0000","last_became_active":"2023-10-08T09:13:02.835587+0000","last_became_peered":"2023-10-08T09:13:02.835587+0000","last_unstale":"2023-10-08T09:13:07.844926+0000","last_undegraded":"2023-10-08T09:13:07.844926+0000","last_fullsized":"2023-10-08T09:13:07.844926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:52.323234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852003+0000","last_change":"2023-10-08T09:13:02.826371+0000","last_active":"2023-10-08T09:13:07.852003+0000","last_peered":"2023-10-08T09:13:07.852003+0000","last_clean":"2023-10-08T09:13:07.852003+0000","last_became_active":"2023-10-08T09:13:02.826093+0000","last_became_peered":"2023-10-08T09:13:02.826093+0000","last_unstale":"2023-10-08T09:13:07.852003+0000","last_undegraded":"2023-10-08T09:13:07.852003+0000","last_fullsized":"2023-10-08T09:13:07.852003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:08.113739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847619+0000","last_change":"2023-10-08T09:13:02.829936+0000","last_active":"2023-10-08T09:13:07.847619+0000","last_peered":"2023-10-08T09:13:07.847619+0000","last_clean":"2023-10-08T09:13:07.847619+0000","last_became_active":"2023-10-08T09:13:02.829498+0000","last_became_peered":"2023-10-08T09:13:02.829498+0000","last_unstale":"2023-10-08T09:13:07.847619+0000","last_undegraded":"2023-10-08T09:13:07.847619+0000","last_fullsized":"2023-10-08T09:13:07.847619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:42.200791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867444+0000","last_change":"2023-10-08T09:13:02.840415+0000","last_active":"2023-10-08T09:13:07.867444+0000","last_peered":"2023-10-08T09:13:07.867444+0000","last_clean":"2023-10-08T09:13:07.867444+0000","last_became_active":"2023-10-08T09:13:02.840075+0000","last_became_peered":"2023-10-08T09:13:02.840075+0000","last_unstale":"2023-10-08T09:13:07.867444+0000","last_undegraded":"2023-10-08T09:13:07.867444+0000","last_fullsized":"2023-10-08T09:13:07.867444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:48.959372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.857306+0000","last_change":"2023-10-08T09:13:02.828337+0000","last_active":"2023-10-08T09:13:07.857306+0000","last_peered":"2023-10-08T09:13:07.857306+0000","last_clean":"2023-10-08T09:13:07.857306+0000","last_became_active":"2023-10-08T09:13:02.827998+0000","last_became_peered":"2023-10-08T09:13:02.827998+0000","last_unstale":"2023-10-08T09:13:07.857306+0000","last_undegraded":"2023-10-08T09:13:07.857306+0000","last_fullsized":"2023-10-08T09:13:07.857306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:27.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867290+0000","last_change":"2023-10-08T09:13:02.833052+0000","last_active":"2023-10-08T09:13:07.867290+0000","last_peered":"2023-10-08T09:13:07.867290+0000","last_clean":"2023-10-08T09:13:07.867290+0000","last_became_active":"2023-10-08T09:13:02.832635+0000","last_became_peered":"2023-10-08T09:13:02.832635+0000","last_unstale":"2023-10-08T09:13:07.867290+0000","last_undegraded":"2023-10-08T09:13:07.867290+0000","last_fullsized":"2023-10-08T09:13:07.867290+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:50.995768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.846114+0000","last_change":"2023-10-08T09:13:02.840921+0000","last_active":"2023-10-08T09:13:07.846114+0000","last_peered":"2023-10-08T09:13:07.846114+0000","last_clean":"2023-10-08T09:13:07.846114+0000","last_became_active":"2023-10-08T09:13:02.840566+0000","last_became_peered":"2023-10-08T09:13:02.840566+0000","last_unstale":"2023-10-08T09:13:07.846114+0000","last_undegraded":"2023-10-08T09:13:07.846114+0000","last_fullsized":"2023-10-08T09:13:07.846114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:20.219581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845198+0000","last_change":"2023-10-08T09:13:02.834491+0000","last_active":"2023-10-08T09:13:07.845198+0000","last_peered":"2023-10-08T09:13:07.845198+0000","last_clean":"2023-10-08T09:13:07.845198+0000","last_became_active":"2023-10-08T09:13:02.834058+0000","last_became_peered":"2023-10-08T09:13:02.834058+0000","last_unstale":"2023-10-08T09:13:07.845198+0000","last_undegraded":"2023-10-08T09:13:07.845198+0000","last_fullsized":"2023-10-08T09:13:07.845198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:47.699423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852481+0000","last_change":"2023-10-08T09:13:02.836433+0000","last_active":"2023-10-08T09:13:07.852481+0000","last_peered":"2023-10-08T09:13:07.852481+0000","last_clean":"2023-10-08T09:13:07.852481+0000","last_became_active":"2023-10-08T09:13:02.835147+0000","last_became_peered":"2023-10-08T09:13:02.835147+0000","last_unstale":"2023-10-08T09:13:07.852481+0000","last_undegraded":"2023-10-08T09:13:07.852481+0000","last_fullsized":"2023-10-08T09:13:07.852481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:51.233662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247538,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8000,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":124,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":23260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514513,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":463649,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.748}]},{"osd":2,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.70699999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.78200000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.83799999999999997}]},{"osd":10,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.82099999999999995}]}]},{"osd":10,"up_from":69,"seq":296352743444,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1058053,"data_compressed":47757,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":1,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.38500000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.80800000000000005}]},{"osd":3,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.70899999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.35899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.77700000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.441},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.68000000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.626}]},{"osd":11,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55500000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906969,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":464231,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.374},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.34899999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.74199999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.442},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68300000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.61199999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168606,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":464056,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.75}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.45000000000000001,"15min":0.436},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.33300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42499999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.52700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0229999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.629},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.51100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":0.999},"last":0.71799999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.93100000000000005,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.53900000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.82799999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57199999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":7,"up_from":49,"seq":210453397538,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1055221,"data_compressed":47757,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41899999999999998,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.44500000000000001,"15min":0.433},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.745},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88200000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.85199999999999998,"5min":1.518,"15min":1.518},"last":0.502}]},{"osd":3,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.71399999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48799999999999999,"15min":0.47199999999999998},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.377,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.437}]},{"osd":5,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69099999999999995,"15min":0.68799999999999994},"min":{"1min":0.503,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.873,"5min":1.587,"15min":1.587},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92200000000000004,"5min":1.496,"15min":1.496},"last":0.54000000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.68000000000000005,"15min":0.66900000000000004},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.73299999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66900000000000004,"15min":0.65900000000000003},"min":{"1min":0.496,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83099999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.80700000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.746}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]}]},{"osd":6,"up_from":44,"seq":188978561062,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":463649,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.374,"15min":0.36599999999999999},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59799999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.39000000000000001,"15min":0.378},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.82999999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.80900000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.88600000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.45200000000000001,"15min":0.435},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.59999999999999998,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.78800000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.433,"15min":0.433},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.1339999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.341},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.71299999999999997,"15min":0.69399999999999995},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.95399999999999996}]},{"osd":8,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.55700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92200000000000004,"5min":1.175,"15min":1.175},"last":1.181}]},{"osd":9,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.075}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.302},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.1140000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":467218,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.497},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.755,"5min":0.999,"15min":0.999},"last":0.68200000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.83499999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.495},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.52200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92100000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.39400000000000002},"min":{"1min":0.26900000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53000000000000003,"5min":0.751,"15min":0.751},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.41099999999999998},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60899999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.90900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.628,"15min":0.628},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93700000000000006,"5min":1.034,"15min":1.034},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89500000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59199999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76200000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.502}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.85299999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.81200000000000006}]},{"osd":10,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.873}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":466217,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.36799999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.53900000000000003,"15min":0.50700000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.59799999999999998,"15min":0.52800000000000002},"min":{"1min":0.53800000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.468,"5min":3.468,"15min":3.468},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.62,"15min":0.55400000000000005},"min":{"1min":0.52900000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.4409999999999998,"5min":3.4409999999999998,"15min":3.4409999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Oct 8 09:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58899999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.47999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79100000000000004,"5min":1.133,"15min":1.133},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.095,"15min":1.095},"last":0.68600000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.747,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.745},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60499999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61499999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0209999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.72899999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.627}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.79800000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78800000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":468147,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80800000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.48699999999999999},"min":{"1min":0.33500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.41099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.318,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67300000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54400000000000004,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.86799999999999999,"5min":1.758,"15min":1.758},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.54600000000000004},"min":{"1min":0.30099999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.84799999999999998,"5min":1.772,"15min":1.772},"last":0.59799999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.746}]},{"osd":5,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45500000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.40400000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":1.839,"15min":1.839},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":1.492,"15min":1.492},"last":0.55800000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73599999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57299999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.61799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":463602,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.38800000000000001,"15min":0.374},"min":{"1min":0.316,"5min":0.217,"15min":0.217},"max":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.40000000000000002},"min":{"1min":0.32700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.45400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.876,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.629},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.996,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.58599999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.48099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88500000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72599999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1399999999999999,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.109,"5min":1.494,"15min":1.494},"last":0.67900000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.89900000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.87},"last":0.79100000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.59999999999999998,"15min":0.58699999999999997},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.52600000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.745},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76300000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80400000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953518,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":464315,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.499,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":1.466,"15min":1.466},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.495},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52600000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.83699999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.46000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.51800000000000002},"min":{"1min":0.44700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.625},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78100000000000003,"5min":1.429,"15min":1.429},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.51800000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82899999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.86099999999999999,"5min":0.873,"15min":0.873},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.94499999999999995,"5min":1.353,"15min":1.353},"last":0.65900000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.872,"5min":1.373,"15min":1.373},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.079,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.81999999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.93100000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1056837,"data_compressed":47757,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.502},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.497},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.627,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.998,"15min":0.998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.77500000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.27400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.84199999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85799999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.69799999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.745}]},{"osd":8,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.91900000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.0149999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":1.052},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:13:35.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:13:36.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 -- 172.21.15.49:0/1219352018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94106380 msgr2=0x7f2e941087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:36.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 --2- 172.21.15.49:0/1219352018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94106380 0x7f2e941087f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00b0a0 tx=0x7f2e8c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 -- 172.21.15.49:0/1219352018 shutdown_connections 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 --2- 172.21.15.49:0/1219352018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e94101120 0x7f2e94103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 --2- 172.21.15.49:0/1219352018 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94106380 0x7f2e941087f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 --2- 172.21.15.49:0/1219352018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e94103a50 0x7f2e94105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 -- 172.21.15.49:0/1219352018 >> 172.21.15.49:0/1219352018 conn(0x7f2e940fab10 msgr2=0x7f2e940fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 -- 172.21.15.49:0/1219352018 shutdown_connections 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.009+0000 7f2e9baec700 1 -- 172.21.15.49:0/1219352018 wait complete. 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 Processor -- start 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 -- start start 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94101120 0x7f2e94100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e94103a50 0x7f2e940ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:36.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e94106380 0x7f2e940ff610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940ffdb0 con 0x7f2e94101120 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940fff20 con 0x7f2e94103a50 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9baec700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e941a0790 con 0x7f2e94106380 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94101120 0x7f2e94100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94101120 0x7f2e94100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48458/0 (socket says 172.21.15.49:48458) 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 -- 172.21.15.49:0/2401735867 learned_addr learned my addr 172.21.15.49:0/2401735867 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9a089700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e94106380 0x7f2e940ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 -- 172.21.15.49:0/2401735867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e94106380 msgr2=0x7f2e940ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e94106380 0x7f2e940ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99087700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e94103a50 0x7f2e940ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 -- 172.21.15.49:0/2401735867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e94103a50 msgr2=0x7f2e940ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:36.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e94103a50 0x7f2e940ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99888700 1 -- 172.21.15.49:0/2401735867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8c009d00 con 0x7f2e94101120 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e9a089700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e94106380 0x7f2e940ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.010+0000 7f2e99087700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e94103a50 0x7f2e940ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.011+0000 7f2e99888700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94101120 0x7f2e94100a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e9c0 tx=0x7f2e9000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.011+0000 7f2e8affd700 1 -- 172.21.15.49:0/2401735867 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900097b0 con 0x7f2e94101120 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.011+0000 7f2e8affd700 1 -- 172.21.15.49:0/2401735867 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e90004830 con 0x7f2e94101120 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.011+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e941a0a20 con 0x7f2e94101120 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.011+0000 7f2e8affd700 1 -- 172.21.15.49:0/2401735867 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90005620 con 0x7f2e94101120 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.011+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e941a0ee0 con 0x7f2e94101120 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.012+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e940fc690 con 0x7f2e94101120 2023-10-08T09:13:36.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.013+0000 7f2e8affd700 1 -- 172.21.15.49:0/2401735867 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2e900049a0 con 0x7f2e94101120 2023-10-08T09:13:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.013+0000 7f2e8affd700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e800a1510 0x7f2e800a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.013+0000 7f2e8affd700 1 -- 172.21.15.49:0/2401735867 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2e90014070 con 0x7f2e94101120 2023-10-08T09:13:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.013+0000 7f2e99087700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e800a1510 0x7f2e800a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:36.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.014+0000 7f2e99087700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e800a1510 0x7f2e800a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e84005d90 tx=0x7f2e84005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:36.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.018+0000 7f2e8affd700 1 -- 172.21.15.49:0/2401735867 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2e900967d0 con 0x7f2e94101120 2023-10-08T09:13:36.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.215+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e9402d020 con 0x7f2e800a1510 2023-10-08T09:13:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.230+0000 7f2e8affd700 1 -- 172.21.15.49:0/2401735867 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422632 (secure 0 0 0) 0x7f2e9402d020 con 0x7f2e800a1510 2023-10-08T09:13:36.232 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e800a1510 msgr2=0x7f2e800a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e800a1510 0x7f2e800a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2e84005d90 tx=0x7f2e84005d00 comp rx=0 tx=0).stop 2023-10-08T09:13:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94101120 msgr2=0x7f2e94100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94101120 0x7f2e94100a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e9c0 tx=0x7f2e9000ed80 comp rx=0 tx=0).stop 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 shutdown_connections 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e800a1510 0x7f2e800a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e94106380 0x7f2e940ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e94101120 0x7f2e94100a40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.234+0000 7f2e9baec700 1 --2- 172.21.15.49:0/2401735867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e94103a50 0x7f2e940ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.235+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 >> 172.21.15.49:0/2401735867 conn(0x7f2e940fab10 msgr2=0x7f2e94101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.235+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 shutdown_connections 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:36.235+0000 7f2e9baec700 1 -- 172.21.15.49:0/2401735867 wait complete. 2023-10-08T09:13:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-10-08T09:13:36.248 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2023-10-08T09:13:35.859604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":635568,"num_objects":92,"num_object_clones":0,"num_object_copies":277,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":180,"num_read_kb":152,"num_write":386,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":59,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":319,"ondisk_log_size":319,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335640,"kb_used_data":11444,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124606184,"statfs":{"total":1151940427776,"available":1151596732416,"internally_reserved":0,"allocated":11718656,"data_stored":7431344,"data_compressed":248499,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40701,"num_objects":68,"num_object_clones":0,"num_object_copies":205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":54,"num_read_kb":43,"num_write":158,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":45,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003765"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.853653+0000","last_change":"2023-10-08T09:13:02.831436+0000","last_active":"2023-10-08T09:13:07.853653+0000","last_peered":"2023-10-08T09:13:07.853653+0000","last_clean":"2023-10-08T09:13:07.853653+0000","last_became_active":"2023-10-08T09:13:02.831123+0000","last_became_peered":"2023-10-08T09:13:02.831123+0000","last_unstale":"2023-10-08T09:13:07.853653+0000","last_undegraded":"2023-10-08T09:13:07.853653+0000","last_fullsized":"2023-10-08T09:13:07.853653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:33.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856370+0000","last_change":"2023-10-08T09:13:02.832286+0000","last_active":"2023-10-08T09:13:07.856370+0000","last_peered":"2023-10-08T09:13:07.856370+0000","last_clean":"2023-10-08T09:13:07.856370+0000","last_became_active":"2023-10-08T09:13:02.831881+0000","last_became_peered":"2023-10-08T09:13:02.831881+0000","last_unstale":"2023-10-08T09:13:07.856370+0000","last_undegraded":"2023-10-08T09:13:07.856370+0000","last_fullsized":"2023-10-08T09:13:07.856370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:31.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856154+0000","last_change":"2023-10-08T09:13:02.831454+0000","last_active":"2023-10-08T09:13:07.856154+0000","last_peered":"2023-10-08T09:13:07.856154+0000","last_clean":"2023-10-08T09:13:07.856154+0000","last_became_active":"2023-10-08T09:13:02.831040+0000","last_became_peered":"2023-10-08T09:13:02.831040+0000","last_unstale":"2023-10-08T09:13:07.856154+0000","last_undegraded":"2023-10-08T09:13:07.856154+0000","last_fullsized":"2023-10-08T09:13:07.856154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:16:02.788279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852354+0000","last_change":"2023-10-08T09:13:02.832699+0000","last_active":"2023-10-08T09:13:07.852354+0000","last_peered":"2023-10-08T09:13:07.852354+0000","last_clean":"2023-10-08T09:13:07.852354+0000","last_became_active":"2023-10-08T09:13:02.832345+0000","last_became_peered":"2023-10-08T09:13:02.832345+0000","last_unstale":"2023-10-08T09:13:07.852354+0000","last_undegraded":"2023-10-08T09:13:07.852354+0000","last_fullsized":"2023-10-08T09:13:07.852354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:01.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851384+0000","last_change":"2023-10-08T09:13:02.827767+0000","last_active":"2023-10-08T09:13:07.851384+0000","last_peered":"2023-10-08T09:13:07.851384+0000","last_clean":"2023-10-08T09:13:07.851384+0000","last_became_active":"2023-10-08T09:13:02.827454+0000","last_became_peered":"2023-10-08T09:13:02.827454+0000","last_unstale":"2023-10-08T09:13:07.851384+0000","last_undegraded":"2023-10-08T09:13:07.851384+0000","last_fullsized":"2023-10-08T09:13:07.851384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:43.122347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.094802+0000","last_change":"2023-10-08T09:13:02.828989+0000","last_active":"2023-10-08T09:13:08.094802+0000","last_peered":"2023-10-08T09:13:08.094802+0000","last_clean":"2023-10-08T09:13:08.094802+0000","last_became_active":"2023-10-08T09:13:02.828648+0000","last_became_peered":"2023-10-08T09:13:02.828648+0000","last_unstale":"2023-10-08T09:13:08.094802+0000","last_undegraded":"2023-10-08T09:13:08.094802+0000","last_fullsized":"2023-10-08T09:13:08.094802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:15:33.751851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852032+0000","last_change":"2023-10-08T09:13:02.831765+0000","last_active":"2023-10-08T09:13:07.852032+0000","last_peered":"2023-10-08T09:13:07.852032+0000","last_clean":"2023-10-08T09:13:07.852032+0000","last_became_active":"2023-10-08T09:13:02.831446+0000","last_became_peered":"2023-10-08T09:13:02.831446+0000","last_unstale":"2023-10-08T09:13:07.852032+0000","last_undegraded":"2023-10-08T09:13:07.852032+0000","last_fullsized":"2023-10-08T09:13:07.852032+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:41.463161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867118+0000","last_change":"2023-10-08T09:13:02.817586+0000","last_active":"2023-10-08T09:13:07.867118+0000","last_peered":"2023-10-08T09:13:07.867118+0000","last_clean":"2023-10-08T09:13:07.867118+0000","last_became_active":"2023-10-08T09:13:02.817231+0000","last_became_peered":"2023-10-08T09:13:02.817231+0000","last_unstale":"2023-10-08T09:13:07.867118+0000","last_undegraded":"2023-10-08T09:13:07.867118+0000","last_fullsized":"2023-10-08T09:13:07.867118+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:23.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.846396+0000","last_change":"2023-10-08T09:13:02.831925+0000","last_active":"2023-10-08T09:13:07.846396+0000","last_peered":"2023-10-08T09:13:07.846396+0000","last_clean":"2023-10-08T09:13:07.846396+0000","last_became_active":"2023-10-08T09:13:02.831625+0000","last_became_peered":"2023-10-08T09:13:02.831625+0000","last_unstale":"2023-10-08T09:13:07.846396+0000","last_undegraded":"2023-10-08T09:13:07.846396+0000","last_fullsized":"2023-10-08T09:13:07.846396+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:39.115740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.095335+0000","last_change":"2023-10-08T09:13:02.831752+0000","last_active":"2023-10-08T09:13:08.095335+0000","last_peered":"2023-10-08T09:13:08.095335+0000","last_clean":"2023-10-08T09:13:08.095335+0000","last_became_active":"2023-10-08T09:13:02.831468+0000","last_became_peered":"2023-10-08T09:13:02.831468+0000","last_unstale":"2023-10-08T09:13:08.095335+0000","last_undegraded":"2023-10-08T09:13:08.095335+0000","last_fullsized":"2023-10-08T09:13:08.095335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:05.758017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852576+0000","last_change":"2023-10-08T09:13:02.828838+0000","last_active":"2023-10-08T09:13:07.852576+0000","last_peered":"2023-10-08T09:13:07.852576+0000","last_clean":"2023-10-08T09:13:07.852576+0000","last_became_active":"2023-10-08T09:13:02.828598+0000","last_became_peered":"2023-10-08T09:13:02.828598+0000","last_unstale":"2023-10-08T09:13:07.852576+0000","last_undegraded":"2023-10-08T09:13:07.852576+0000","last_fullsized":"2023-10-08T09:13:07.852576+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:04.317649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852193+0000","last_change":"2023-10-08T09:13:02.831730+0000","last_active":"2023-10-08T09:13:07.852193+0000","last_peered":"2023-10-08T09:13:07.852193+0000","last_clean":"2023-10-08T09:13:07.852193+0000","last_became_active":"2023-10-08T09:13:02.831404+0000","last_became_peered":"2023-10-08T09:13:02.831404+0000","last_unstale":"2023-10-08T09:13:07.852193+0000","last_undegraded":"2023-10-08T09:13:07.852193+0000","last_fullsized":"2023-10-08T09:13:07.852193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:22.067908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845131+0000","last_change":"2023-10-08T09:13:02.832869+0000","last_active":"2023-10-08T09:13:07.845131+0000","last_peered":"2023-10-08T09:13:07.845131+0000","last_clean":"2023-10-08T09:13:07.845131+0000","last_became_active":"2023-10-08T09:13:02.832332+0000","last_became_peered":"2023-10-08T09:13:02.832332+0000","last_unstale":"2023-10-08T09:13:07.845131+0000","last_undegraded":"2023-10-08T09:13:07.845131+0000","last_fullsized":"2023-10-08T09:13:07.845131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:45:54.609041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856060+0000","last_change":"2023-10-08T09:13:02.829027+0000","last_active":"2023-10-08T09:13:07.856060+0000","last_peered":"2023-10-08T09:13:07.856060+0000","last_clean":"2023-10-08T09:13:07.856060+0000","last_became_active":"2023-10-08T09:13:02.828714+0000","last_became_peered":"2023-10-08T09:13:02.828714+0000","last_unstale":"2023-10-08T09:13:07.856060+0000","last_undegraded":"2023-10-08T09:13:07.856060+0000","last_fullsized":"2023-10-08T09:13:07.856060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:13.121274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856252+0000","last_change":"2023-10-08T09:13:02.838640+0000","last_active":"2023-10-08T09:13:07.856252+0000","last_peered":"2023-10-08T09:13:07.856252+0000","last_clean":"2023-10-08T09:13:07.856252+0000","last_became_active":"2023-10-08T09:13:02.838302+0000","last_became_peered":"2023-10-08T09:13:02.838302+0000","last_unstale":"2023-10-08T09:13:07.856252+0000","last_undegraded":"2023-10-08T09:13:07.856252+0000","last_fullsized":"2023-10-08T09:13:07.856252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:54.268315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845967+0000","last_change":"2023-10-08T09:13:02.834027+0000","last_active":"2023-10-08T09:13:07.845967+0000","last_peered":"2023-10-08T09:13:07.845967+0000","last_clean":"2023-10-08T09:13:07.845967+0000","last_became_active":"2023-10-08T09:13:02.833733+0000","last_became_peered":"2023-10-08T09:13:02.833733+0000","last_unstale":"2023-10-08T09:13:07.845967+0000","last_undegraded":"2023-10-08T09:13:07.845967+0000","last_fullsized":"2023-10-08T09:13:07.845967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:03.644640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.853559+0000","last_change":"2023-10-08T09:13:02.832040+0000","last_active":"2023-10-08T09:13:07.853559+0000","last_peered":"2023-10-08T09:13:07.853559+0000","last_clean":"2023-10-08T09:13:07.853559+0000","last_became_active":"2023-10-08T09:13:02.831752+0000","last_became_peered":"2023-10-08T09:13:02.831752+0000","last_unstale":"2023-10-08T09:13:07.853559+0000","last_undegraded":"2023-10-08T09:13:07.853559+0000","last_fullsized":"2023-10-08T09:13:07.853559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:38.938703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852839+0000","last_change":"2023-10-08T09:13:02.830535+0000","last_active":"2023-10-08T09:13:07.852839+0000","last_peered":"2023-10-08T09:13:07.852839+0000","last_clean":"2023-10-08T09:13:07.852839+0000","last_became_active":"2023-10-08T09:13:02.830144+0000","last_became_peered":"2023-10-08T09:13:02.830144+0000","last_unstale":"2023-10-08T09:13:07.852839+0000","last_undegraded":"2023-10-08T09:13:07.852839+0000","last_fullsized":"2023-10-08T09:13:07.852839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:33.850815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847951+0000","last_change":"2023-10-08T09:13:02.830938+0000","last_active":"2023-10-08T09:13:07.847951+0000","last_peered":"2023-10-08T09:13:07.847951+0000","last_clean":"2023-10-08T09:13:07.847951+0000","last_became_active":"2023-10-08T09:13:02.830551+0000","last_became_peered":"2023-10-08T09:13:02.830551+0000","last_unstale":"2023-10-08T09:13:07.847951+0000","last_undegraded":"2023-10-08T09:13:07.847951+0000","last_fullsized":"2023-10-08T09:13:07.847951+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:26.894614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066152+0000","last_change":"2023-10-08T09:13:07.871322+0000","last_active":"2023-10-08T09:13:09.066152+0000","last_peered":"2023-10-08T09:13:09.066152+0000","last_clean":"2023-10-08T09:13:09.066152+0000","last_became_active":"2023-10-08T09:13:07.870828+0000","last_became_peered":"2023-10-08T09:13:07.870828+0000","last_unstale":"2023-10-08T09:13:09.066152+0000","last_undegraded":"2023-10-08T09:13:09.066152+0000","last_fullsized":"2023-10-08T09:13:09.066152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:37.145705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.421863+0000","last_change":"2023-10-08T09:13:02.373510+0000","last_active":"2023-10-08T09:13:32.421863+0000","last_peered":"2023-10-08T09:13:32.421863+0000","last_clean":"2023-10-08T09:13:32.421863+0000","last_became_active":"2023-10-08T09:13:02.373174+0000","last_became_peered":"2023-10-08T09:13:02.373174+0000","last_unstale":"2023-10-08T09:13:32.421863+0000","last_undegraded":"2023-10-08T09:13:32.421863+0000","last_fullsized":"2023-10-08T09:13:32.421863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867265+0000","last_change":"2023-10-08T09:13:02.835195+0000","last_active":"2023-10-08T09:13:07.867265+0000","last_peered":"2023-10-08T09:13:07.867265+0000","last_clean":"2023-10-08T09:13:07.867265+0000","last_became_active":"2023-10-08T09:13:02.834893+0000","last_became_peered":"2023-10-08T09:13:02.834893+0000","last_unstale":"2023-10-08T09:13:07.867265+0000","last_undegraded":"2023-10-08T09:13:07.867265+0000","last_fullsized":"2023-10-08T09:13:07.867265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:35.835017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856567+0000","last_change":"2023-10-08T09:13:07.871418+0000","last_active":"2023-10-08T09:13:08.856567+0000","last_peered":"2023-10-08T09:13:08.856567+0000","last_clean":"2023-10-08T09:13:08.856567+0000","last_became_active":"2023-10-08T09:13:07.870961+0000","last_became_peered":"2023-10-08T09:13:07.870961+0000","last_unstale":"2023-10-08T09:13:08.856567+0000","last_undegraded":"2023-10-08T09:13:08.856567+0000","last_fullsized":"2023-10-08T09:13:08.856567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:31.723498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857106+0000","last_change":"2023-10-08T09:13:02.370728+0000","last_active":"2023-10-08T09:13:08.857106+0000","last_peered":"2023-10-08T09:13:08.857106+0000","last_clean":"2023-10-08T09:13:08.857106+0000","last_became_active":"2023-10-08T09:13:02.370396+0000","last_became_peered":"2023-10-08T09:13:02.370396+0000","last_unstale":"2023-10-08T09:13:08.857106+0000","last_undegraded":"2023-10-08T09:13:08.857106+0000","last_fullsized":"2023-10-08T09:13:08.857106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:32.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858747+0000","last_change":"2023-10-08T09:13:02.827823+0000","last_active":"2023-10-08T09:13:07.858747+0000","last_peered":"2023-10-08T09:13:07.858747+0000","last_clean":"2023-10-08T09:13:07.858747+0000","last_became_active":"2023-10-08T09:13:02.827417+0000","last_became_peered":"2023-10-08T09:13:02.827417+0000","last_unstale":"2023-10-08T09:13:07.858747+0000","last_undegraded":"2023-10-08T09:13:07.858747+0000","last_fullsized":"2023-10-08T09:13:07.858747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:38.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856926+0000","last_change":"2023-10-08T09:13:07.868912+0000","last_active":"2023-10-08T09:13:08.856926+0000","last_peered":"2023-10-08T09:13:08.856926+0000","last_clean":"2023-10-08T09:13:08.856926+0000","last_became_active":"2023-10-08T09:13:07.868504+0000","last_became_peered":"2023-10-08T09:13:07.868504+0000","last_unstale":"2023-10-08T09:13:08.856926+0000","last_undegraded":"2023-10-08T09:13:08.856926+0000","last_fullsized":"2023-10-08T09:13:08.856926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:04.686985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.420183+0000","last_change":"2023-10-08T09:13:02.375244+0000","last_active":"2023-10-08T09:13:31.420183+0000","last_peered":"2023-10-08T09:13:31.420183+0000","last_clean":"2023-10-08T09:13:31.420183+0000","last_became_active":"2023-10-08T09:13:02.374526+0000","last_became_peered":"2023-10-08T09:13:02.374526+0000","last_unstale":"2023-10-08T09:13:31.420183+0000","last_undegraded":"2023-10-08T09:13:31.420183+0000","last_fullsized":"2023-10-08T09:13:31.420183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:36.380491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845834+0000","last_change":"2023-10-08T09:13:02.838373+0000","last_active":"2023-10-08T09:13:07.845834+0000","last_peered":"2023-10-08T09:13:07.845834+0000","last_clean":"2023-10-08T09:13:07.845834+0000","last_became_active":"2023-10-08T09:13:02.838100+0000","last_became_peered":"2023-10-08T09:13:02.838100+0000","last_unstale":"2023-10-08T09:13:07.845834+0000","last_undegraded":"2023-10-08T09:13:07.845834+0000","last_fullsized":"2023-10-08T09:13:07.845834+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:59.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856678+0000","last_change":"2023-10-08T09:13:07.868929+0000","last_active":"2023-10-08T09:13:08.856678+0000","last_peered":"2023-10-08T09:13:08.856678+0000","last_clean":"2023-10-08T09:13:08.856678+0000","last_became_active":"2023-10-08T09:13:07.868451+0000","last_became_peered":"2023-10-08T09:13:07.868451+0000","last_unstale":"2023-10-08T09:13:08.856678+0000","last_undegraded":"2023-10-08T09:13:08.856678+0000","last_fullsized":"2023-10-08T09:13:08.856678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:36.540319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.855228+0000","last_change":"2023-10-08T09:13:02.366404+0000","last_active":"2023-10-08T09:13:07.855228+0000","last_peered":"2023-10-08T09:13:07.855228+0000","last_clean":"2023-10-08T09:13:07.855228+0000","last_became_active":"2023-10-08T09:13:02.366025+0000","last_became_peered":"2023-10-08T09:13:02.366025+0000","last_unstale":"2023-10-08T09:13:07.855228+0000","last_undegraded":"2023-10-08T09:13:07.855228+0000","last_fullsized":"2023-10-08T09:13:07.855228+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:51.916565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858268+0000","last_change":"2023-10-08T09:13:02.832852+0000","last_active":"2023-10-08T09:13:07.858268+0000","last_peered":"2023-10-08T09:13:07.858268+0000","last_clean":"2023-10-08T09:13:07.858268+0000","last_became_active":"2023-10-08T09:13:02.832530+0000","last_became_peered":"2023-10-08T09:13:02.832530+0000","last_unstale":"2023-10-08T09:13:07.858268+0000","last_undegraded":"2023-10-08T09:13:07.858268+0000","last_fullsized":"2023-10-08T09:13:07.858268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:20.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066518+0000","last_change":"2023-10-08T09:13:07.863864+0000","last_active":"2023-10-08T09:13:09.066518+0000","last_peered":"2023-10-08T09:13:09.066518+0000","last_clean":"2023-10-08T09:13:09.066518+0000","last_became_active":"2023-10-08T09:13:07.863509+0000","last_became_peered":"2023-10-08T09:13:07.863509+0000","last_unstale":"2023-10-08T09:13:09.066518+0000","last_undegraded":"2023-10-08T09:13:09.066518+0000","last_fullsized":"2023-10-08T09:13:09.066518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:22.933327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.413622+0000","last_change":"2023-10-08T09:13:02.374215+0000","last_active":"2023-10-08T09:13:32.413622+0000","last_peered":"2023-10-08T09:13:32.413622+0000","last_clean":"2023-10-08T09:13:32.413622+0000","last_became_active":"2023-10-08T09:13:02.373912+0000","last_became_peered":"2023-10-08T09:13:02.373912+0000","last_unstale":"2023-10-08T09:13:32.413622+0000","last_undegraded":"2023-10-08T09:13:32.413622+0000","last_fullsized":"2023-10-08T09:13:32.413622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:18.315244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852561+0000","last_change":"2023-10-08T09:13:02.830651+0000","last_active":"2023-10-08T09:13:07.852561+0000","last_peered":"2023-10-08T09:13:07.852561+0000","last_clean":"2023-10-08T09:13:07.852561+0000","last_became_active":"2023-10-08T09:13:02.830324+0000","last_became_peered":"2023-10-08T09:13:02.830324+0000","last_unstale":"2023-10-08T09:13:07.852561+0000","last_undegraded":"2023-10-08T09:13:07.852561+0000","last_fullsized":"2023-10-08T09:13:07.852561+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:02.143794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066375+0000","last_change":"2023-10-08T09:13:07.862270+0000","last_active":"2023-10-08T09:13:09.066375+0000","last_peered":"2023-10-08T09:13:09.066375+0000","last_clean":"2023-10-08T09:13:09.066375+0000","last_became_active":"2023-10-08T09:13:07.861847+0000","last_became_peered":"2023-10-08T09:13:07.861847+0000","last_unstale":"2023-10-08T09:13:09.066375+0000","last_undegraded":"2023-10-08T09:13:09.066375+0000","last_fullsized":"2023-10-08T09:13:09.066375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:33.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.420047+0000","last_change":"2023-10-08T09:13:02.370334+0000","last_active":"2023-10-08T09:13:31.420047+0000","last_peered":"2023-10-08T09:13:31.420047+0000","last_clean":"2023-10-08T09:13:31.420047+0000","last_became_active":"2023-10-08T09:13:02.370068+0000","last_became_peered":"2023-10-08T09:13:02.370068+0000","last_unstale":"2023-10-08T09:13:31.420047+0000","last_undegraded":"2023-10-08T09:13:31.420047+0000","last_fullsized":"2023-10-08T09:13:31.420047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.962779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.855103+0000","last_change":"2023-10-08T09:13:02.832346+0000","last_active":"2023-10-08T09:13:07.855103+0000","last_peered":"2023-10-08T09:13:07.855103+0000","last_clean":"2023-10-08T09:13:07.855103+0000","last_became_active":"2023-10-08T09:13:02.832059+0000","last_became_peered":"2023-10-08T09:13:02.832059+0000","last_unstale":"2023-10-08T09:13:07.855103+0000","last_undegraded":"2023-10-08T09:13:07.855103+0000","last_fullsized":"2023-10-08T09:13:07.855103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:30.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197219+0000","last_change":"2023-10-08T09:13:07.863466+0000","last_active":"2023-10-08T09:13:09.197219+0000","last_peered":"2023-10-08T09:13:09.197219+0000","last_clean":"2023-10-08T09:13:09.197219+0000","last_became_active":"2023-10-08T09:13:07.863015+0000","last_became_peered":"2023-10-08T09:13:07.863015+0000","last_unstale":"2023-10-08T09:13:09.197219+0000","last_undegraded":"2023-10-08T09:13:09.197219+0000","last_fullsized":"2023-10-08T09:13:09.197219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:30.316625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:30.467879+0000","last_change":"2023-10-08T09:13:02.369444+0000","last_active":"2023-10-08T09:13:30.467879+0000","last_peered":"2023-10-08T09:13:30.467879+0000","last_clean":"2023-10-08T09:13:30.467879+0000","last_became_active":"2023-10-08T09:13:02.369142+0000","last_became_peered":"2023-10-08T09:13:02.369142+0000","last_unstale":"2023-10-08T09:13:30.467879+0000","last_undegraded":"2023-10-08T09:13:30.467879+0000","last_fullsized":"2023-10-08T09:13:30.467879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:28.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867230+0000","last_change":"2023-10-08T09:13:02.836125+0000","last_active":"2023-10-08T09:13:07.867230+0000","last_peered":"2023-10-08T09:13:07.867230+0000","last_clean":"2023-10-08T09:13:07.867230+0000","last_became_active":"2023-10-08T09:13:02.835809+0000","last_became_peered":"2023-10-08T09:13:02.835809+0000","last_unstale":"2023-10-08T09:13:07.867230+0000","last_undegraded":"2023-10-08T09:13:07.867230+0000","last_fullsized":"2023-10-08T09:13:07.867230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:15.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.198700+0000","last_change":"2023-10-08T09:13:07.861633+0000","last_active":"2023-10-08T09:13:09.198700+0000","last_peered":"2023-10-08T09:13:09.198700+0000","last_clean":"2023-10-08T09:13:09.198700+0000","last_became_active":"2023-10-08T09:13:07.861168+0000","last_became_peered":"2023-10-08T09:13:07.861168+0000","last_unstale":"2023-10-08T09:13:09.198700+0000","last_undegraded":"2023-10-08T09:13:09.198700+0000","last_fullsized":"2023-10-08T09:13:09.198700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:47.283892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.424293+0000","last_change":"2023-10-08T09:13:02.365703+0000","last_active":"2023-10-08T09:13:32.424293+0000","last_peered":"2023-10-08T09:13:32.424293+0000","last_clean":"2023-10-08T09:13:32.424293+0000","last_became_active":"2023-10-08T09:13:02.365421+0000","last_became_peered":"2023-10-08T09:13:02.365421+0000","last_unstale":"2023-10-08T09:13:32.424293+0000","last_undegraded":"2023-10-08T09:13:32.424293+0000","last_fullsized":"2023-10-08T09:13:32.424293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:43.124511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.850748+0000","last_change":"2023-10-08T09:13:02.840941+0000","last_active":"2023-10-08T09:13:07.850748+0000","last_peered":"2023-10-08T09:13:07.850748+0000","last_clean":"2023-10-08T09:13:07.850748+0000","last_became_active":"2023-10-08T09:13:02.840605+0000","last_became_peered":"2023-10-08T09:13:02.840605+0000","last_unstale":"2023-10-08T09:13:07.850748+0000","last_undegraded":"2023-10-08T09:13:07.850748+0000","last_fullsized":"2023-10-08T09:13:07.850748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:24:13.417028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856661+0000","last_change":"2023-10-08T09:13:07.868950+0000","last_active":"2023-10-08T09:13:08.856661+0000","last_peered":"2023-10-08T09:13:08.856661+0000","last_clean":"2023-10-08T09:13:08.856661+0000","last_became_active":"2023-10-08T09:13:07.868542+0000","last_became_peered":"2023-10-08T09:13:07.868542+0000","last_unstale":"2023-10-08T09:13:08.856661+0000","last_undegraded":"2023-10-08T09:13:08.856661+0000","last_fullsized":"2023-10-08T09:13:08.856661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:11.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:33.414606+0000","last_change":"2023-10-08T09:13:02.376188+0000","last_active":"2023-10-08T09:13:33.414606+0000","last_peered":"2023-10-08T09:13:33.414606+0000","last_clean":"2023-10-08T09:13:33.414606+0000","last_became_active":"2023-10-08T09:13:02.375914+0000","last_became_peered":"2023-10-08T09:13:02.375914+0000","last_unstale":"2023-10-08T09:13:33.414606+0000","last_undegraded":"2023-10-08T09:13:33.414606+0000","last_fullsized":"2023-10-08T09:13:33.414606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:10.744085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.846172+0000","last_change":"2023-10-08T09:13:02.833429+0000","last_active":"2023-10-08T09:13:07.846172+0000","last_peered":"2023-10-08T09:13:07.846172+0000","last_clean":"2023-10-08T09:13:07.846172+0000","last_became_active":"2023-10-08T09:13:02.832792+0000","last_became_peered":"2023-10-08T09:13:02.832792+0000","last_unstale":"2023-10-08T09:13:07.846172+0000","last_undegraded":"2023-10-08T09:13:07.846172+0000","last_fullsized":"2023-10-08T09:13:07.846172+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:24.563447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196856+0000","last_change":"2023-10-08T09:13:07.862688+0000","last_active":"2023-10-08T09:13:09.196856+0000","last_peered":"2023-10-08T09:13:09.196856+0000","last_clean":"2023-10-08T09:13:09.196856+0000","last_became_active":"2023-10-08T09:13:07.862197+0000","last_became_peered":"2023-10-08T09:13:07.862197+0000","last_unstale":"2023-10-08T09:13:09.196856+0000","last_undegraded":"2023-10-08T09:13:09.196856+0000","last_fullsized":"2023-10-08T09:13:09.196856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:56.792045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.422449+0000","last_change":"2023-10-08T09:13:02.380283+0000","last_active":"2023-10-08T09:13:32.422449+0000","last_peered":"2023-10-08T09:13:32.422449+0000","last_clean":"2023-10-08T09:13:32.422449+0000","last_became_active":"2023-10-08T09:13:02.379844+0000","last_became_peered":"2023-10-08T09:13:02.379844+0000","last_unstale":"2023-10-08T09:13:32.422449+0000","last_undegraded":"2023-10-08T09:13:32.422449+0000","last_fullsized":"2023-10-08T09:13:32.422449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:18.162437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858856+0000","last_change":"2023-10-08T09:13:02.839211+0000","last_active":"2023-10-08T09:13:07.858856+0000","last_peered":"2023-10-08T09:13:07.858856+0000","last_clean":"2023-10-08T09:13:07.858856+0000","last_became_active":"2023-10-08T09:13:02.838968+0000","last_became_peered":"2023-10-08T09:13:02.838968+0000","last_unstale":"2023-10-08T09:13:07.858856+0000","last_undegraded":"2023-10-08T09:13:07.858856+0000","last_fullsized":"2023-10-08T09:13:07.858856+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:24:02.541749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857159+0000","last_change":"2023-10-08T09:13:07.857016+0000","last_active":"2023-10-08T09:13:08.857159+0000","last_peered":"2023-10-08T09:13:08.857159+0000","last_clean":"2023-10-08T09:13:08.857159+0000","last_became_active":"2023-10-08T09:13:07.856562+0000","last_became_peered":"2023-10-08T09:13:07.856562+0000","last_unstale":"2023-10-08T09:13:08.857159+0000","last_undegraded":"2023-10-08T09:13:08.857159+0000","last_fullsized":"2023-10-08T09:13:08.857159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:27.073675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.411103+0000","last_change":"2023-10-08T09:13:02.377027+0000","last_active":"2023-10-08T09:13:31.411103+0000","last_peered":"2023-10-08T09:13:31.411103+0000","last_clean":"2023-10-08T09:13:31.411103+0000","last_became_active":"2023-10-08T09:13:02.376746+0000","last_became_peered":"2023-10-08T09:13:02.376746+0000","last_unstale":"2023-10-08T09:13:31.411103+0000","last_undegraded":"2023-10-08T09:13:31.411103+0000","last_fullsized":"2023-10-08T09:13:31.411103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:19.062296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845910+0000","last_change":"2023-10-08T09:13:02.837279+0000","last_active":"2023-10-08T09:13:07.845910+0000","last_peered":"2023-10-08T09:13:07.845910+0000","last_clean":"2023-10-08T09:13:07.845910+0000","last_became_active":"2023-10-08T09:13:02.836966+0000","last_became_peered":"2023-10-08T09:13:02.836966+0000","last_unstale":"2023-10-08T09:13:07.845910+0000","last_undegraded":"2023-10-08T09:13:07.845910+0000","last_fullsized":"2023-10-08T09:13:07.845910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:25.462835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197967+0000","last_change":"2023-10-08T09:13:07.865538+0000","last_active":"2023-10-08T09:13:09.197967+0000","last_peered":"2023-10-08T09:13:09.197967+0000","last_clean":"2023-10-08T09:13:09.197967+0000","last_became_active":"2023-10-08T09:13:07.865118+0000","last_became_peered":"2023-10-08T09:13:07.865118+0000","last_unstale":"2023-10-08T09:13:09.197967+0000","last_undegraded":"2023-10-08T09:13:09.197967+0000","last_fullsized":"2023-10-08T09:13:09.197967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:32.167439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'24","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:34.263972+0000","last_change":"2023-10-08T09:13:02.378173+0000","last_active":"2023-10-08T09:13:34.263972+0000","last_peered":"2023-10-08T09:13:34.263972+0000","last_clean":"2023-10-08T09:13:34.263972+0000","last_became_active":"2023-10-08T09:13:02.377880+0000","last_became_peered":"2023-10-08T09:13:02.377880+0000","last_unstale":"2023-10-08T09:13:34.263972+0000","last_undegraded":"2023-10-08T09:13:34.263972+0000","last_fullsized":"2023-10-08T09:13:34.263972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38547,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":26,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851336+0000","last_change":"2023-10-08T09:13:02.829874+0000","last_active":"2023-10-08T09:13:07.851336+0000","last_peered":"2023-10-08T09:13:07.851336+0000","last_clean":"2023-10-08T09:13:07.851336+0000","last_became_active":"2023-10-08T09:13:02.829552+0000","last_became_peered":"2023-10-08T09:13:02.829552+0000","last_unstale":"2023-10-08T09:13:07.851336+0000","last_undegraded":"2023-10-08T09:13:07.851336+0000","last_fullsized":"2023-10-08T09:13:07.851336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:29.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857511+0000","last_change":"2023-10-08T09:13:07.869661+0000","last_active":"2023-10-08T09:13:08.857511+0000","last_peered":"2023-10-08T09:13:08.857511+0000","last_clean":"2023-10-08T09:13:08.857511+0000","last_became_active":"2023-10-08T09:13:07.869236+0000","last_became_peered":"2023-10-08T09:13:07.869236+0000","last_unstale":"2023-10-08T09:13:08.857511+0000","last_undegraded":"2023-10-08T09:13:08.857511+0000","last_fullsized":"2023-10-08T09:13:08.857511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:26.354461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.424722+0000","last_change":"2023-10-08T09:13:02.376941+0000","last_active":"2023-10-08T09:13:32.424722+0000","last_peered":"2023-10-08T09:13:32.424722+0000","last_clean":"2023-10-08T09:13:32.424722+0000","last_became_active":"2023-10-08T09:13:02.375810+0000","last_became_peered":"2023-10-08T09:13:02.375810+0000","last_unstale":"2023-10-08T09:13:32.424722+0000","last_undegraded":"2023-10-08T09:13:32.424722+0000","last_fullsized":"2023-10-08T09:13:32.424722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:17.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844323+0000","last_change":"2023-10-08T09:13:02.836781+0000","last_active":"2023-10-08T09:13:07.844323+0000","last_peered":"2023-10-08T09:13:07.844323+0000","last_clean":"2023-10-08T09:13:07.844323+0000","last_became_active":"2023-10-08T09:13:02.836493+0000","last_became_peered":"2023-10-08T09:13:02.836493+0000","last_unstale":"2023-10-08T09:13:07.844323+0000","last_undegraded":"2023-10-08T09:13:07.844323+0000","last_fullsized":"2023-10-08T09:13:07.844323+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:03.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196963+0000","last_change":"2023-10-08T09:13:07.859104+0000","last_active":"2023-10-08T09:13:09.196963+0000","last_peered":"2023-10-08T09:13:09.196963+0000","last_clean":"2023-10-08T09:13:09.196963+0000","last_became_active":"2023-10-08T09:13:07.858691+0000","last_became_peered":"2023-10-08T09:13:07.858691+0000","last_unstale":"2023-10-08T09:13:09.196963+0000","last_undegraded":"2023-10-08T09:13:09.196963+0000","last_fullsized":"2023-10-08T09:13:09.196963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:33.256044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.424682+0000","last_change":"2023-10-08T09:13:02.364472+0000","last_active":"2023-10-08T09:13:32.424682+0000","last_peered":"2023-10-08T09:13:32.424682+0000","last_clean":"2023-10-08T09:13:32.424682+0000","last_became_active":"2023-10-08T09:13:02.364130+0000","last_became_peered":"2023-10-08T09:13:02.364130+0000","last_unstale":"2023-10-08T09:13:32.424682+0000","last_undegraded":"2023-10-08T09:13:32.424682+0000","last_fullsized":"2023-10-08T09:13:32.424682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:18.156186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851785+0000","last_change":"2023-10-08T09:13:02.836119+0000","last_active":"2023-10-08T09:13:07.851785+0000","last_peered":"2023-10-08T09:13:07.851785+0000","last_clean":"2023-10-08T09:13:07.851785+0000","last_became_active":"2023-10-08T09:13:02.835548+0000","last_became_peered":"2023-10-08T09:13:02.835548+0000","last_unstale":"2023-10-08T09:13:07.851785+0000","last_undegraded":"2023-10-08T09:13:07.851785+0000","last_fullsized":"2023-10-08T09:13:07.851785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:34.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196717+0000","last_change":"2023-10-08T09:13:07.871447+0000","last_active":"2023-10-08T09:13:09.196717+0000","last_peered":"2023-10-08T09:13:09.196717+0000","last_clean":"2023-10-08T09:13:09.196717+0000","last_became_active":"2023-10-08T09:13:07.871018+0000","last_became_peered":"2023-10-08T09:13:07.871018+0000","last_unstale":"2023-10-08T09:13:09.196717+0000","last_undegraded":"2023-10-08T09:13:09.196717+0000","last_fullsized":"2023-10-08T09:13:09.196717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:34.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419512+0000","last_change":"2023-10-08T09:13:02.378954+0000","last_active":"2023-10-08T09:13:31.419512+0000","last_peered":"2023-10-08T09:13:31.419512+0000","last_clean":"2023-10-08T09:13:31.419512+0000","last_became_active":"2023-10-08T09:13:02.378635+0000","last_became_peered":"2023-10-08T09:13:02.378635+0000","last_unstale":"2023-10-08T09:13:31.419512+0000","last_undegraded":"2023-10-08T09:13:31.419512+0000","last_fullsized":"2023-10-08T09:13:31.419512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:09.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851405+0000","last_change":"2023-10-08T09:13:02.828732+0000","last_active":"2023-10-08T09:13:07.851405+0000","last_peered":"2023-10-08T09:13:07.851405+0000","last_clean":"2023-10-08T09:13:07.851405+0000","last_became_active":"2023-10-08T09:13:02.828449+0000","last_became_peered":"2023-10-08T09:13:02.828449+0000","last_unstale":"2023-10-08T09:13:07.851405+0000","last_undegraded":"2023-10-08T09:13:07.851405+0000","last_fullsized":"2023-10-08T09:13:07.851405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:09.883908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197375+0000","last_change":"2023-10-08T09:13:07.869610+0000","last_active":"2023-10-08T09:13:09.197375+0000","last_peered":"2023-10-08T09:13:09.197375+0000","last_clean":"2023-10-08T09:13:09.197375+0000","last_became_active":"2023-10-08T09:13:07.869225+0000","last_became_peered":"2023-10-08T09:13:07.869225+0000","last_unstale":"2023-10-08T09:13:09.197375+0000","last_undegraded":"2023-10-08T09:13:09.197375+0000","last_fullsized":"2023-10-08T09:13:09.197375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:35.225452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'16","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.407181+0000","last_change":"2023-10-08T09:13:02.375658+0000","last_active":"2023-10-08T09:13:31.407181+0000","last_peered":"2023-10-08T09:13:31.407181+0000","last_clean":"2023-10-08T09:13:31.407181+0000","last_became_active":"2023-10-08T09:13:02.375383+0000","last_became_peered":"2023-10-08T09:13:02.375383+0000","last_unstale":"2023-10-08T09:13:31.407181+0000","last_undegraded":"2023-10-08T09:13:31.407181+0000","last_fullsized":"2023-10-08T09:13:31.407181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:52.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851572+0000","last_change":"2023-10-08T09:13:02.827684+0000","last_active":"2023-10-08T09:13:07.851572+0000","last_peered":"2023-10-08T09:13:07.851572+0000","last_clean":"2023-10-08T09:13:07.851572+0000","last_became_active":"2023-10-08T09:13:02.827337+0000","last_became_peered":"2023-10-08T09:13:02.827337+0000","last_unstale":"2023-10-08T09:13:07.851572+0000","last_undegraded":"2023-10-08T09:13:07.851572+0000","last_fullsized":"2023-10-08T09:13:07.851572+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:37.435065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856874+0000","last_change":"2023-10-08T09:13:07.871441+0000","last_active":"2023-10-08T09:13:08.856874+0000","last_peered":"2023-10-08T09:13:08.856874+0000","last_clean":"2023-10-08T09:13:08.856874+0000","last_became_active":"2023-10-08T09:13:07.870968+0000","last_became_peered":"2023-10-08T09:13:07.870968+0000","last_unstale":"2023-10-08T09:13:08.856874+0000","last_undegraded":"2023-10-08T09:13:08.856874+0000","last_fullsized":"2023-10-08T09:13:08.856874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:48.746147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.934155+0000","last_change":"2023-10-08T09:13:02.372514+0000","last_active":"2023-10-08T09:13:08.934155+0000","last_peered":"2023-10-08T09:13:08.934155+0000","last_clean":"2023-10-08T09:13:08.934155+0000","last_became_active":"2023-10-08T09:13:02.372181+0000","last_became_peered":"2023-10-08T09:13:02.372181+0000","last_unstale":"2023-10-08T09:13:08.934155+0000","last_undegraded":"2023-10-08T09:13:08.934155+0000","last_fullsized":"2023-10-08T09:13:08.934155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:28.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851785+0000","last_change":"2023-10-08T09:13:02.835220+0000","last_active":"2023-10-08T09:13:07.851785+0000","last_peered":"2023-10-08T09:13:07.851785+0000","last_clean":"2023-10-08T09:13:07.851785+0000","last_became_active":"2023-10-08T09:13:02.834914+0000","last_became_peered":"2023-10-08T09:13:02.834914+0000","last_unstale":"2023-10-08T09:13:07.851785+0000","last_undegraded":"2023-10-08T09:13:07.851785+0000","last_fullsized":"2023-10-08T09:13:07.851785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:37.447418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.933840+0000","last_change":"2023-10-08T09:13:08.096146+0000","last_active":"2023-10-08T09:13:08.933840+0000","last_peered":"2023-10-08T09:13:08.933840+0000","last_clean":"2023-10-08T09:13:08.933840+0000","last_became_active":"2023-10-08T09:13:08.095703+0000","last_became_peered":"2023-10-08T09:13:08.095703+0000","last_unstale":"2023-10-08T09:13:08.933840+0000","last_undegraded":"2023-10-08T09:13:08.933840+0000","last_fullsized":"2023-10-08T09:13:08.933840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:37.511094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197450+0000","last_change":"2023-10-08T09:13:02.372836+0000","last_active":"2023-10-08T09:13:09.197450+0000","last_peered":"2023-10-08T09:13:09.197450+0000","last_clean":"2023-10-08T09:13:09.197450+0000","last_became_active":"2023-10-08T09:13:02.372488+0000","last_became_peered":"2023-10-08T09:13:02.372488+0000","last_unstale":"2023-10-08T09:13:09.197450+0000","last_undegraded":"2023-10-08T09:13:09.197450+0000","last_fullsized":"2023-10-08T09:13:09.197450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:19.269539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.095649+0000","last_change":"2023-10-08T09:13:02.827306+0000","last_active":"2023-10-08T09:13:08.095649+0000","last_peered":"2023-10-08T09:13:08.095649+0000","last_clean":"2023-10-08T09:13:08.095649+0000","last_became_active":"2023-10-08T09:13:02.826965+0000","last_became_peered":"2023-10-08T09:13:02.826965+0000","last_unstale":"2023-10-08T09:13:08.095649+0000","last_undegraded":"2023-10-08T09:13:08.095649+0000","last_fullsized":"2023-10-08T09:13:08.095649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:27.513860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197872+0000","last_change":"2023-10-08T09:13:07.862637+0000","last_active":"2023-10-08T09:13:09.197872+0000","last_peered":"2023-10-08T09:13:09.197872+0000","last_clean":"2023-10-08T09:13:09.197872+0000","last_became_active":"2023-10-08T09:13:07.862117+0000","last_became_peered":"2023-10-08T09:13:07.862117+0000","last_unstale":"2023-10-08T09:13:09.197872+0000","last_undegraded":"2023-10-08T09:13:09.197872+0000","last_fullsized":"2023-10-08T09:13:09.197872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:14.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:34.435135+0000","last_change":"2023-10-08T09:13:02.379585+0000","last_active":"2023-10-08T09:13:34.435135+0000","last_peered":"2023-10-08T09:13:34.435135+0000","last_clean":"2023-10-08T09:13:34.435135+0000","last_became_active":"2023-10-08T09:13:02.379324+0000","last_became_peered":"2023-10-08T09:13:02.379324+0000","last_unstale":"2023-10-08T09:13:34.435135+0000","last_undegraded":"2023-10-08T09:13:34.435135+0000","last_fullsized":"2023-10-08T09:13:34.435135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:31.609558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.094946+0000","last_change":"2023-10-08T09:13:02.833416+0000","last_active":"2023-10-08T09:13:08.094946+0000","last_peered":"2023-10-08T09:13:08.094946+0000","last_clean":"2023-10-08T09:13:08.094946+0000","last_became_active":"2023-10-08T09:13:02.833040+0000","last_became_peered":"2023-10-08T09:13:02.833040+0000","last_unstale":"2023-10-08T09:13:08.094946+0000","last_undegraded":"2023-10-08T09:13:08.094946+0000","last_fullsized":"2023-10-08T09:13:08.094946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:44.480445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196474+0000","last_change":"2023-10-08T09:13:08.095723+0000","last_active":"2023-10-08T09:13:09.196474+0000","last_peered":"2023-10-08T09:13:09.196474+0000","last_clean":"2023-10-08T09:13:09.196474+0000","last_became_active":"2023-10-08T09:13:08.095271+0000","last_became_peered":"2023-10-08T09:13:08.095271+0000","last_unstale":"2023-10-08T09:13:09.196474+0000","last_undegraded":"2023-10-08T09:13:09.196474+0000","last_fullsized":"2023-10-08T09:13:09.196474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:04.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858052+0000","last_change":"2023-10-08T09:13:02.829663+0000","last_active":"2023-10-08T09:13:07.858052+0000","last_peered":"2023-10-08T09:13:07.858052+0000","last_clean":"2023-10-08T09:13:07.858052+0000","last_became_active":"2023-10-08T09:13:02.829354+0000","last_became_peered":"2023-10-08T09:13:02.829354+0000","last_unstale":"2023-10-08T09:13:07.858052+0000","last_undegraded":"2023-10-08T09:13:07.858052+0000","last_fullsized":"2023-10-08T09:13:07.858052+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:55.332346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:33.427604+0000","last_change":"2023-10-08T09:13:02.372114+0000","last_active":"2023-10-08T09:13:33.427604+0000","last_peered":"2023-10-08T09:13:33.427604+0000","last_clean":"2023-10-08T09:13:33.427604+0000","last_became_active":"2023-10-08T09:13:02.371603+0000","last_became_peered":"2023-10-08T09:13:02.371603+0000","last_unstale":"2023-10-08T09:13:33.427604+0000","last_undegraded":"2023-10-08T09:13:33.427604+0000","last_fullsized":"2023-10-08T09:13:33.427604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:39.078307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:10.614632+0000","last_change":"2023-10-08T09:13:07.863492+0000","last_active":"2023-10-08T09:13:10.614632+0000","last_peered":"2023-10-08T09:13:10.614632+0000","last_clean":"2023-10-08T09:13:10.614632+0000","last_became_active":"2023-10-08T09:13:07.862803+0000","last_became_peered":"2023-10-08T09:13:07.862803+0000","last_unstale":"2023-10-08T09:13:10.614632+0000","last_undegraded":"2023-10-08T09:13:10.614632+0000","last_fullsized":"2023-10-08T09:13:10.614632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:39.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867318+0000","last_change":"2023-10-08T09:13:02.836596+0000","last_active":"2023-10-08T09:13:07.867318+0000","last_peered":"2023-10-08T09:13:07.867318+0000","last_clean":"2023-10-08T09:13:07.867318+0000","last_became_active":"2023-10-08T09:13:02.836283+0000","last_became_peered":"2023-10-08T09:13:02.836283+0000","last_unstale":"2023-10-08T09:13:07.867318+0000","last_undegraded":"2023-10-08T09:13:07.867318+0000","last_fullsized":"2023-10-08T09:13:07.867318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:02.393575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419022+0000","last_change":"2023-10-08T09:13:02.379516+0000","last_active":"2023-10-08T09:13:31.419022+0000","last_peered":"2023-10-08T09:13:31.419022+0000","last_clean":"2023-10-08T09:13:31.419022+0000","last_became_active":"2023-10-08T09:13:02.379229+0000","last_became_peered":"2023-10-08T09:13:02.379229+0000","last_unstale":"2023-10-08T09:13:31.419022+0000","last_undegraded":"2023-10-08T09:13:31.419022+0000","last_fullsized":"2023-10-08T09:13:31.419022+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:55.294651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":310,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851134+0000","last_change":"2023-10-08T09:12:02.620769+0000","last_active":"2023-10-08T09:13:07.851134+0000","last_peered":"2023-10-08T09:13:07.851134+0000","last_clean":"2023-10-08T09:13:07.851134+0000","last_became_active":"2023-10-08T09:12:02.620388+0000","last_became_peered":"2023-10-08T09:12:02.620388+0000","last_unstale":"2023-10-08T09:13:07.851134+0000","last_undegraded":"2023-10-08T09:13:07.851134+0000","last_fullsized":"2023-10-08T09:13:07.851134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_clean_scrub_stamp":"2023-10-08T09:09:10.746288+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:55:35.713060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857458+0000","last_change":"2023-10-08T09:13:08.096318+0000","last_active":"2023-10-08T09:13:08.857458+0000","last_peered":"2023-10-08T09:13:08.857458+0000","last_clean":"2023-10-08T09:13:08.857458+0000","last_became_active":"2023-10-08T09:13:08.095774+0000","last_became_peered":"2023-10-08T09:13:08.095774+0000","last_unstale":"2023-10-08T09:13:08.857458+0000","last_undegraded":"2023-10-08T09:13:08.857458+0000","last_fullsized":"2023-10-08T09:13:08.857458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:53.089468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.843966+0000","last_change":"2023-10-08T09:13:02.835167+0000","last_active":"2023-10-08T09:13:07.843966+0000","last_peered":"2023-10-08T09:13:07.843966+0000","last_clean":"2023-10-08T09:13:07.843966+0000","last_became_active":"2023-10-08T09:13:02.834850+0000","last_became_peered":"2023-10-08T09:13:02.834850+0000","last_unstale":"2023-10-08T09:13:07.843966+0000","last_undegraded":"2023-10-08T09:13:07.843966+0000","last_fullsized":"2023-10-08T09:13:07.843966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:19.422372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:30.467031+0000","last_change":"2023-10-08T09:13:02.377746+0000","last_active":"2023-10-08T09:13:30.467031+0000","last_peered":"2023-10-08T09:13:30.467031+0000","last_clean":"2023-10-08T09:13:30.467031+0000","last_became_active":"2023-10-08T09:13:02.377422+0000","last_became_peered":"2023-10-08T09:13:02.377422+0000","last_unstale":"2023-10-08T09:13:30.467031+0000","last_undegraded":"2023-10-08T09:13:30.467031+0000","last_fullsized":"2023-10-08T09:13:30.467031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:13.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196921+0000","last_change":"2023-10-08T09:13:07.864678+0000","last_active":"2023-10-08T09:13:09.196921+0000","last_peered":"2023-10-08T09:13:09.196921+0000","last_clean":"2023-10-08T09:13:09.196921+0000","last_became_active":"2023-10-08T09:13:07.864226+0000","last_became_peered":"2023-10-08T09:13:07.864226+0000","last_unstale":"2023-10-08T09:13:09.196921+0000","last_undegraded":"2023-10-08T09:13:09.196921+0000","last_fullsized":"2023-10-08T09:13:09.196921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:43:51.179853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.420310+0000","last_change":"2023-10-08T09:13:02.370014+0000","last_active":"2023-10-08T09:13:31.420310+0000","last_peered":"2023-10-08T09:13:31.420310+0000","last_clean":"2023-10-08T09:13:31.420310+0000","last_became_active":"2023-10-08T09:13:02.369704+0000","last_became_peered":"2023-10-08T09:13:02.369704+0000","last_unstale":"2023-10-08T09:13:31.420310+0000","last_undegraded":"2023-10-08T09:13:31.420310+0000","last_fullsized":"2023-10-08T09:13:31.420310+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:28.902274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844826+0000","last_change":"2023-10-08T09:13:02.833859+0000","last_active":"2023-10-08T09:13:07.844826+0000","last_peered":"2023-10-08T09:13:07.844826+0000","last_clean":"2023-10-08T09:13:07.844826+0000","last_became_active":"2023-10-08T09:13:02.833546+0000","last_became_peered":"2023-10-08T09:13:02.833546+0000","last_unstale":"2023-10-08T09:13:07.844826+0000","last_undegraded":"2023-10-08T09:13:07.844826+0000","last_fullsized":"2023-10-08T09:13:07.844826+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:35.534588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856554+0000","last_change":"2023-10-08T09:13:08.095622+0000","last_active":"2023-10-08T09:13:08.856554+0000","last_peered":"2023-10-08T09:13:08.856554+0000","last_clean":"2023-10-08T09:13:08.856554+0000","last_became_active":"2023-10-08T09:13:08.095143+0000","last_became_peered":"2023-10-08T09:13:08.095143+0000","last_unstale":"2023-10-08T09:13:08.856554+0000","last_undegraded":"2023-10-08T09:13:08.856554+0000","last_fullsized":"2023-10-08T09:13:08.856554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:46.738102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419584+0000","last_change":"2023-10-08T09:13:02.366393+0000","last_active":"2023-10-08T09:13:31.419584+0000","last_peered":"2023-10-08T09:13:31.419584+0000","last_clean":"2023-10-08T09:13:31.419584+0000","last_became_active":"2023-10-08T09:13:02.366050+0000","last_became_peered":"2023-10-08T09:13:02.366050+0000","last_unstale":"2023-10-08T09:13:31.419584+0000","last_undegraded":"2023-10-08T09:13:31.419584+0000","last_fullsized":"2023-10-08T09:13:31.419584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:16.607422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867284+0000","last_change":"2023-10-08T09:13:02.834053+0000","last_active":"2023-10-08T09:13:07.867284+0000","last_peered":"2023-10-08T09:13:07.867284+0000","last_clean":"2023-10-08T09:13:07.867284+0000","last_became_active":"2023-10-08T09:13:02.833707+0000","last_became_peered":"2023-10-08T09:13:02.833707+0000","last_unstale":"2023-10-08T09:13:07.867284+0000","last_undegraded":"2023-10-08T09:13:07.867284+0000","last_fullsized":"2023-10-08T09:13:07.867284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:31.372003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.066008+0000","last_change":"2023-10-08T09:13:07.855734+0000","last_active":"2023-10-08T09:13:09.066008+0000","last_peered":"2023-10-08T09:13:09.066008+0000","last_clean":"2023-10-08T09:13:09.066008+0000","last_became_active":"2023-10-08T09:13:07.855313+0000","last_became_peered":"2023-10-08T09:13:07.855313+0000","last_unstale":"2023-10-08T09:13:09.066008+0000","last_undegraded":"2023-10-08T09:13:09.066008+0000","last_fullsized":"2023-10-08T09:13:09.066008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:15.207337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.421788+0000","last_change":"2023-10-08T09:13:02.371319+0000","last_active":"2023-10-08T09:13:32.421788+0000","last_peered":"2023-10-08T09:13:32.421788+0000","last_clean":"2023-10-08T09:13:32.421788+0000","last_became_active":"2023-10-08T09:13:02.371049+0000","last_became_peered":"2023-10-08T09:13:02.371049+0000","last_unstale":"2023-10-08T09:13:32.421788+0000","last_undegraded":"2023-10-08T09:13:32.421788+0000","last_fullsized":"2023-10-08T09:13:32.421788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:28.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.856489+0000","last_change":"2023-10-08T09:13:02.838682+0000","last_active":"2023-10-08T09:13:07.856489+0000","last_peered":"2023-10-08T09:13:07.856489+0000","last_clean":"2023-10-08T09:13:07.856489+0000","last_became_active":"2023-10-08T09:13:02.838389+0000","last_became_peered":"2023-10-08T09:13:02.838389+0000","last_unstale":"2023-10-08T09:13:07.856489+0000","last_undegraded":"2023-10-08T09:13:07.856489+0000","last_fullsized":"2023-10-08T09:13:07.856489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:52.937723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857116+0000","last_change":"2023-10-08T09:13:08.098131+0000","last_active":"2023-10-08T09:13:08.857116+0000","last_peered":"2023-10-08T09:13:08.857116+0000","last_clean":"2023-10-08T09:13:08.857116+0000","last_became_active":"2023-10-08T09:13:08.097352+0000","last_became_peered":"2023-10-08T09:13:08.097352+0000","last_unstale":"2023-10-08T09:13:08.857116+0000","last_undegraded":"2023-10-08T09:13:08.857116+0000","last_fullsized":"2023-10-08T09:13:08.857116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:30.388875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:29.458619+0000","last_change":"2023-10-08T09:13:02.372129+0000","last_active":"2023-10-08T09:13:29.458619+0000","last_peered":"2023-10-08T09:13:29.458619+0000","last_clean":"2023-10-08T09:13:29.458619+0000","last_became_active":"2023-10-08T09:13:02.371806+0000","last_became_peered":"2023-10-08T09:13:02.371806+0000","last_unstale":"2023-10-08T09:13:29.458619+0000","last_undegraded":"2023-10-08T09:13:29.458619+0000","last_fullsized":"2023-10-08T09:13:29.458619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:14:33.930261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851260+0000","last_change":"2023-10-08T09:13:02.830017+0000","last_active":"2023-10-08T09:13:07.851260+0000","last_peered":"2023-10-08T09:13:07.851260+0000","last_clean":"2023-10-08T09:13:07.851260+0000","last_became_active":"2023-10-08T09:13:02.829700+0000","last_became_peered":"2023-10-08T09:13:02.829700+0000","last_unstale":"2023-10-08T09:13:07.851260+0000","last_undegraded":"2023-10-08T09:13:07.851260+0000","last_fullsized":"2023-10-08T09:13:07.851260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:31:31.387049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.198221+0000","last_change":"2023-10-08T09:13:07.869097+0000","last_active":"2023-10-08T09:13:09.198221+0000","last_peered":"2023-10-08T09:13:09.198221+0000","last_clean":"2023-10-08T09:13:09.198221+0000","last_became_active":"2023-10-08T09:13:07.868710+0000","last_became_peered":"2023-10-08T09:13:07.868710+0000","last_unstale":"2023-10-08T09:13:09.198221+0000","last_undegraded":"2023-10-08T09:13:09.198221+0000","last_fullsized":"2023-10-08T09:13:09.198221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:37:27.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419277+0000","last_change":"2023-10-08T09:13:02.370694+0000","last_active":"2023-10-08T09:13:31.419277+0000","last_peered":"2023-10-08T09:13:31.419277+0000","last_clean":"2023-10-08T09:13:31.419277+0000","last_became_active":"2023-10-08T09:13:02.370161+0000","last_became_peered":"2023-10-08T09:13:02.370161+0000","last_unstale":"2023-10-08T09:13:31.419277+0000","last_undegraded":"2023-10-08T09:13:31.419277+0000","last_fullsized":"2023-10-08T09:13:31.419277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:04.809703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852727+0000","last_change":"2023-10-08T09:13:02.831026+0000","last_active":"2023-10-08T09:13:07.852727+0000","last_peered":"2023-10-08T09:13:07.852727+0000","last_clean":"2023-10-08T09:13:07.852727+0000","last_became_active":"2023-10-08T09:13:02.830715+0000","last_became_peered":"2023-10-08T09:13:02.830715+0000","last_unstale":"2023-10-08T09:13:07.852727+0000","last_undegraded":"2023-10-08T09:13:07.852727+0000","last_fullsized":"2023-10-08T09:13:07.852727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:47.709820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857049+0000","last_change":"2023-10-08T09:13:07.860464+0000","last_active":"2023-10-08T09:13:08.857049+0000","last_peered":"2023-10-08T09:13:08.857049+0000","last_clean":"2023-10-08T09:13:08.857049+0000","last_became_active":"2023-10-08T09:13:07.859962+0000","last_became_peered":"2023-10-08T09:13:07.859962+0000","last_unstale":"2023-10-08T09:13:08.857049+0000","last_undegraded":"2023-10-08T09:13:08.857049+0000","last_fullsized":"2023-10-08T09:13:08.857049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:04.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.419784+0000","last_change":"2023-10-08T09:13:02.371308+0000","last_active":"2023-10-08T09:13:31.419784+0000","last_peered":"2023-10-08T09:13:31.419784+0000","last_clean":"2023-10-08T09:13:31.419784+0000","last_became_active":"2023-10-08T09:13:02.370968+0000","last_became_peered":"2023-10-08T09:13:02.370968+0000","last_unstale":"2023-10-08T09:13:31.419784+0000","last_undegraded":"2023-10-08T09:13:31.419784+0000","last_fullsized":"2023-10-08T09:13:31.419784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:58.491112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867614+0000","last_change":"2023-10-08T09:13:02.826511+0000","last_active":"2023-10-08T09:13:07.867614+0000","last_peered":"2023-10-08T09:13:07.867614+0000","last_clean":"2023-10-08T09:13:07.867614+0000","last_became_active":"2023-10-08T09:13:02.826205+0000","last_became_peered":"2023-10-08T09:13:02.826205+0000","last_unstale":"2023-10-08T09:13:07.867614+0000","last_undegraded":"2023-10-08T09:13:07.867614+0000","last_fullsized":"2023-10-08T09:13:07.867614+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:12.270176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.857078+0000","last_change":"2023-10-08T09:13:08.096360+0000","last_active":"2023-10-08T09:13:08.857078+0000","last_peered":"2023-10-08T09:13:08.857078+0000","last_clean":"2023-10-08T09:13:08.857078+0000","last_became_active":"2023-10-08T09:13:08.095915+0000","last_became_peered":"2023-10-08T09:13:08.095915+0000","last_unstale":"2023-10-08T09:13:08.857078+0000","last_undegraded":"2023-10-08T09:13:08.857078+0000","last_fullsized":"2023-10-08T09:13:08.857078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:59.412471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:29.460947+0000","last_change":"2023-10-08T09:13:02.375287+0000","last_active":"2023-10-08T09:13:29.460947+0000","last_peered":"2023-10-08T09:13:29.460947+0000","last_clean":"2023-10-08T09:13:29.460947+0000","last_became_active":"2023-10-08T09:13:02.374668+0000","last_became_peered":"2023-10-08T09:13:02.374668+0000","last_unstale":"2023-10-08T09:13:29.460947+0000","last_undegraded":"2023-10-08T09:13:29.460947+0000","last_fullsized":"2023-10-08T09:13:29.460947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:22.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852423+0000","last_change":"2023-10-08T09:13:02.828240+0000","last_active":"2023-10-08T09:13:07.852423+0000","last_peered":"2023-10-08T09:13:07.852423+0000","last_clean":"2023-10-08T09:13:07.852423+0000","last_became_active":"2023-10-08T09:13:02.827951+0000","last_became_peered":"2023-10-08T09:13:02.827951+0000","last_unstale":"2023-10-08T09:13:07.852423+0000","last_undegraded":"2023-10-08T09:13:07.852423+0000","last_fullsized":"2023-10-08T09:13:07.852423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:37.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.197019+0000","last_change":"2023-10-08T09:13:07.865846+0000","last_active":"2023-10-08T09:13:09.197019+0000","last_peered":"2023-10-08T09:13:09.197019+0000","last_clean":"2023-10-08T09:13:09.197019+0000","last_became_active":"2023-10-08T09:13:07.865529+0000","last_became_peered":"2023-10-08T09:13:07.865529+0000","last_unstale":"2023-10-08T09:13:09.197019+0000","last_undegraded":"2023-10-08T09:13:09.197019+0000","last_fullsized":"2023-10-08T09:13:09.197019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:34:03.408036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:34.436388+0000","last_change":"2023-10-08T09:13:02.380128+0000","last_active":"2023-10-08T09:13:34.436388+0000","last_peered":"2023-10-08T09:13:34.436388+0000","last_clean":"2023-10-08T09:13:34.436388+0000","last_became_active":"2023-10-08T09:13:02.379800+0000","last_became_peered":"2023-10-08T09:13:02.379800+0000","last_unstale":"2023-10-08T09:13:34.436388+0000","last_undegraded":"2023-10-08T09:13:34.436388+0000","last_fullsized":"2023-10-08T09:13:34.436388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:01.546289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852605+0000","last_change":"2023-10-08T09:13:02.829474+0000","last_active":"2023-10-08T09:13:07.852605+0000","last_peered":"2023-10-08T09:13:07.852605+0000","last_clean":"2023-10-08T09:13:07.852605+0000","last_became_active":"2023-10-08T09:13:02.829173+0000","last_became_peered":"2023-10-08T09:13:02.829173+0000","last_unstale":"2023-10-08T09:13:07.852605+0000","last_undegraded":"2023-10-08T09:13:07.852605+0000","last_fullsized":"2023-10-08T09:13:07.852605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:02.469158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:09.196717+0000","last_change":"2023-10-08T09:13:07.863163+0000","last_active":"2023-10-08T09:13:09.196717+0000","last_peered":"2023-10-08T09:13:09.196717+0000","last_clean":"2023-10-08T09:13:09.196717+0000","last_became_active":"2023-10-08T09:13:07.862755+0000","last_became_peered":"2023-10-08T09:13:07.862755+0000","last_unstale":"2023-10-08T09:13:09.196717+0000","last_undegraded":"2023-10-08T09:13:09.196717+0000","last_fullsized":"2023-10-08T09:13:09.196717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:14:25.878241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:31.420326+0000","last_change":"2023-10-08T09:13:02.376641+0000","last_active":"2023-10-08T09:13:31.420326+0000","last_peered":"2023-10-08T09:13:31.420326+0000","last_clean":"2023-10-08T09:13:31.420326+0000","last_became_active":"2023-10-08T09:13:02.376348+0000","last_became_peered":"2023-10-08T09:13:02.376348+0000","last_unstale":"2023-10-08T09:13:31.420326+0000","last_undegraded":"2023-10-08T09:13:31.420326+0000","last_fullsized":"2023-10-08T09:13:31.420326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:42.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.851741+0000","last_change":"2023-10-08T09:13:02.836201+0000","last_active":"2023-10-08T09:13:07.851741+0000","last_peered":"2023-10-08T09:13:07.851741+0000","last_clean":"2023-10-08T09:13:07.851741+0000","last_became_active":"2023-10-08T09:13:02.835885+0000","last_became_peered":"2023-10-08T09:13:02.835885+0000","last_unstale":"2023-10-08T09:13:07.851741+0000","last_undegraded":"2023-10-08T09:13:07.851741+0000","last_fullsized":"2023-10-08T09:13:07.851741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:34.310854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:08.856610+0000","last_change":"2023-10-08T09:13:07.863854+0000","last_active":"2023-10-08T09:13:08.856610+0000","last_peered":"2023-10-08T09:13:08.856610+0000","last_clean":"2023-10-08T09:13:08.856610+0000","last_became_active":"2023-10-08T09:13:07.863460+0000","last_became_peered":"2023-10-08T09:13:07.863460+0000","last_unstale":"2023-10-08T09:13:08.856610+0000","last_undegraded":"2023-10-08T09:13:08.856610+0000","last_fullsized":"2023-10-08T09:13:08.856610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:03.093858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T09:13:32.421449+0000","last_change":"2023-10-08T09:13:02.372336+0000","last_active":"2023-10-08T09:13:32.421449+0000","last_peered":"2023-10-08T09:13:32.421449+0000","last_clean":"2023-10-08T09:13:32.421449+0000","last_became_active":"2023-10-08T09:13:02.372022+0000","last_became_peered":"2023-10-08T09:13:02.372022+0000","last_unstale":"2023-10-08T09:13:32.421449+0000","last_undegraded":"2023-10-08T09:13:32.421449+0000","last_fullsized":"2023-10-08T09:13:32.421449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:10.610332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852815+0000","last_change":"2023-10-08T09:13:02.832430+0000","last_active":"2023-10-08T09:13:07.852815+0000","last_peered":"2023-10-08T09:13:07.852815+0000","last_clean":"2023-10-08T09:13:07.852815+0000","last_became_active":"2023-10-08T09:13:02.832092+0000","last_became_peered":"2023-10-08T09:13:02.832092+0000","last_unstale":"2023-10-08T09:13:07.852815+0000","last_undegraded":"2023-10-08T09:13:07.852815+0000","last_fullsized":"2023-10-08T09:13:07.852815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:00.798900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852833+0000","last_change":"2023-10-08T09:13:02.833114+0000","last_active":"2023-10-08T09:13:07.852833+0000","last_peered":"2023-10-08T09:13:07.852833+0000","last_clean":"2023-10-08T09:13:07.852833+0000","last_became_active":"2023-10-08T09:13:02.830959+0000","last_became_peered":"2023-10-08T09:13:02.830959+0000","last_unstale":"2023-10-08T09:13:07.852833+0000","last_undegraded":"2023-10-08T09:13:07.852833+0000","last_fullsized":"2023-10-08T09:13:07.852833+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:13.853585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847750+0000","last_change":"2023-10-08T09:13:02.836046+0000","last_active":"2023-10-08T09:13:07.847750+0000","last_peered":"2023-10-08T09:13:07.847750+0000","last_clean":"2023-10-08T09:13:07.847750+0000","last_became_active":"2023-10-08T09:13:02.835722+0000","last_became_peered":"2023-10-08T09:13:02.835722+0000","last_unstale":"2023-10-08T09:13:07.847750+0000","last_undegraded":"2023-10-08T09:13:07.847750+0000","last_fullsized":"2023-10-08T09:13:07.847750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:00.316259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847033+0000","last_change":"2023-10-08T09:13:02.830021+0000","last_active":"2023-10-08T09:13:07.847033+0000","last_peered":"2023-10-08T09:13:07.847033+0000","last_clean":"2023-10-08T09:13:07.847033+0000","last_became_active":"2023-10-08T09:13:02.829397+0000","last_became_peered":"2023-10-08T09:13:02.829397+0000","last_unstale":"2023-10-08T09:13:07.847033+0000","last_undegraded":"2023-10-08T09:13:07.847033+0000","last_fullsized":"2023-10-08T09:13:07.847033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:29.153457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.858028+0000","last_change":"2023-10-08T09:13:02.832002+0000","last_active":"2023-10-08T09:13:07.858028+0000","last_peered":"2023-10-08T09:13:07.858028+0000","last_clean":"2023-10-08T09:13:07.858028+0000","last_became_active":"2023-10-08T09:13:02.831666+0000","last_became_peered":"2023-10-08T09:13:02.831666+0000","last_unstale":"2023-10-08T09:13:07.858028+0000","last_undegraded":"2023-10-08T09:13:07.858028+0000","last_fullsized":"2023-10-08T09:13:07.858028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:16.414127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.844926+0000","last_change":"2023-10-08T09:13:02.835881+0000","last_active":"2023-10-08T09:13:07.844926+0000","last_peered":"2023-10-08T09:13:07.844926+0000","last_clean":"2023-10-08T09:13:07.844926+0000","last_became_active":"2023-10-08T09:13:02.835587+0000","last_became_peered":"2023-10-08T09:13:02.835587+0000","last_unstale":"2023-10-08T09:13:07.844926+0000","last_undegraded":"2023-10-08T09:13:07.844926+0000","last_fullsized":"2023-10-08T09:13:07.844926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:52.323234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852003+0000","last_change":"2023-10-08T09:13:02.826371+0000","last_active":"2023-10-08T09:13:07.852003+0000","last_peered":"2023-10-08T09:13:07.852003+0000","last_clean":"2023-10-08T09:13:07.852003+0000","last_became_active":"2023-10-08T09:13:02.826093+0000","last_became_peered":"2023-10-08T09:13:02.826093+0000","last_unstale":"2023-10-08T09:13:07.852003+0000","last_undegraded":"2023-10-08T09:13:07.852003+0000","last_fullsized":"2023-10-08T09:13:07.852003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:08.113739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.847619+0000","last_change":"2023-10-08T09:13:02.829936+0000","last_active":"2023-10-08T09:13:07.847619+0000","last_peered":"2023-10-08T09:13:07.847619+0000","last_clean":"2023-10-08T09:13:07.847619+0000","last_became_active":"2023-10-08T09:13:02.829498+0000","last_became_peered":"2023-10-08T09:13:02.829498+0000","last_unstale":"2023-10-08T09:13:07.847619+0000","last_undegraded":"2023-10-08T09:13:07.847619+0000","last_fullsized":"2023-10-08T09:13:07.847619+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:42.200791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867444+0000","last_change":"2023-10-08T09:13:02.840415+0000","last_active":"2023-10-08T09:13:07.867444+0000","last_peered":"2023-10-08T09:13:07.867444+0000","last_clean":"2023-10-08T09:13:07.867444+0000","last_became_active":"2023-10-08T09:13:02.840075+0000","last_became_peered":"2023-10-08T09:13:02.840075+0000","last_unstale":"2023-10-08T09:13:07.867444+0000","last_undegraded":"2023-10-08T09:13:07.867444+0000","last_fullsized":"2023-10-08T09:13:07.867444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:48.959372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.857306+0000","last_change":"2023-10-08T09:13:02.828337+0000","last_active":"2023-10-08T09:13:07.857306+0000","last_peered":"2023-10-08T09:13:07.857306+0000","last_clean":"2023-10-08T09:13:07.857306+0000","last_became_active":"2023-10-08T09:13:02.827998+0000","last_became_peered":"2023-10-08T09:13:02.827998+0000","last_unstale":"2023-10-08T09:13:07.857306+0000","last_undegraded":"2023-10-08T09:13:07.857306+0000","last_fullsized":"2023-10-08T09:13:07.857306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:27.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.867290+0000","last_change":"2023-10-08T09:13:02.833052+0000","last_active":"2023-10-08T09:13:07.867290+0000","last_peered":"2023-10-08T09:13:07.867290+0000","last_clean":"2023-10-08T09:13:07.867290+0000","last_became_active":"2023-10-08T09:13:02.832635+0000","last_became_peered":"2023-10-08T09:13:02.832635+0000","last_unstale":"2023-10-08T09:13:07.867290+0000","last_undegraded":"2023-10-08T09:13:07.867290+0000","last_fullsized":"2023-10-08T09:13:07.867290+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:50.995768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.846114+0000","last_change":"2023-10-08T09:13:02.840921+0000","last_active":"2023-10-08T09:13:07.846114+0000","last_peered":"2023-10-08T09:13:07.846114+0000","last_clean":"2023-10-08T09:13:07.846114+0000","last_became_active":"2023-10-08T09:13:02.840566+0000","last_became_peered":"2023-10-08T09:13:02.840566+0000","last_unstale":"2023-10-08T09:13:07.846114+0000","last_undegraded":"2023-10-08T09:13:07.846114+0000","last_fullsized":"2023-10-08T09:13:07.846114+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:20.219581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.845198+0000","last_change":"2023-10-08T09:13:02.834491+0000","last_active":"2023-10-08T09:13:07.845198+0000","last_peered":"2023-10-08T09:13:07.845198+0000","last_clean":"2023-10-08T09:13:07.845198+0000","last_became_active":"2023-10-08T09:13:02.834058+0000","last_became_peered":"2023-10-08T09:13:02.834058+0000","last_unstale":"2023-10-08T09:13:07.845198+0000","last_undegraded":"2023-10-08T09:13:07.845198+0000","last_fullsized":"2023-10-08T09:13:07.845198+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:47.699423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-08T09:13:07.852481+0000","last_change":"2023-10-08T09:13:02.836433+0000","last_active":"2023-10-08T09:13:07.852481+0000","last_peered":"2023-10-08T09:13:07.852481+0000","last_clean":"2023-10-08T09:13:07.852481+0000","last_became_active":"2023-10-08T09:13:02.835147+0000","last_became_peered":"2023-10-08T09:13:02.835147+0000","last_unstale":"2023-10-08T09:13:07.852481+0000","last_undegraded":"2023-10-08T09:13:07.852481+0000","last_fullsized":"2023-10-08T09:13:07.852481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:51.233662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":20480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247538,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":45077,"num_objects":89,"num_object_clones":0,"num_object_copies":267,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":54,"num_read_kb":43,"num_write":204,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":59,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":479232,"data_stored":135463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514513,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":463649,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.51700000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.748}]},{"osd":2,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.70699999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.78200000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.83799999999999997}]},{"osd":10,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.82099999999999995}]}]},{"osd":10,"up_from":69,"seq":296352743445,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":1076976,"data_compressed":47757,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.313}]},{"osd":2,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.36199999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.501},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.47099999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.40899999999999997}]},{"osd":6,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.441},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.67500000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.70799999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.61899999999999999}]}]},{"osd":9,"up_from":64,"seq":274877906970,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":468867,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.627}]},{"osd":1,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.49199999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.753},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.67500000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78200000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.70099999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.68999999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.81799999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":8,"up_from":56,"seq":240518168606,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":464056,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.75}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.45000000000000001,"15min":0.436},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.33300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.38}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.42499999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.52700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0229999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.629},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.51100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":0.999},"last":0.71799999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.93100000000000005,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.53900000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.82799999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57199999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":7,"up_from":49,"seq":210453397539,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1061645,"data_compressed":47757,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.41899999999999998,"15min":0.40999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.44500000000000001,"15min":0.433},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67500000000000004,"5min":0.745,"15min":0.745},"last":0.51600000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88200000000000001,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.46899999999999997,"5min":0.435,"15min":0.435},"max":{"1min":0.85199999999999998,"5min":1.518,"15min":1.518},"last":0.72299999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67000000000000004}]},{"osd":4,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48799999999999999,"15min":0.47199999999999998},"min":{"1min":0.32900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.377,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.57499999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69099999999999995,"15min":0.68799999999999994},"min":{"1min":0.503,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.873,"5min":1.587,"15min":1.587},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92200000000000004,"5min":1.496,"15min":1.496},"last":0.82999999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.68000000000000005,"15min":0.66900000000000004},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.68999999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66900000000000004,"15min":0.65900000000000003},"min":{"1min":0.496,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.89900000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.88600000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78500000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":6,"up_from":44,"seq":188978561063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":464761,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.374,"15min":0.36599999999999999},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59799999999999998,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.39000000000000001,"15min":0.378},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.42499999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.53500000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.64600000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.45200000000000001,"15min":0.435},"min":{"1min":0.33600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.752,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.46999999999999997,"15min":0.45600000000000002},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47999999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.59999999999999998,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.626,"15min":0.59999999999999998},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.501,"5min":0.433,"15min":0.433},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.71299999999999997,"15min":0.69399999999999995},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71499999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.55700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92200000000000004,"5min":1.175,"15min":1.175},"last":0.80800000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.76200000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":471765,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.497},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51900000000000002,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.755,"5min":0.999,"15min":0.999},"last":0.39300000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64200000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.495},"min":{"1min":0.38600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.52200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92100000000000004,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.49399999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.39400000000000002},"min":{"1min":0.26900000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.53000000000000003,"5min":0.751,"15min":0.751},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.42299999999999999,"15min":0.41099999999999998},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60899999999999999,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53600000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.623},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.747}]},{"osd":6,"last update":"Sun Oct 8 09:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.628,"15min":0.628},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93700000000000006,"5min":1.034,"15min":1.034},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.45400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89500000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58099999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55300000000000005},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.81599999999999995,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76200000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66900000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68700000000000006}]},{"osd":9,"last update":"Sun Oct 8 09:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65600000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.84299999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":466217,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.36799999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.53900000000000003,"15min":0.50700000000000001},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.59799999999999998,"15min":0.52800000000000002},"min":{"1min":0.53800000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.468,"5min":3.468,"15min":3.468},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.62,"15min":0.55400000000000005},"min":{"1min":0.52900000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.4409999999999998,"5min":3.4409999999999998,"15min":3.4409999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:11:34 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.27600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.45700000000000002,"15min":0.45200000000000001},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Sun Oct 8 09:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.58899999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.47999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79100000000000004,"5min":1.133,"15min":1.133},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.095,"15min":1.095},"last":0.68600000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.747,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.745},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.31900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60499999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.42999999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61499999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0209999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.72899999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.627}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.79800000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78800000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":508654,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80800000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.48699999999999999},"min":{"1min":0.33500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.66900000000000004}]},{"osd":1,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.41099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.318,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.89900000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88600000000000001,"5min":1.758,"15min":1.758},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59399999999999997,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.93100000000000005,"5min":1.772,"15min":1.772},"last":0.93100000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.56100000000000005},"min":{"1min":0.34799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.97299999999999998}]},{"osd":5,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.77000000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.62},"min":{"1min":0.40400000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":1.839,"15min":1.839},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":1.492,"15min":1.492},"last":1.014}]},{"osd":7,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.434,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45500000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95899999999999996}]},{"osd":8,"last update":"Sun Oct 8 09:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.113},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91800000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.877},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.997}]},{"osd":10,"last update":"Sun Oct 8 09:13:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":1.0289999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]}]},{"osd":3,"up_from":25,"seq":107374182451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":463602,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.38800000000000001,"15min":0.374},"min":{"1min":0.316,"5min":0.217,"15min":0.217},"max":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.40000000000000002},"min":{"1min":0.32700000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.59099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.56799999999999995,"15min":0.54200000000000004},"min":{"1min":0.45400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.876,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.629},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.57199999999999995},"min":{"1min":0.42999999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.996,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.58599999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.48099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88500000000000001,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72599999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60599999999999998,"15min":0.59999999999999998},"min":{"1min":0.41299999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1.1399999999999999,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.109,"5min":1.494,"15min":1.494},"last":0.67900000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.89900000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.87},"last":0.79100000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.59999999999999998,"15min":0.58699999999999997},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.56699999999999995,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.52600000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.745},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.76300000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80400000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]}]},{"osd":4,"up_from":32,"seq":137438953518,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":464315,"data_compressed":11692,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.499,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":1.466,"15min":1.466},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.495},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38400000000000001,"15min":0.37},"min":{"1min":0.30499999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.52600000000000002,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79500000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.83699999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54200000000000004,"15min":0.51800000000000002},"min":{"1min":0.46000000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.76900000000000002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.51800000000000002},"min":{"1min":0.44700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.625},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78100000000000003,"5min":1.429,"15min":1.429},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.502,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.876,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.51800000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82899999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.86099999999999999,"5min":0.873,"15min":0.873},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.94499999999999995,"5min":1.353,"15min":1.353},"last":0.65900000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.872,"5min":1.373,"15min":1.373},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.079,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.81999999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.93100000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:13:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":1056837,"data_compressed":47757,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.502},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.497},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.627,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.998,"15min":0.998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.77500000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.27400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.84199999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85799999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.623,"15min":0.60899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.69799999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93400000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.745}]},{"osd":8,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.91900000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:12:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.0149999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":1.052},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.83199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":40689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":40283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:13:36.257 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T09:13:36.257 INFO:tasks.cephadm:Setting up client nodes... 2023-10-08T09:13:36.258 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-08T09:13:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1878820473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-08T09:13:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-10-08T09:13:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-10-08T09:13:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-10-08T09:13:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-08T09:13:36.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-10-08T09:13:36.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-10-08T09:13:36.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2023-10-08T09:13:36.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-10-08T09:13:36.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-10-08T09:13:36.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1878820473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:36.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-10-08T09:13:36.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1878820473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2023-10-08T09:13:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2023-10-08T09:13:36.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2023-10-08T09:13:36.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2023-10-08T09:13:36.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-10-08T09:13:36.869 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:37.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:13:37.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: pgmap v286: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-10-08T09:13:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-10-08T09:13:37.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:37 smithi049 ceph-mon[99485]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T09:13:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:13:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: pgmap v286: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2023-10-08T09:13:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-10-08T09:13:37.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:37 smithi184 ceph-mon[103675]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T09:13:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:13:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: pgmap v286: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2023-10-08T09:13:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:13:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-08T09:13:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-10-08T09:13:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-10-08T09:13:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2023-10-08T09:13:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-10-08T09:13:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2023-10-08T09:13:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2023-10-08T09:13:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2023-10-08T09:13:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2023-10-08T09:13:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-10-08T09:13:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:37 smithi132 ceph-mon[103389]: osdmap e87: 12 total, 12 up, 12 in 2023-10-08T09:13:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 -- 172.21.15.49:0/2803441763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0760101a20 msgr2=0x7f0760101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 --2- 172.21.15.49:0/2803441763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0760101a20 0x7f0760101e80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f07500099c0 tx=0x7f0750009cd0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 -- 172.21.15.49:0/2803441763 shutdown_connections 2023-10-08T09:13:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 --2- 172.21.15.49:0/2803441763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07601023c0 0x7f0760108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 --2- 172.21.15.49:0/2803441763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0760100820 0x7f0760100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 --2- 172.21.15.49:0/2803441763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0760101a20 0x7f0760101e80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 -- 172.21.15.49:0/2803441763 >> 172.21.15.49:0/2803441763 conn(0x7f07600fbdc0 msgr2=0x7f07600fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:38.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 -- 172.21.15.49:0/2803441763 shutdown_connections 2023-10-08T09:13:38.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.077+0000 7f0765fdd700 1 -- 172.21.15.49:0/2803441763 wait complete. 2023-10-08T09:13:38.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.078+0000 7f0765fdd700 1 Processor -- start 2023-10-08T09:13:38.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.078+0000 7f0765fdd700 1 -- start start 2023-10-08T09:13:38.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f0765fdd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0760100820 0x7f0760196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:38.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f0765fdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0760101a20 0x7f0760196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:38.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f0765fdd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07601023c0 0x7f076019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f0765fdd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076019e370 con 0x7f07601023c0 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f0765fdd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076019e4b0 con 0x7f0760101a20 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f0765fdd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076019e640 con 0x7f0760100820 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07601023c0 0x7f076019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07601023c0 0x7f076019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48476/0 (socket says 172.21.15.49:48476) 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 -- 172.21.15.49:0/704165645 learned_addr learned my addr 172.21.15.49:0/704165645 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075f7fe700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0760100820 0x7f0760196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075effd700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0760101a20 0x7f0760196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 -- 172.21.15.49:0/704165645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0760100820 msgr2=0x7f0760196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0760100820 0x7f0760196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 -- 172.21.15.49:0/704165645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0760101a20 msgr2=0x7f0760196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:38.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0760101a20 0x7f0760196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 -- 172.21.15.49:0/704165645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07500095f0 con 0x7f07601023c0 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075f7fe700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0760100820 0x7f0760196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.079+0000 7f075ffff700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07601023c0 0x7f076019dda0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f075800b9a0 tx=0x7f075800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.080+0000 7f075cff9700 1 -- 172.21.15.49:0/704165645 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075800c9b0 con 0x7f07601023c0 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.080+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076019e920 con 0x7f07601023c0 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.080+0000 7f075cff9700 1 -- 172.21.15.49:0/704165645 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0758014440 con 0x7f07601023c0 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.080+0000 7f075cff9700 1 -- 172.21.15.49:0/704165645 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07580075d0 con 0x7f07601023c0 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.080+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076019edd0 con 0x7f07601023c0 2023-10-08T09:13:38.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.080+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0760107c80 con 0x7f07601023c0 2023-10-08T09:13:38.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.082+0000 7f075cff9700 1 -- 172.21.15.49:0/704165645 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f07580145b0 con 0x7f07601023c0 2023-10-08T09:13:38.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.083+0000 7f075cff9700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07480a1510 0x7f07480a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:38.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.083+0000 7f075f7fe700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07480a1510 0x7f07480a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:38.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.083+0000 7f075cff9700 1 -- 172.21.15.49:0/704165645 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0758006e40 con 0x7f07601023c0 2023-10-08T09:13:38.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.083+0000 7f075f7fe700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07480a1510 0x7f07480a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0760101880 tx=0x7f0754008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:38.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.086+0000 7f075cff9700 1 -- 172.21.15.49:0/704165645 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0758096000 con 0x7f07601023c0 2023-10-08T09:13:38.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.367+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f076004e4c0 con 0x7f07601023c0 2023-10-08T09:13:38.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.369+0000 7f075cff9700 1 -- 172.21.15.49:0/704165645 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0758099620 con 0x7f07601023c0 2023-10-08T09:13:38.371 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2023-10-08T09:13:38.371 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQDCciJlbHoEFhAAFfaFgmdvgVH2qDT99YPNng== 2023-10-08T09:13:38.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.371+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07480a1510 msgr2=0x7f07480a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:38.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.371+0000 7f0765fdd700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07480a1510 0x7f07480a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0760101880 tx=0x7f0754008040 comp rx=0 tx=0).stop 2023-10-08T09:13:38.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.371+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07601023c0 msgr2=0x7f076019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:38.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.371+0000 7f0765fdd700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07601023c0 0x7f076019dda0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f075800b9a0 tx=0x7f075800bd60 comp rx=0 tx=0).stop 2023-10-08T09:13:38.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 shutdown_connections 2023-10-08T09:13:38.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07480a1510 0x7f07480a39d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0760100820 0x7f0760196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07601023c0 0x7f076019dda0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 --2- 172.21.15.49:0/704165645 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0760101a20 0x7f0760196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:38.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 >> 172.21.15.49:0/704165645 conn(0x7f07600fbdc0 msgr2=0x7f0760104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:38.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 shutdown_connections 2023-10-08T09:13:38.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:38.372+0000 7f0765fdd700 1 -- 172.21.15.49:0/704165645 wait complete. 2023-10-08T09:13:38.712 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:38 smithi049 ceph-mon[99485]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T09:13:38.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/704165645' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T09:13:38.713 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/704165645' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T09:13:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:38 smithi184 ceph-mon[103675]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T09:13:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/704165645' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T09:13:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/704165645' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T09:13:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:38 smithi132 ceph-mon[103389]: osdmap e88: 12 total, 12 up, 12 in 2023-10-08T09:13:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/704165645' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-08T09:13:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/704165645' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-08T09:13:38.823 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:13:38.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-08T09:13:38.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T09:13:38.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph config log 1 --format=json 2023-10-08T09:13:39.468 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:39.701 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:39 smithi049 ceph-mon[99485]: pgmap v289: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 KiB/s wr, 29 op/s; 15 B/s, 0 objects/s recovering 2023-10-08T09:13:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:39 smithi184 ceph-mon[103675]: pgmap v289: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 KiB/s wr, 29 op/s; 15 B/s, 0 objects/s recovering 2023-10-08T09:13:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:39 smithi132 ceph-mon[103389]: pgmap v289: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 KiB/s wr, 29 op/s; 15 B/s, 0 objects/s recovering 2023-10-08T09:13:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 -- 172.21.15.49:0/3251954188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 msgr2=0x7fdb44105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 --2- 172.21.15.49:0/3251954188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44105ba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009990 tx=0x7fdb34009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 -- 172.21.15.49:0/3251954188 shutdown_connections 2023-10-08T09:13:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 --2- 172.21.15.49:0/3251954188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb44101260 0x7fdb4410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 --2- 172.21.15.49:0/3251954188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44105ba0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 --2- 172.21.15.49:0/3251954188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb441008a0 0x7fdb44100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 -- 172.21.15.49:0/3251954188 >> 172.21.15.49:0/3251954188 conn(0x7fdb440779e0 msgr2=0x7fdb44077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:40.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 -- 172.21.15.49:0/3251954188 shutdown_connections 2023-10-08T09:13:40.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.692+0000 7fdb4887a700 1 -- 172.21.15.49:0/3251954188 wait complete. 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 Processor -- start 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 -- start start 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb441008a0 0x7fdb44072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb44101260 0x7fdb44073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb441a4f60 con 0x7fdb441057c0 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb441a50d0 con 0x7fdb441008a0 2023-10-08T09:13:40.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4887a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb441a5290 con 0x7fdb44101260 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb42d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb42d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48490/0 (socket says 172.21.15.49:48490) 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb42d9d700 1 -- 172.21.15.49:0/1498922978 learned_addr learned my addr 172.21.15.49:0/1498922978 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb4259c700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb441008a0 0x7fdb44072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.693+0000 7fdb41d9b700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb44101260 0x7fdb44073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb42d9d700 1 -- 172.21.15.49:0/1498922978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb44101260 msgr2=0x7fdb44073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb42d9d700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb44101260 0x7fdb44073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb42d9d700 1 -- 172.21.15.49:0/1498922978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb441008a0 msgr2=0x7fdb44072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb42d9d700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb441008a0 0x7fdb44072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb42d9d700 1 -- 172.21.15.49:0/1498922978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340095f0 con 0x7fdb441057c0 2023-10-08T09:13:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb42d9d700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44073570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00ab50 tx=0x7fdb3c00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb337fe700 1 -- 172.21.15.49:0/1498922978 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00f070 con 0x7fdb441057c0 2023-10-08T09:13:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb441a5570 con 0x7fdb441057c0 2023-10-08T09:13:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb337fe700 1 -- 172.21.15.49:0/1498922978 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3c007a80 con 0x7fdb441057c0 2023-10-08T09:13:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb337fe700 1 -- 172.21.15.49:0/1498922978 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c005280 con 0x7fdb441057c0 2023-10-08T09:13:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.694+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb441a5a70 con 0x7fdb441057c0 2023-10-08T09:13:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.695+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb44108d80 con 0x7fdb441057c0 2023-10-08T09:13:40.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.696+0000 7fdb337fe700 1 -- 172.21.15.49:0/1498922978 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fdb3c00c620 con 0x7fdb441057c0 2023-10-08T09:13:40.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.697+0000 7fdb337fe700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb2c0a1510 0x7fdb2c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:40.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.697+0000 7fdb4259c700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb2c0a1510 0x7fdb2c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:40.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.697+0000 7fdb337fe700 1 -- 172.21.15.49:0/1498922978 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdb3c0cc630 con 0x7fdb441057c0 2023-10-08T09:13:40.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.697+0000 7fdb4259c700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb2c0a1510 0x7fdb2c0a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006010 tx=0x7fdb34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:40.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.701+0000 7fdb337fe700 1 -- 172.21.15.49:0/1498922978 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdb3c0958b0 con 0x7fdb441057c0 2023-10-08T09:13:40.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.901+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7fdb44073fd0 con 0x7fdb441057c0 2023-10-08T09:13:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.901+0000 7fdb337fe700 1 -- 172.21.15.49:0/1498922978 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+133 (secure 0 0 0) 0x7fdb3c098ed0 con 0x7fdb441057c0 2023-10-08T09:13:40.903 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb2c0a1510 msgr2=0x7fdb2c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb2c0a1510 0x7fdb2c0a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006010 tx=0x7fdb34005960 comp rx=0 tx=0).stop 2023-10-08T09:13:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 msgr2=0x7fdb44073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44073570 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00ab50 tx=0x7fdb3c00ae60 comp rx=0 tx=0).stop 2023-10-08T09:13:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 shutdown_connections 2023-10-08T09:13:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb2c0a1510 0x7fdb2c0a39d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb441057c0 0x7fdb44073570 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb441008a0 0x7fdb44072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 --2- 172.21.15.49:0/1498922978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb44101260 0x7fdb44073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.903+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 >> 172.21.15.49:0/1498922978 conn(0x7fdb440779e0 msgr2=0x7fdb440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.904+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 shutdown_connections 2023-10-08T09:13:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:40.904+0000 7fdb4887a700 1 -- 172.21.15.49:0/1498922978 wait complete. 2023-10-08T09:13:41.321 INFO:teuthology.orchestra.run.smithi049.stdout:[{"version":43,"timestamp":"2023-10-08T09:12:56.353070+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-10-08T09:13:41.322 INFO:tasks.ceph_manager:config epoch is 43 2023-10-08T09:13:41.322 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T09:13:41.322 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-08T09:13:41.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-08T09:13:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:41 smithi049 ceph-mon[99485]: pgmap v290: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 31 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:41 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1498922978' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T09:13:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:41 smithi184 ceph-mon[103675]: pgmap v290: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 31 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:41 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1498922978' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T09:13:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:41 smithi132 ceph-mon[103389]: pgmap v290: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 31 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:41 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1498922978' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-08T09:13:41.890 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:43.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 -- 172.21.15.49:0/1546393080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c1056b0 msgr2=0x7f729c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:43.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 --2- 172.21.15.49:0/1546393080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c1056b0 0x7f729c105a90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f728c009990 tx=0x7f728c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 -- 172.21.15.49:0/1546393080 shutdown_connections 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 --2- 172.21.15.49:0/1546393080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f729c100110 0x7f729c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 --2- 172.21.15.49:0/1546393080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c1056b0 0x7f729c105a90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 --2- 172.21.15.49:0/1546393080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f729c0ff750 0x7f729c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 -- 172.21.15.49:0/1546393080 >> 172.21.15.49:0/1546393080 conn(0x7f729c075790 msgr2=0x7f729c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 -- 172.21.15.49:0/1546393080 shutdown_connections 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.218+0000 7f72a29af700 1 -- 172.21.15.49:0/1546393080 wait complete. 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 Processor -- start 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 -- start start 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c0ff750 0x7f729c198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:43.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f729c100110 0x7f729c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f729c1056b0 0x7f729c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c19dd00 con 0x7f729c0ff750 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c19de70 con 0x7f729c1056b0 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a29af700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c19e030 con 0x7f729c100110 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f729bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c0ff750 0x7f729c198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f729bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c0ff750 0x7f729c198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48500/0 (socket says 172.21.15.49:48500) 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f729bfff700 1 -- 172.21.15.49:0/696763923 learned_addr learned my addr 172.21.15.49:0/696763923 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72937fe700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f729c100110 0x7f729c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.219+0000 7f72a0f4c700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f729c1056b0 0x7f729c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f729bfff700 1 -- 172.21.15.49:0/696763923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f729c100110 msgr2=0x7f729c199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f729bfff700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f729c100110 0x7f729c199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f729bfff700 1 -- 172.21.15.49:0/696763923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f729c1056b0 msgr2=0x7f729c19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f729bfff700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f729c1056b0 0x7f729c19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f729bfff700 1 -- 172.21.15.49:0/696763923 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0095f0 con 0x7f729c0ff750 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f72a0f4c700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f729c1056b0 0x7f729c19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f729bfff700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c0ff750 0x7f729c198e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f728c0098e0 tx=0x7f728c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f7299ffb700 1 -- 172.21.15.49:0/696763923 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c01d070 con 0x7f729c0ff750 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c19e2b0 con 0x7f729c0ff750 2023-10-08T09:13:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f7299ffb700 1 -- 172.21.15.49:0/696763923 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f728c022470 con 0x7f729c0ff750 2023-10-08T09:13:43.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f7299ffb700 1 -- 172.21.15.49:0/696763923 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c017650 con 0x7f729c0ff750 2023-10-08T09:13:43.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.220+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c1a5b70 con 0x7f729c0ff750 2023-10-08T09:13:43.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.221+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c100df0 con 0x7f729c0ff750 2023-10-08T09:13:43.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.223+0000 7f7299ffb700 1 -- 172.21.15.49:0/696763923 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f728c0225e0 con 0x7f729c0ff750 2023-10-08T09:13:43.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.223+0000 7f7299ffb700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72880a1510 0x7f72880a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:43.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.223+0000 7f72937fe700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72880a1510 0x7f72880a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:43.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.224+0000 7f7299ffb700 1 -- 172.21.15.49:0/696763923 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f728c0ce5c0 con 0x7f729c0ff750 2023-10-08T09:13:43.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.224+0000 7f72937fe700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72880a1510 0x7f72880a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f729c19a360 tx=0x7f72840073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:43.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.227+0000 7f7299ffb700 1 -- 172.21.15.49:0/696763923 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f728c0977e0 con 0x7f729c0ff750 2023-10-08T09:13:43.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.489+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f729c19a110 con 0x7f729c0ff750 2023-10-08T09:13:43.495 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.494+0000 7f7299ffb700 1 -- 172.21.15.49:0/696763923 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260406 (secure 0 0 0) 0x7f728c09ae00 con 0x7f729c0ff750 2023-10-08T09:13:43.498 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:43.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72880a1510 msgr2=0x7f72880a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:43.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72880a1510 0x7f72880a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f729c19a360 tx=0x7f72840073d0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c0ff750 msgr2=0x7f729c198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:43.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c0ff750 0x7f729c198e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f728c0098e0 tx=0x7f728c00f670 comp rx=0 tx=0).stop 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 shutdown_connections 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f72880a1510 0x7f72880a39d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f729c1056b0 0x7f729c19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f729c0ff750 0x7f729c198e00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 --2- 172.21.15.49:0/696763923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f729c100110 0x7f729c199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 >> 172.21.15.49:0/696763923 conn(0x7f729c075790 msgr2=0x7f729c0fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 shutdown_connections 2023-10-08T09:13:43.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:43.497+0000 7f72a29af700 1 -- 172.21.15.49:0/696763923 wait complete. 2023-10-08T09:13:43.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:43 smithi184 ceph-mon[103675]: pgmap v291: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 23 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:43.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:43 smithi132 ceph-mon[103389]: pgmap v291: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 23 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:43.860 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:43 smithi049 ceph-mon[99485]: pgmap v291: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 23 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:43.949 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1314466902},{"type":"v1","addr":"172.21.15.49:6801","nonce":1314466902}]},"active_addr":"172.21.15.49:6801/1314466902","active_change":"2023-10-08T09:06:35.771694+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.49:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":567267847}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3236157652}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":889425091}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":3838045796}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.49:0","nonce":1367368925}]}]} 2023-10-08T09:13:43.955 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-08T09:13:43.955 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-08T09:13:43.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T09:13:44.523 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:44.717 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:44 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/696763923' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T09:13:44.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:44 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/696763923' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T09:13:44.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:44 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/696763923' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-08T09:13:45.524 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:45 smithi049 ceph-mon[99485]: pgmap v292: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 -- 172.21.15.49:0/3480538487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8460101260 msgr2=0x7f846010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 --2- 172.21.15.49:0/3480538487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8460101260 0x7f846010d650 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f845800b0a0 tx=0x7f845800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 -- 172.21.15.49:0/3480538487 shutdown_connections 2023-10-08T09:13:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 --2- 172.21.15.49:0/3480538487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f84601057c0 0x7f8460105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 --2- 172.21.15.49:0/3480538487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8460101260 0x7f846010d650 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 --2- 172.21.15.49:0/3480538487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84601008a0 0x7f8460100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 -- 172.21.15.49:0/3480538487 >> 172.21.15.49:0/3480538487 conn(0x7f84600779e0 msgr2=0x7f8460077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 -- 172.21.15.49:0/3480538487 shutdown_connections 2023-10-08T09:13:45.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.695+0000 7f846677d700 1 -- 172.21.15.49:0/3480538487 wait complete. 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 Processor -- start 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 -- start start 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84601008a0 0x7f8460072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8460101260 0x7f8460073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84601057c0 0x7f8460073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a50c0 con 0x7f84601057c0 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a5230 con 0x7f84601008a0 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f846677d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84601a53f0 con 0x7f8460101260 2023-10-08T09:13:45.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f8464d1a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84601057c0 0x7f8460073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f8464d1a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84601057c0 0x7f8460073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48518/0 (socket says 172.21.15.49:48518) 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.696+0000 7f8464d1a700 1 -- 172.21.15.49:0/699639972 learned_addr learned my addr 172.21.15.49:0/699639972 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f845ffff700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84601008a0 0x7f8460072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f845f7fe700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8460101260 0x7f8460073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f8464d1a700 1 -- 172.21.15.49:0/699639972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8460101260 msgr2=0x7f8460073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f8464d1a700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8460101260 0x7f8460073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f8464d1a700 1 -- 172.21.15.49:0/699639972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84601008a0 msgr2=0x7f8460072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f8464d1a700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84601008a0 0x7f8460072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f8464d1a700 1 -- 172.21.15.49:0/699639972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458009d00 con 0x7f84601057c0 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f8464d1a700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84601057c0 0x7f8460073570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f845800b070 tx=0x7f84580094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:45.701 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f845d7fa700 1 -- 172.21.15.49:0/699639972 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800e040 con 0x7f84601057c0 2023-10-08T09:13:45.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f846677d700 1 -- 172.21.15.49:0/699639972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84601a5670 con 0x7f84601057c0 2023-10-08T09:13:45.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f845d7fa700 1 -- 172.21.15.49:0/699639972 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84580078d0 con 0x7f84601057c0 2023-10-08T09:13:45.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f845d7fa700 1 -- 172.21.15.49:0/699639972 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845801b450 con 0x7f84601057c0 2023-10-08T09:13:45.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.697+0000 7f846677d700 1 -- 172.21.15.49:0/699639972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84601a5b30 con 0x7f84601057c0 2023-10-08T09:13:45.702 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.698+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84440052f0 con 0x7f84601057c0 2023-10-08T09:13:45.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.699+0000 7f845d7fa700 1 -- 172.21.15.49:0/699639972 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f845801d580 con 0x7f84601057c0 2023-10-08T09:13:45.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.700+0000 7f845d7fa700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f844c0a1510 0x7f844c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:45.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.700+0000 7f845d7fa700 1 -- 172.21.15.49:0/699639972 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f84580ce590 con 0x7f84601057c0 2023-10-08T09:13:45.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.700+0000 7f845ffff700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f844c0a1510 0x7f844c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:45.703 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.700+0000 7f845ffff700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f844c0a1510 0x7f844c0a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8460101930 tx=0x7f845000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:45.705 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.704+0000 7f845d7fa700 1 -- 172.21.15.49:0/699639972 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8458097810 con 0x7f84601057c0 2023-10-08T09:13:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:45 smithi184 ceph-mon[103675]: pgmap v292: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:45 smithi132 ceph-mon[103389]: pgmap v292: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s; 15 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:45.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.904+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8444005160 con 0x7f84601057c0 2023-10-08T09:13:45.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.906+0000 7f845d7fa700 1 -- 172.21.15.49:0/699639972 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24441 (secure 0 0 0) 0x7f845809ae30 con 0x7f84601057c0 2023-10-08T09:13:45.908 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:45.908 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:37.433726+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f844c0a1510 msgr2=0x7f844c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f844c0a1510 0x7f844c0a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8460101930 tx=0x7f845000b410 comp rx=0 tx=0).stop 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84601057c0 msgr2=0x7f8460073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84601057c0 0x7f8460073570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f845800b070 tx=0x7f84580094c0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 shutdown_connections 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f844c0a1510 0x7f844c0a39d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84601057c0 0x7f8460073570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84601008a0 0x7f8460072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 --2- 172.21.15.49:0/699639972 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8460101260 0x7f8460073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:45.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 >> 172.21.15.49:0/699639972 conn(0x7f84600779e0 msgr2=0x7f84600fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:45.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.908+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 shutdown_connections 2023-10-08T09:13:45.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:45.909+0000 7f8456ffd700 1 -- 172.21.15.49:0/699639972 wait complete. 2023-10-08T09:13:46.276 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-08T09:13:46.276 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd dump --format=json 2023-10-08T09:13:46.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:46 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/699639972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:46.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:46 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/699639972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:46.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:46 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/699639972' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:46.844 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:47 smithi184 ceph-mon[103675]: pgmap v293: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 KiB/s wr, 14 op/s; 13 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:47 smithi132 ceph-mon[103389]: pgmap v293: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 KiB/s wr, 14 op/s; 13 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:47.869 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:47 smithi049 ceph-mon[99485]: pgmap v293: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.4 KiB/s wr, 14 op/s; 13 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:48.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 -- 172.21.15.49:0/188969097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c1070f0 msgr2=0x7f708c1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:48.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 --2- 172.21.15.49:0/188969097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c1070f0 0x7f708c1008c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7088009990 tx=0x7f7088009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 -- 172.21.15.49:0/188969097 shutdown_connections 2023-10-08T09:13:48.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 --2- 172.21.15.49:0/188969097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708c1067d0 0x7f708c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 --2- 172.21.15.49:0/188969097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c100ff0 0x7f708c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 --2- 172.21.15.49:0/188969097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c1070f0 0x7f708c1008c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 -- 172.21.15.49:0/188969097 >> 172.21.15.49:0/188969097 conn(0x7f708c077910 msgr2=0x7f708c077d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:48.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 -- 172.21.15.49:0/188969097 shutdown_connections 2023-10-08T09:13:48.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.172+0000 7f7093860700 1 -- 172.21.15.49:0/188969097 wait complete. 2023-10-08T09:13:48.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.173+0000 7f7093860700 1 Processor -- start 2023-10-08T09:13:48.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.173+0000 7f7093860700 1 -- start start 2023-10-08T09:13:48.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f7093860700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c100ff0 0x7f708c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:48.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f7093860700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c1067d0 0x7f708c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f7093860700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708c196ae0 0x7f708c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f7093860700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c197560 con 0x7f708c100ff0 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f7093860700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1976d0 con 0x7f708c1067d0 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f7093860700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c197840 con 0x7f708c196ae0 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c100ff0 0x7f708c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c100ff0 0x7f708c19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48526/0 (socket says 172.21.15.49:48526) 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 -- 172.21.15.49:0/374342572 learned_addr learned my addr 172.21.15.49:0/374342572 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f7090dfb700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c1067d0 0x7f708c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 -- 172.21.15.49:0/374342572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708c196ae0 msgr2=0x7f708c196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708c196ae0 0x7f708c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 -- 172.21.15.49:0/374342572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c1067d0 msgr2=0x7f708c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c1067d0 0x7f708c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 -- 172.21.15.49:0/374342572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880095f0 con 0x7f708c100ff0 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70915fc700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c100ff0 0x7f708c19c9a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c920 tx=0x7f707c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.174+0000 7f70827fc700 1 -- 172.21.15.49:0/374342572 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c007b00 con 0x7f708c100ff0 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.175+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1a3820 con 0x7f708c100ff0 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.175+0000 7f70827fc700 1 -- 172.21.15.49:0/374342572 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f707c00ce70 con 0x7f708c100ff0 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.175+0000 7f70827fc700 1 -- 172.21.15.49:0/374342572 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c005460 con 0x7f708c100ff0 2023-10-08T09:13:48.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.175+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c1a3d40 con 0x7f708c100ff0 2023-10-08T09:13:48.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.176+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c108d30 con 0x7f708c100ff0 2023-10-08T09:13:48.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.177+0000 7f70827fc700 1 -- 172.21.15.49:0/374342572 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f707c004750 con 0x7f708c100ff0 2023-10-08T09:13:48.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.177+0000 7f70827fc700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f70780a1510 0x7f70780a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:48.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.177+0000 7f70827fc700 1 -- 172.21.15.49:0/374342572 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f707c0ccd80 con 0x7f708c100ff0 2023-10-08T09:13:48.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.177+0000 7f7090dfb700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f70780a1510 0x7f70780a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:48.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.178+0000 7f7090dfb700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f70780a1510 0x7f70780a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f708800b5c0 tx=0x7f7088005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:48.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.182+0000 7f70827fc700 1 -- 172.21.15.49:0/374342572 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f707c095f70 con 0x7f708c100ff0 2023-10-08T09:13:48.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.385+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f708c101f00 con 0x7f708c100ff0 2023-10-08T09:13:48.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.386+0000 7f70827fc700 1 -- 172.21.15.49:0/374342572 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24441 (secure 0 0 0) 0x7f708c101f00 con 0x7f708c100ff0 2023-10-08T09:13:48.387 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:13:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:37.433726+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f70780a1510 msgr2=0x7f70780a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f70780a1510 0x7f70780a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f708800b5c0 tx=0x7f7088005fd0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c100ff0 msgr2=0x7f708c19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c100ff0 0x7f708c19c9a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c920 tx=0x7f707c00cc30 comp rx=0 tx=0).stop 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 shutdown_connections 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f70780a1510 0x7f70780a39d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708c196ae0 0x7f708c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f708c100ff0 0x7f708c19c9a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 --2- 172.21.15.49:0/374342572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f708c1067d0 0x7f708c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 >> 172.21.15.49:0/374342572 conn(0x7f708c077910 msgr2=0x7f708c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 shutdown_connections 2023-10-08T09:13:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:48.388+0000 7f7093860700 1 -- 172.21.15.49:0/374342572 wait complete. 2023-10-08T09:13:48.716 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:48 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/374342572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:48 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/374342572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:48 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/374342572' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:13:48.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-08T09:13:48.799 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-08T09:13:48.799 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-08T09:13:48.800 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-08T09:13:48.800 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-08T09:13:48.800 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-08T09:13:48.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-08T09:13:48.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-08T09:13:48.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-08T09:13:48.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-08T09:13:48.803 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-08T09:13:48.803 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-08T09:13:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:49 smithi184 ceph-mon[103675]: pgmap v294: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:49 smithi132 ceph-mon[103389]: pgmap v294: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:49.833 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:49.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:49 smithi049 ceph-mon[99485]: pgmap v294: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 2.0 KiB/s wr, 11 op/s; 11 B/s, 1 keys/s, 0 objects/s recovering 2023-10-08T09:13:49.981 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:49.982 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.173 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.449 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.451 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.452 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.452 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.452 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.520 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.521 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:50.521 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:13:51.478 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:51 smithi049 ceph-mon[99485]: pgmap v295: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-10-08T09:13:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:51 smithi184 ceph-mon[103675]: pgmap v295: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-10-08T09:13:51.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:51 smithi132 ceph-mon[103389]: pgmap v295: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-10-08T09:13:53.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 -- 172.21.15.49:0/1577192023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 msgr2=0x7f53f00ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:53.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 --2- 172.21.15.49:0/1577192023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 0x7f53f00ffbd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009990 tx=0x7f53e4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 -- 172.21.15.49:0/1577192023 shutdown_connections 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 --2- 172.21.15.49:0/1577192023 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f0100110 0x7f53f010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 --2- 172.21.15.49:0/1577192023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f01056b0 0x7f53f0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 --2- 172.21.15.49:0/1577192023 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 0x7f53f00ffbd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 -- 172.21.15.49:0/1577192023 >> 172.21.15.49:0/1577192023 conn(0x7f53f0075790 msgr2=0x7f53f0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 -- 172.21.15.49:0/1577192023 shutdown_connections 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 -- 172.21.15.49:0/1577192023 wait complete. 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.130+0000 7f53f643b700 1 Processor -- start 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f643b700 1 -- start start 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f643b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 0x7f53f019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:53.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f643b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f0100110 0x7f53f019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f643b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f01056b0 0x7f53f0194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f643b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f0194ea0 con 0x7f53f00ff750 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f643b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f0195010 con 0x7f53f01056b0 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f643b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01951d0 con 0x7f53f0100110 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53effff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 0x7f53f019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53effff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 0x7f53f019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48538/0 (socket says 172.21.15.49:48538) 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53effff700 1 -- 172.21.15.49:0/2366056361 learned_addr learned my addr 172.21.15.49:0/2366056361 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53ef7fe700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f0100110 0x7f53f019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.131+0000 7f53f49d8700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f01056b0 0x7f53f0194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53effff700 1 -- 172.21.15.49:0/2366056361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f0100110 msgr2=0x7f53f019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:53.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53effff700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f0100110 0x7f53f019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53effff700 1 -- 172.21.15.49:0/2366056361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f01056b0 msgr2=0x7f53f0194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53effff700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53f01056b0 0x7f53f0194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53effff700 1 -- 172.21.15.49:0/2366056361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40095f0 con 0x7f53f00ff750 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53ef7fe700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53f0100110 0x7f53f019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53effff700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 0x7f53f019a6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f53e000d980 tx=0x7f53e000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000bb60 con 0x7f53f00ff750 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e0007640 con 0x7f53f00ff750 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f01954b0 con 0x7f53f00ff750 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0004e90 con 0x7f53f00ff750 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.132+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f01a5af0 con 0x7f53f00ff750 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.133+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53f010e810 con 0x7f53f00ff750 2023-10-08T09:13:53.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.134+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f53e000bcc0 con 0x7f53f00ff750 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53ed7fa700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d80a14f0 0x7f53d80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53e00cd6d0 con 0x7f53f00ff750 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53ef7fe700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d80a14f0 0x7f53d80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53ed7fa700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] conn(0x7f53d80ac3a0 0x7f53d80ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 --> [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53d80aef10 con 0x7f53d80ac3a0 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e00cdae0 con 0x7f53f00ff750 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53f49d8700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] conn(0x7f53d80ac3a0 0x7f53d80ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.135+0000 7f53ef7fe700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d80a14f0 0x7f53d80a39b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f53e4006010 tx=0x7f53e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.136+0000 7f53f49d8700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] conn(0x7f53d80ac3a0 0x7f53d80ae840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.136+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== osd.9 v2:172.21.15.49:6826/2343500504 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f53d80aef10 con 0x7f53d80ac3a0 2023-10-08T09:13:53.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.152+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 --> [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f53f0060e70 con 0x7f53d80ac3a0 2023-10-08T09:13:53.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53ed7fa700 1 -- 172.21.15.49:0/2366056361 <== osd.9 v2:172.21.15.49:6826/2343500504 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f53f0060e70 con 0x7f53d80ac3a0 2023-10-08T09:13:53.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] conn(0x7f53d80ac3a0 msgr2=0x7f53d80ae840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:53.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] conn(0x7f53d80ac3a0 0x7f53d80ae840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:53.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d80a14f0 msgr2=0x7f53d80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:53.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d80a14f0 0x7f53d80a39b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f53e4006010 tx=0x7f53e401a040 comp rx=0 tx=0).stop 2023-10-08T09:13:53.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 msgr2=0x7f53f019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:53.365 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 --2- 172.21.15.49:0/2366056361 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f00ff750 0x7f53f019a6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f53e000d980 tx=0x7f53e000dd40 comp rx=0 tx=0).stop 2023-10-08T09:13:53.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f49d8700 1 -- 172.21.15.49:0/2366056361 reap_dead start 2023-10-08T09:13:53.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 shutdown_connections 2023-10-08T09:13:53.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 >> 172.21.15.49:0/2366056361 conn(0x7f53f0075790 msgr2=0x7f53f00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:53.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 shutdown_connections 2023-10-08T09:13:53.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:53.153+0000 7f53f643b700 1 -- 172.21.15.49:0/2366056361 wait complete. 2023-10-08T09:13:53.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:53 smithi184 ceph-mon[103675]: pgmap v296: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:13:53.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:53 smithi049 ceph-mon[99485]: pgmap v296: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:13:53.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:53 smithi132 ceph-mon[103389]: pgmap v296: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:13:55.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:55 smithi184 ceph-mon[103675]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:13:55.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:55 smithi049 ceph-mon[99485]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:13:55.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:55 smithi132 ceph-mon[103389]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:13:56.225 INFO:teuthology.orchestra.run.smithi049.stdout:274877906974 2023-10-08T09:13:56.287 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-08T09:13:56.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 -- 172.21.15.49:0/4041112510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8081056c0 msgr2=0x7fb808105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 --2- 172.21.15.49:0/4041112510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8081056c0 0x7fb808105aa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb804009990 tx=0x7fb804009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 -- 172.21.15.49:0/4041112510 shutdown_connections 2023-10-08T09:13:56.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 --2- 172.21.15.49:0/4041112510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb808100810 0x7fb80810c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 --2- 172.21.15.49:0/4041112510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8081056c0 0x7fb808105aa0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 --2- 172.21.15.49:0/4041112510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8080ffe50 0x7fb8081002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 -- 172.21.15.49:0/4041112510 >> 172.21.15.49:0/4041112510 conn(0x7fb808075cf0 msgr2=0x7fb8080fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 -- 172.21.15.49:0/4041112510 shutdown_connections 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 -- 172.21.15.49:0/4041112510 wait complete. 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.244+0000 7fb8107b8700 1 Processor -- start 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb8107b8700 1 -- start start 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb8107b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8080ffe50 0x7fb8081979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb8107b8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb808100810 0x7fb808197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb8107b8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8081056c0 0x7fb8081984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb8107b8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081a2d00 con 0x7fb808100810 2023-10-08T09:13:56.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb8107b8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081a2e70 con 0x7fb8080ffe50 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb8107b8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081a3030 con 0x7fb8081056c0 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb808100810 0x7fb808197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb808100810 0x7fb808197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48556/0 (socket says 172.21.15.49:48556) 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 -- 172.21.15.49:0/1837584968 learned_addr learned my addr 172.21.15.49:0/1837584968 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80e554700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8080ffe50 0x7fb8081979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80ed55700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8081056c0 0x7fb8081984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 -- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8081056c0 msgr2=0x7fb8081984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8081056c0 0x7fb8081984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 -- 172.21.15.49:0/1837584968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8080ffe50 msgr2=0x7fb8081979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8080ffe50 0x7fb8081979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 -- 172.21.15.49:0/1837584968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040095f0 con 0x7fb808100810 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80ed55700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8081056c0 0x7fb8081984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb80dd53700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb808100810 0x7fb808197f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800d950 tx=0x7fb7f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.245+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8009940 con 0x7fb808100810 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.246+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7f8010460 con 0x7fb808100810 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.246+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800f620 con 0x7fb808100810 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.246+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8081a3310 con 0x7fb808100810 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.246+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8081a3810 con 0x7fb808100810 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.246+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb80810f910 con 0x7fb808100810 2023-10-08T09:13:56.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.248+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fb7f80108d0 con 0x7fb808100810 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.248+0000 7fb7ff7fe700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7f40a1590 0x7fb7f40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.248+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb7f80cd480 con 0x7fb808100810 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.248+0000 7fb80e554700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7f40a1590 0x7fb7f40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.248+0000 7fb7ff7fe700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] conn(0x7fb7f40ac5b0 0x7fb7f40aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.248+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 --> [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7f40af120 con 0x7fb7f40ac5b0 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.248+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7f80cd890 con 0x7fb808100810 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.249+0000 7fb80e554700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7f40a1590 0x7fb7f40a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb8040098e0 tx=0x7fb804005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.249+0000 7fb80ed55700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] conn(0x7fb7f40ac5b0 0x7fb7f40aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.529 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.249+0000 7fb80ed55700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] conn(0x7fb7f40ac5b0 0x7fb7f40aea50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:56.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.250+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== osd.5 v2:172.21.15.184:6808/3484166138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb7f40af120 con 0x7fb7f40ac5b0 2023-10-08T09:13:56.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.266+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 --> [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb808060ef0 con 0x7fb7f40ac5b0 2023-10-08T09:13:56.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.267+0000 7fb7ff7fe700 1 -- 172.21.15.49:0/1837584968 <== osd.5 v2:172.21.15.184:6808/3484166138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb808060ef0 con 0x7fb7f40ac5b0 2023-10-08T09:13:56.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.267+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] conn(0x7fb7f40ac5b0 msgr2=0x7fb7f40aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.267+0000 7fb8107b8700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] conn(0x7fb7f40ac5b0 0x7fb7f40aea50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.530 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.267+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7f40a1590 msgr2=0x7fb7f40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.267+0000 7fb8107b8700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7f40a1590 0x7fb7f40a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb8040098e0 tx=0x7fb804005960 comp rx=0 tx=0).stop 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.268+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb808100810 msgr2=0x7fb808197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.268+0000 7fb8107b8700 1 --2- 172.21.15.49:0/1837584968 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb808100810 0x7fb808197f30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800d950 tx=0x7fb7f800dc60 comp rx=0 tx=0).stop 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.268+0000 7fb80ed55700 1 -- 172.21.15.49:0/1837584968 reap_dead start 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.268+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 shutdown_connections 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.268+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 >> 172.21.15.49:0/1837584968 conn(0x7fb808075cf0 msgr2=0x7fb808102500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.268+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 shutdown_connections 2023-10-08T09:13:56.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.268+0000 7fb8107b8700 1 -- 172.21.15.49:0/1837584968 wait complete. 2023-10-08T09:13:56.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 -- 172.21.15.49:0/349243849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 msgr2=0x7f857010d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 --2- 172.21.15.49:0/349243849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 0x7f857010d6a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f856800b0a0 tx=0x7f856800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 -- 172.21.15.49:0/349243849 shutdown_connections 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 --2- 172.21.15.49:0/349243849 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85701057a0 0x7f8570105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 --2- 172.21.15.49:0/349243849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 0x7f857010d6a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 --2- 172.21.15.49:0/349243849 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8570100840 0x7f8570100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 -- 172.21.15.49:0/349243849 >> 172.21.15.49:0/349243849 conn(0x7f8570077950 msgr2=0x7f8570077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 -- 172.21.15.49:0/349243849 shutdown_connections 2023-10-08T09:13:56.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.567+0000 7f8576455700 1 -- 172.21.15.49:0/349243849 wait complete. 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 Processor -- start 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 -- start start 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570100840 0x7f857019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 0x7f857019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85701057a0 0x7f8570196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701971e0 con 0x7f8570101200 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8570197350 con 0x7f85701057a0 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f8576455700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8570197510 con 0x7f8570100840 2023-10-08T09:13:56.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f856f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 0x7f857019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f856f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 0x7f857019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48572/0 (socket says 172.21.15.49:48572) 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f856f7fe700 1 -- 172.21.15.49:0/1057045816 learned_addr learned my addr 172.21.15.49:0/1057045816 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.568+0000 7f856ffff700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570100840 0x7f857019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f85749f2700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85701057a0 0x7f8570196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856f7fe700 1 -- 172.21.15.49:0/1057045816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570100840 msgr2=0x7f857019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856f7fe700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8570100840 0x7f857019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856f7fe700 1 -- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85701057a0 msgr2=0x7f8570196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856f7fe700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85701057a0 0x7f8570196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856f7fe700 1 -- 172.21.15.49:0/1057045816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8568009d00 con 0x7f8570101200 2023-10-08T09:13:56.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856f7fe700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 0x7f857019cf80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f856400e9a0 tx=0x7f856400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564009790 con 0x7f8570101200 2023-10-08T09:13:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85701977f0 con 0x7f8570101200 2023-10-08T09:13:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8564010430 con 0x7f8570101200 2023-10-08T09:13:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564018680 con 0x7f8570101200 2023-10-08T09:13:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.569+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85701a3a30 con 0x7f8570101200 2023-10-08T09:13:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.570+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85701109c0 con 0x7f8570101200 2023-10-08T09:13:56.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.572+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f85640105a0 con 0x7f8570101200 2023-10-08T09:13:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.572+0000 7f856d7fa700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f85580a1510 0x7f85580a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.572+0000 7f856ffff700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f85580a1510 0x7f85580a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.572+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8564014070 con 0x7f8570101200 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.573+0000 7f856d7fa700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] conn(0x7f85580ac530 0x7f85580ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.573+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 --> [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85580af0a0 con 0x7f85580ac530 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.573+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85640cda10 con 0x7f8570101200 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.573+0000 7f856ffff700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f85580a1510 0x7f85580a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8560009be0 tx=0x7f8560009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.573+0000 7f85749f2700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] conn(0x7f85580ac530 0x7f85580ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.573+0000 7f85749f2700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] conn(0x7f85580ac530 0x7f85580ae9d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:56.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.574+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== osd.7 v2:172.21.15.132:6816/1427224102 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f85580af0a0 con 0x7f85580ac530 2023-10-08T09:13:56.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.590+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 --> [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8570060e70 con 0x7f85580ac530 2023-10-08T09:13:56.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f856d7fa700 1 -- 172.21.15.49:0/1057045816 <== osd.7 v2:172.21.15.132:6816/1427224102 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8570060e70 con 0x7f85580ac530 2023-10-08T09:13:56.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] conn(0x7f85580ac530 msgr2=0x7f85580ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f8576455700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] conn(0x7f85580ac530 0x7f85580ae9d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f85580a1510 msgr2=0x7f85580a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f8576455700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f85580a1510 0x7f85580a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8560009be0 tx=0x7f8560009260 comp rx=0 tx=0).stop 2023-10-08T09:13:56.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 msgr2=0x7f857019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:56.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f8576455700 1 --2- 172.21.15.49:0/1057045816 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8570101200 0x7f857019cf80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f856400e9a0 tx=0x7f856400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:13:56.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.591+0000 7f85749f2700 1 -- 172.21.15.49:0/1057045816 reap_dead start 2023-10-08T09:13:56.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.592+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 shutdown_connections 2023-10-08T09:13:56.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.592+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 >> 172.21.15.49:0/1057045816 conn(0x7f8570077950 msgr2=0x7f85700fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:56.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.592+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 shutdown_connections 2023-10-08T09:13:56.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:56.592+0000 7f8576455700 1 -- 172.21.15.49:0/1057045816 wait complete. 2023-10-08T09:13:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.623+0000 7f030cd34700 1 -- 172.21.15.49:0/3484885902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 msgr2=0x7f03080ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.623+0000 7f030cd34700 1 --2- 172.21.15.49:0/3484885902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 0x7f03080ffbd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009990 tx=0x7f02f0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.623+0000 7f030cd34700 1 -- 172.21.15.49:0/3484885902 shutdown_connections 2023-10-08T09:13:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.623+0000 7f030cd34700 1 --2- 172.21.15.49:0/3484885902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0308100110 0x7f030810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.623+0000 7f030cd34700 1 --2- 172.21.15.49:0/3484885902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03081056b0 0x7f0308105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.623+0000 7f030cd34700 1 --2- 172.21.15.49:0/3484885902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 0x7f03080ffbd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.658 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.623+0000 7f030cd34700 1 -- 172.21.15.49:0/3484885902 >> 172.21.15.49:0/3484885902 conn(0x7f0308075790 msgr2=0x7f0308075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.624+0000 7f030cd34700 1 -- 172.21.15.49:0/3484885902 shutdown_connections 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.624+0000 7f030cd34700 1 -- 172.21.15.49:0/3484885902 wait complete. 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.624+0000 7f030cd34700 1 Processor -- start 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030cd34700 1 -- start start 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030cd34700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 0x7f0308195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030cd34700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0308100110 0x7f0308195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030cd34700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03081056b0 0x7f030819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030cd34700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030819a910 con 0x7f03080ff750 2023-10-08T09:13:57.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030cd34700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030819aa80 con 0x7f03081056b0 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030cd34700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030819ac40 con 0x7f0308100110 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 0x7f0308195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 0x7f0308195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48584/0 (socket says 172.21.15.49:48584) 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 -- 172.21.15.49:0/3278827482 learned_addr learned my addr 172.21.15.49:0/3278827482 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f0305d9b700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0308100110 0x7f0308195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f0306d9d700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03081056b0 0x7f030819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 -- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0308100110 msgr2=0x7f0308195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0308100110 0x7f0308195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 -- 172.21.15.49:0/3278827482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03081056b0 msgr2=0x7f030819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03081056b0 0x7f030819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 -- 172.21.15.49:0/3278827482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00095f0 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f0306d9d700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03081056b0 0x7f030819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.625+0000 7f030659c700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 0x7f0308195a20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c9b0 tx=0x7f02f800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.626+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8007b00 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.626+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030819af20 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.626+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02f8007c60 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.626+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8018660 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.626+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03081a27c0 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.627+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f030810f910 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.628+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f02f800f450 con 0x7f03080ff750 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.628+0000 7f02ff7fe700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02f40a1510 0x7f02f40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:57.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f0305d9b700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02f40a1510 0x7f02f40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:57.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f02f80cd5a0 con 0x7f03080ff750 2023-10-08T09:13:57.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f02ff7fe700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] conn(0x7f02f40ac530 0x7f02f40ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:57.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 --> [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02f40af0a0 con 0x7f02f40ac530 2023-10-08T09:13:57.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02f80cd9b0 con 0x7f03080ff750 2023-10-08T09:13:57.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f0305d9b700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02f40a1510 0x7f02f40a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02f000b5c0 tx=0x7f02f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:57.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f0306d9d700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] conn(0x7f02f40ac530 0x7f02f40ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:57.662 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.629+0000 7f0306d9d700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] conn(0x7f02f40ac530 0x7f02f40ae9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.630+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== osd.8 v2:172.21.15.184:6816/936308582 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f02f40af0a0 con 0x7f02f40ac530 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.646+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 --> [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0308100df0 con 0x7f02f40ac530 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f02ff7fe700 1 -- 172.21.15.49:0/3278827482 <== osd.8 v2:172.21.15.184:6816/936308582 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0308100df0 con 0x7f02f40ac530 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] conn(0x7f02f40ac530 msgr2=0x7f02f40ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] conn(0x7f02f40ac530 0x7f02f40ae9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02f40a1510 msgr2=0x7f02f40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02f40a1510 0x7f02f40a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02f000b5c0 tx=0x7f02f0005960 comp rx=0 tx=0).stop 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 msgr2=0x7f0308195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 --2- 172.21.15.49:0/3278827482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03080ff750 0x7f0308195a20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c9b0 tx=0x7f02f800ccc0 comp rx=0 tx=0).stop 2023-10-08T09:13:57.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f0306d9d700 1 -- 172.21.15.49:0/3278827482 reap_dead start 2023-10-08T09:13:57.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 shutdown_connections 2023-10-08T09:13:57.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 >> 172.21.15.49:0/3278827482 conn(0x7f0308075790 msgr2=0x7f03080fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:57.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 shutdown_connections 2023-10-08T09:13:57.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:57.647+0000 7f030cd34700 1 -- 172.21.15.49:0/3278827482 wait complete. 2023-10-08T09:13:57.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:57 smithi184 ceph-mon[103675]: pgmap v298: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-10-08T09:13:57.751 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:57 smithi049 ceph-mon[99485]: pgmap v298: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-10-08T09:13:57.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:57 smithi132 ceph-mon[103389]: pgmap v298: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-10-08T09:13:57.953 INFO:teuthology.orchestra.run.smithi049.stdout:210453397544 2023-10-08T09:13:57.953 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-08T09:13:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 -- 172.21.15.49:0/4142512065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ff3f0 msgr2=0x7f0a380ff7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 --2- 172.21.15.49:0/4142512065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ff3f0 0x7f0a380ff7d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009990 tx=0x7f0a34009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 -- 172.21.15.49:0/4142512065 shutdown_connections 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 --2- 172.21.15.49:0/4142512065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a38108fc0 0x7f0a3810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 --2- 172.21.15.49:0/4142512065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ff3f0 0x7f0a380ff7d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 --2- 172.21.15.49:0/4142512065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a380ffda0 0x7f0a381088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 -- 172.21.15.49:0/4142512065 >> 172.21.15.49:0/4142512065 conn(0x7f0a38074a10 msgr2=0x7f0a38074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 -- 172.21.15.49:0/4142512065 shutdown_connections 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 -- 172.21.15.49:0/4142512065 wait complete. 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.049+0000 7f0a40320700 1 Processor -- start 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a40320700 1 -- start start 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a40320700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a380ff3f0 0x7f0a38198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a40320700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ffda0 0x7f0a38199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a40320700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a38108fc0 0x7f0a3819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a40320700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3819dd50 con 0x7f0a380ffda0 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a40320700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3819dec0 con 0x7f0a38108fc0 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a40320700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3819e080 con 0x7f0a380ff3f0 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ffda0 0x7f0a38199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ffda0 0x7f0a38199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48602/0 (socket says 172.21.15.49:48602) 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 -- 172.21.15.49:0/2752859661 learned_addr learned my addr 172.21.15.49:0/2752859661 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3e0bc700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a380ff3f0 0x7f0a38198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3e8bd700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a38108fc0 0x7f0a3819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 -- 172.21.15.49:0/2752859661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a380ff3f0 msgr2=0x7f0a38198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a380ff3f0 0x7f0a38198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 -- 172.21.15.49:0/2752859661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a38108fc0 msgr2=0x7f0a3819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a38108fc0 0x7f0a3819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3d8bb700 1 -- 172.21.15.49:0/2752859661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340095f0 con 0x7f0a380ffda0 2023-10-08T09:13:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.050+0000 7f0a3e8bd700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a38108fc0 0x7f0a3819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.051+0000 7f0a3d8bb700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ffda0 0x7f0a38199390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800d950 tx=0x7f0a2800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.051+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800bb60 con 0x7f0a380ffda0 2023-10-08T09:13:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.051+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3819e360 con 0x7f0a380ffda0 2023-10-08T09:13:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.051+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a28007620 con 0x7f0a380ffda0 2023-10-08T09:13:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.051+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28005560 con 0x7f0a380ffda0 2023-10-08T09:13:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.051+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381a5bc0 con 0x7f0a380ffda0 2023-10-08T09:13:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.052+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a38112c80 con 0x7f0a380ffda0 2023-10-08T09:13:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.053+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0a28004750 con 0x7f0a380ffda0 2023-10-08T09:13:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.054+0000 7f0a2f7fe700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a240a1510 0x7f0a240a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.054+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0a280cd6b0 con 0x7f0a380ffda0 2023-10-08T09:13:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.054+0000 7f0a3e0bc700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a240a1510 0x7f0a240a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.054+0000 7f0a2f7fe700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] conn(0x7f0a240ac530 0x7f0a240ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.054+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 --> [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a240af0a0 con 0x7f0a240ac530 2023-10-08T09:13:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.054+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a280cdac0 con 0x7f0a380ffda0 2023-10-08T09:13:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.055+0000 7f0a3e8bd700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] conn(0x7f0a240ac530 0x7f0a240ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.055+0000 7f0a3e0bc700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a240a1510 0x7f0a240a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a34005850 tx=0x7f0a3401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.055+0000 7f0a3e8bd700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] conn(0x7f0a240ac530 0x7f0a240ae9d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.056+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== osd.6 v2:172.21.15.49:6818/3113806039 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0a240af0a0 con 0x7f0a240ac530 2023-10-08T09:13:58.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.072+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 --> [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a38060e70 con 0x7f0a240ac530 2023-10-08T09:13:58.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.072+0000 7f0a2f7fe700 1 -- 172.21.15.49:0/2752859661 <== osd.6 v2:172.21.15.49:6818/3113806039 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0a38060e70 con 0x7f0a240ac530 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] conn(0x7f0a240ac530 msgr2=0x7f0a240ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] conn(0x7f0a240ac530 0x7f0a240ae9d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a240a1510 msgr2=0x7f0a240a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a240a1510 0x7f0a240a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a34005850 tx=0x7f0a3401a040 comp rx=0 tx=0).stop 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ffda0 msgr2=0x7f0a38199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 --2- 172.21.15.49:0/2752859661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0a380ffda0 0x7f0a38199390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800d950 tx=0x7f0a2800dd10 comp rx=0 tx=0).stop 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a3e8bd700 1 -- 172.21.15.49:0/2752859661 reap_dead start 2023-10-08T09:13:58.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 shutdown_connections 2023-10-08T09:13:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 >> 172.21.15.49:0/2752859661 conn(0x7f0a38074a10 msgr2=0x7f0a38106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 shutdown_connections 2023-10-08T09:13:58.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.073+0000 7f0a40320700 1 -- 172.21.15.49:0/2752859661 wait complete. 2023-10-08T09:13:58.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 -- 172.21.15.49:0/827149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4601067a0 msgr2=0x7fe460106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 --2- 172.21.15.49:0/827149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4601067a0 0x7fe460106b80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009990 tx=0x7fe45c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 -- 172.21.15.49:0/827149 shutdown_connections 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 --2- 172.21.15.49:0/827149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe460068b30 0x7fe46010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 --2- 172.21.15.49:0/827149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4601067a0 0x7fe460106b80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 --2- 172.21.15.49:0/827149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe460068190 0x7fe4600685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 -- 172.21.15.49:0/827149 >> 172.21.15.49:0/827149 conn(0x7fe460075790 msgr2=0x7fe460075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 -- 172.21.15.49:0/827149 shutdown_connections 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.127+0000 7fe468014700 1 -- 172.21.15.49:0/827149 wait complete. 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 Processor -- start 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 -- start start 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe460068190 0x7fe4600ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe460068b30 0x7fe4600ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4601067a0 0x7fe4601002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601a4fb0 con 0x7fe460068190 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601a5120 con 0x7fe460068b30 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe468014700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601a52e0 con 0x7fe4601067a0 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.128+0000 7fe4655af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe460068b30 0x7fe4600ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe465db0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe460068190 0x7fe4600ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe460068b30 0x7fe4600ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:38232/0 (socket says 172.21.15.49:38232) 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 -- 172.21.15.49:0/1306043271 learned_addr learned my addr 172.21.15.49:0/1306043271 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4665b1700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4601067a0 0x7fe4601002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 -- 172.21.15.49:0/1306043271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4601067a0 msgr2=0x7fe4601002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4601067a0 0x7fe4601002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 -- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe460068190 msgr2=0x7fe4600ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe460068190 0x7fe4600ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 -- 172.21.15.49:0/1306043271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c0095f0 con 0x7fe460068b30 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.129+0000 7fe4655af700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe460068b30 0x7fe4600ffd60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe45000e9a0 tx=0x7fe45000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.130+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450009790 con 0x7fe460068b30 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.130+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe450010430 con 0x7fe460068b30 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.130+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450018680 con 0x7fe460068b30 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.130+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4601a55c0 con 0x7fe460068b30 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.130+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4601a5a70 con 0x7fe460068b30 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.131+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4601109c0 con 0x7fe460068b30 2023-10-08T09:13:58.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.132+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe4500105a0 con 0x7fe460068b30 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.133+0000 7fe456ffd700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe44c0a1660 0x7fe44c0a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.134+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe450014070 con 0x7fe460068b30 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.134+0000 7fe465db0700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe44c0a1660 0x7fe44c0a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.134+0000 7fe456ffd700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] conn(0x7fe44c0ac680 0x7fe44c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.134+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 --> [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe44c0af1f0 con 0x7fe44c0ac680 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.134+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4500cf750 con 0x7fe460068b30 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.134+0000 7fe4665b1700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] conn(0x7fe44c0ac680 0x7fe44c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.134+0000 7fe465db0700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe44c0a1660 0x7fe44c0a3b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00b2e0 tx=0x7fe45c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.135+0000 7fe4665b1700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] conn(0x7fe44c0ac680 0x7fe44c0aeb20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.135+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== osd.3 v2:172.21.15.49:6810/3100042133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe44c0af1f0 con 0x7fe44c0ac680 2023-10-08T09:13:58.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.162+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 --> [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe460060ef0 con 0x7fe44c0ac680 2023-10-08T09:13:58.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe456ffd700 1 -- 172.21.15.49:0/1306043271 <== osd.3 v2:172.21.15.49:6810/3100042133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe460060ef0 con 0x7fe44c0ac680 2023-10-08T09:13:58.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] conn(0x7fe44c0ac680 msgr2=0x7fe44c0aeb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe468014700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] conn(0x7fe44c0ac680 0x7fe44c0aeb20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe44c0a1660 msgr2=0x7fe44c0a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe468014700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe44c0a1660 0x7fe44c0a3b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00b2e0 tx=0x7fe45c005fd0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe460068b30 msgr2=0x7fe4600ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe468014700 1 --2- 172.21.15.49:0/1306043271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe460068b30 0x7fe4600ffd60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe45000e9a0 tx=0x7fe45000ecb0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe4665b1700 1 -- 172.21.15.49:0/1306043271 reap_dead start 2023-10-08T09:13:58.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.164+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 shutdown_connections 2023-10-08T09:13:58.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.165+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 >> 172.21.15.49:0/1306043271 conn(0x7fe460075790 msgr2=0x7fe4600fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.165+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 shutdown_connections 2023-10-08T09:13:58.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.165+0000 7fe468014700 1 -- 172.21.15.49:0/1306043271 wait complete. 2023-10-08T09:13:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 -- 172.21.15.49:0/3194664765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 msgr2=0x7f9ea410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 --2- 172.21.15.49:0/3194664765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 0x7f9ea410cdc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009990 tx=0x7f9e94009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 -- 172.21.15.49:0/3194664765 shutdown_connections 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 --2- 172.21.15.49:0/3194664765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea410d300 0x7f9ea410f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 --2- 172.21.15.49:0/3194664765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ea4068190 0x7f9ea4068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 --2- 172.21.15.49:0/3194664765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 0x7f9ea410cdc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 -- 172.21.15.49:0/3194664765 >> 172.21.15.49:0/3194664765 conn(0x7f9ea4075790 msgr2=0x7f9ea4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 -- 172.21.15.49:0/3194664765 shutdown_connections 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 -- 172.21.15.49:0/3194664765 wait complete. 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 Processor -- start 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.580+0000 7f9eac48c700 1 -- start start 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eac48c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4068190 0x7f9ea4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eac48c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 0x7f9ea41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eac48c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ea410d300 0x7f9ea419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eac48c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea419dc60 con 0x7f9ea4068ab0 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eac48c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea419ddd0 con 0x7f9ea410d300 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eac48c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea419df90 con 0x7f9ea4068190 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 0x7f9ea41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 0x7f9ea41992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48634/0 (socket says 172.21.15.49:48634) 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 -- 172.21.15.49:0/3760859275 learned_addr learned my addr 172.21.15.49:0/3760859275 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eaa228700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4068190 0x7f9ea4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eaaa29700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ea410d300 0x7f9ea419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 -- 172.21.15.49:0/3760859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4068190 msgr2=0x7f9ea4198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4068190 0x7f9ea4198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 -- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ea410d300 msgr2=0x7f9ea419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ea410d300 0x7f9ea419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 -- 172.21.15.49:0/3760859275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940095f0 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eaa228700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ea4068190 0x7f9ea4198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eaaa29700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ea410d300 0x7f9ea419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9ea9a27700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 0x7f9ea41992a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400f6a0 tx=0x7f9e9400f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9401d070 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e9400fca0 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.581+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea419e210 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.582+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94017650 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.582+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea41a5ad0 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.582+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ea4112bf0 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.583+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9e94005080 con 0x7f9ea4068ab0 2023-10-08T09:13:58.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.584+0000 7f9e9b7fe700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9e900a1510 0x7f9e900a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.584+0000 7f9eaa228700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9e900a1510 0x7f9e900a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.584+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9e940ce950 con 0x7f9ea4068ab0 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.584+0000 7f9e9b7fe700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] conn(0x7f9e900ac530 0x7f9e900ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.584+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 --> [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e900af0a0 con 0x7f9e900ac530 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.584+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e940ced60 con 0x7f9ea4068ab0 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.585+0000 7f9eaa228700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9e900a1510 0x7f9e900a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005fd0 tx=0x7f9ea0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.585+0000 7f9eaaa29700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] conn(0x7f9e900ac530 0x7f9e900ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.585+0000 7f9eaaa29700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] conn(0x7f9e900ac530 0x7f9e900ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.586+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== osd.1 v2:172.21.15.132:6800/2026160799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9e900af0a0 con 0x7f9e900ac530 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.602+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 --> [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ea4060ef0 con 0x7f9e900ac530 2023-10-08T09:13:58.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.603+0000 7f9e9b7fe700 1 -- 172.21.15.49:0/3760859275 <== osd.1 v2:172.21.15.132:6800/2026160799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9ea4060ef0 con 0x7f9e900ac530 2023-10-08T09:13:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] conn(0x7f9e900ac530 msgr2=0x7f9e900ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] conn(0x7f9e900ac530 0x7f9e900ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9e900a1510 msgr2=0x7f9e900a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9e900a1510 0x7f9e900a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005fd0 tx=0x7f9ea0005e30 comp rx=0 tx=0).stop 2023-10-08T09:13:58.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 msgr2=0x7f9ea41992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 --2- 172.21.15.49:0/3760859275 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9ea4068ab0 0x7f9ea41992a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400f6a0 tx=0x7f9e9400f780 comp rx=0 tx=0).stop 2023-10-08T09:13:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eaaa29700 1 -- 172.21.15.49:0/3760859275 reap_dead start 2023-10-08T09:13:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 shutdown_connections 2023-10-08T09:13:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 >> 172.21.15.49:0/3760859275 conn(0x7f9ea4075790 msgr2=0x7f9ea410e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 shutdown_connections 2023-10-08T09:13:58.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.604+0000 7f9eac48c700 1 -- 172.21.15.49:0/3760859275 wait complete. 2023-10-08T09:13:58.939 INFO:teuthology.orchestra.run.smithi049.stdout:188978561068 2023-10-08T09:13:58.940 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-08T09:13:58.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/479489077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 msgr2=0x7f0b080692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 --2- 172.21.15.49:0/479489077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 0x7f0b080692b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009990 tx=0x7f0af8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/479489077 shutdown_connections 2023-10-08T09:13:58.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 --2- 172.21.15.49:0/479489077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b081067d0 0x7f0b08106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 --2- 172.21.15.49:0/479489077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b080697f0 0x7f0b0810d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 --2- 172.21.15.49:0/479489077 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 0x7f0b080692b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/479489077 >> 172.21.15.49:0/479489077 conn(0x7f0b080766f0 msgr2=0x7f0b08076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/479489077 shutdown_connections 2023-10-08T09:13:58.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.943+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/479489077 wait complete. 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 Processor -- start 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 -- start start 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 0x7f0b080ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b080697f0 0x7f0b080ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b081067d0 0x7f0b08100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08101770 con 0x7f0b08068e50 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0819ad80 con 0x7f0b080697f0 2023-10-08T09:13:58.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0f3fd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0819af40 con 0x7f0b081067d0 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0d199700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 0x7f0b080ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0d199700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 0x7f0b080ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48646/0 (socket says 172.21.15.49:48646) 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.944+0000 7f0b0d199700 1 -- 172.21.15.49:0/2060576876 learned_addr learned my addr 172.21.15.49:0/2060576876 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.945+0000 7f0b0d99a700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b081067d0 0x7f0b08100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.945+0000 7f0b0d99a700 1 -- 172.21.15.49:0/2060576876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b080697f0 msgr2=0x7f0b080ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.945+0000 7f0b0d99a700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0b080697f0 0x7f0b080ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.945+0000 7f0b0d99a700 1 -- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 msgr2=0x7f0b080ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.945+0000 7f0b0d99a700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 0x7f0b080ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.945+0000 7f0b0d99a700 1 -- 172.21.15.49:0/2060576876 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80095f0 con 0x7f0b081067d0 2023-10-08T09:13:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.945+0000 7f0b0d199700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0b08068e50 0x7f0b080ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:13:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.946+0000 7f0b0d99a700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b081067d0 0x7f0b08100230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000ab70 tx=0x7f0b0000ae80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.946+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000f070 con 0x7f0b081067d0 2023-10-08T09:13:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.946+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b00004bb0 con 0x7f0b081067d0 2023-10-08T09:13:58.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.946+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b00005260 con 0x7f0b081067d0 2023-10-08T09:13:58.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.946+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0819b220 con 0x7f0b081067d0 2023-10-08T09:13:58.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.946+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0819b6d0 con 0x7f0b081067d0 2023-10-08T09:13:58.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.947+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b081109e0 con 0x7f0b081067d0 2023-10-08T09:13:58.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.950+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0b00004d20 con 0x7f0b081067d0 2023-10-08T09:13:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0afe7fc700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0af40a1510 0x7f0af40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0b0d199700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0af40a1510 0x7f0af40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0b000ccb60 con 0x7f0b081067d0 2023-10-08T09:13:58.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0afe7fc700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] conn(0x7f0af40ac530 0x7f0af40ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 --> [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0af40af0a0 con 0x7f0af40ac530 2023-10-08T09:13:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b000cced0 con 0x7f0b081067d0 2023-10-08T09:13:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0b0c998700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] conn(0x7f0af40ac530 0x7f0af40ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.951+0000 7f0b0d199700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0af40a1510 0x7f0af40a39d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b0400a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.952+0000 7f0b0c998700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] conn(0x7f0af40ac530 0x7f0af40ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:58.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.952+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== osd.0 v2:172.21.15.49:6802/3366428162 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0af40af0a0 con 0x7f0af40ac530 2023-10-08T09:13:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.970+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 --> [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b08101f10 con 0x7f0af40ac530 2023-10-08T09:13:58.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0afe7fc700 1 -- 172.21.15.49:0/2060576876 <== osd.0 v2:172.21.15.49:6802/3366428162 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0b08101f10 con 0x7f0af40ac530 2023-10-08T09:13:58.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] conn(0x7f0af40ac530 msgr2=0x7f0af40ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] conn(0x7f0af40ac530 0x7f0af40ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0af40a1510 msgr2=0x7f0af40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0af40a1510 0x7f0af40a39d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005950 tx=0x7f0b0400a400 comp rx=0 tx=0).stop 2023-10-08T09:13:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b081067d0 msgr2=0x7f0b08100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 --2- 172.21.15.49:0/2060576876 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b081067d0 0x7f0b08100230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b0000ab70 tx=0x7f0b0000ae80 comp rx=0 tx=0).stop 2023-10-08T09:13:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0d99a700 1 -- 172.21.15.49:0/2060576876 reap_dead start 2023-10-08T09:13:58.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 shutdown_connections 2023-10-08T09:13:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.971+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 >> 172.21.15.49:0/2060576876 conn(0x7f0b080766f0 msgr2=0x7f0b080fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.972+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 shutdown_connections 2023-10-08T09:13:58.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:58.972+0000 7f0b0f3fd700 1 -- 172.21.15.49:0/2060576876 wait complete. 2023-10-08T09:13:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:13:59 smithi184 ceph-mon[103675]: pgmap v299: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-08T09:13:59.762 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:13:59 smithi049 ceph-mon[99485]: pgmap v299: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-08T09:13:59.763 INFO:teuthology.orchestra.run.smithi049.stdout:34359738438 2023-10-08T09:13:59.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-08T09:13:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:13:59 smithi132 ceph-mon[103389]: pgmap v299: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-08T09:13:59.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.726+0000 7f21fc2ce700 1 -- 172.21.15.49:0/3157185412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 msgr2=0x7f21f4105cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.726+0000 7f21fc2ce700 1 --2- 172.21.15.49:0/3157185412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 0x7f21f4105cb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009990 tx=0x7f21f0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- 172.21.15.49:0/3157185412 shutdown_connections 2023-10-08T09:13:59.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 --2- 172.21.15.49:0/3157185412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ffb30 0x7f21f410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 --2- 172.21.15.49:0/3157185412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 0x7f21f4105cb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 --2- 172.21.15.49:0/3157185412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21f40ff190 0x7f21f40ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- 172.21.15.49:0/3157185412 >> 172.21.15.49:0/3157185412 conn(0x7f21f4074a10 msgr2=0x7f21f4074e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:59.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- 172.21.15.49:0/3157185412 shutdown_connections 2023-10-08T09:13:59.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- 172.21.15.49:0/3157185412 wait complete. 2023-10-08T09:13:59.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 Processor -- start 2023-10-08T09:13:59.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- start start 2023-10-08T09:13:59.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ff190 0x7f21f419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21f40ffb30 0x7f21f419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 0x7f21f4196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41970a0 con 0x7f21f41058d0 2023-10-08T09:13:59.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4197210 con 0x7f21f40ffb30 2023-10-08T09:13:59.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.727+0000 7f21fc2ce700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41973d0 con 0x7f21f40ff190 2023-10-08T09:13:59.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 0x7f21f4196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 0x7f21f4196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48666/0 (socket says 172.21.15.49:48666) 2023-10-08T09:13:59.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 -- 172.21.15.49:0/1949196833 learned_addr learned my addr 172.21.15.49:0/1949196833 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:59.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa06a700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ff190 0x7f21f419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21f9869700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21f40ffb30 0x7f21f419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 -- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ff190 msgr2=0x7f21f419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21f40ff190 0x7f21f419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 -- 172.21.15.49:0/1949196833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21f40ffb30 msgr2=0x7f21f419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21f40ffb30 0x7f21f419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 -- 172.21.15.49:0/1949196833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f00095f0 con 0x7f21f41058d0 2023-10-08T09:13:59.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21f9869700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21f40ffb30 0x7f21f419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:13:59.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fa86b700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 0x7f21f4196a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0094f0 tx=0x7f21ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:59.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00ddd0 con 0x7f21f41058d0 2023-10-08T09:13:59.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21ec00f040 con 0x7f21f41058d0 2023-10-08T09:13:59.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f41976b0 con 0x7f21f41058d0 2023-10-08T09:13:59.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec015400 con 0x7f21f41058d0 2023-10-08T09:13:59.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.728+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f41a38f0 con 0x7f21f41058d0 2023-10-08T09:13:59.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.729+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21f41109c0 con 0x7f21f41058d0 2023-10-08T09:13:59.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.731+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f21ec004c40 con 0x7f21f41058d0 2023-10-08T09:13:59.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.731+0000 7f21eb7fe700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f21e00a1590 0x7f21e00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.731+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f21ec0cd5f0 con 0x7f21f41058d0 2023-10-08T09:13:59.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.731+0000 7f21fa06a700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f21e00a1590 0x7f21e00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.731+0000 7f21eb7fe700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] conn(0x7f21e00ac5b0 0x7f21e00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.815 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.731+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 --> [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21e00af120 con 0x7f21e00ac5b0 2023-10-08T09:13:59.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.731+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ec0cda00 con 0x7f21f41058d0 2023-10-08T09:13:59.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.732+0000 7f21fa06a700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f21e00a1590 0x7f21e00a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21f0006010 tx=0x7f21f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:59.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.732+0000 7f21f9869700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] conn(0x7f21e00ac5b0 0x7f21e00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.732+0000 7f21f9869700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] conn(0x7f21e00ac5b0 0x7f21e00aea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:59.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.733+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== osd.2 v2:172.21.15.184:6800/2213406776 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f21e00af120 con 0x7f21e00ac5b0 2023-10-08T09:13:59.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.754+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 --> [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f21f4060e70 con 0x7f21e00ac5b0 2023-10-08T09:13:59.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21eb7fe700 1 -- 172.21.15.49:0/1949196833 <== osd.2 v2:172.21.15.184:6800/2213406776 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f21f4060e70 con 0x7f21e00ac5b0 2023-10-08T09:13:59.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] conn(0x7f21e00ac5b0 msgr2=0x7f21e00aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] conn(0x7f21e00ac5b0 0x7f21e00aea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f21e00a1590 msgr2=0x7f21e00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f21e00a1590 0x7f21e00a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21f0006010 tx=0x7f21f0005960 comp rx=0 tx=0).stop 2023-10-08T09:13:59.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 msgr2=0x7f21f4196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 --2- 172.21.15.49:0/1949196833 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f21f41058d0 0x7f21f4196a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0094f0 tx=0x7f21ec00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fa86b700 1 -- 172.21.15.49:0/1949196833 reap_dead start 2023-10-08T09:13:59.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 shutdown_connections 2023-10-08T09:13:59.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 >> 172.21.15.49:0/1949196833 conn(0x7f21f4074a10 msgr2=0x7f21f40fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:59.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 shutdown_connections 2023-10-08T09:13:59.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.755+0000 7f21fc2ce700 1 -- 172.21.15.49:0/1949196833 wait complete. 2023-10-08T09:13:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 -- 172.21.15.49:0/3270096205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af01058d0 msgr2=0x7f9af0105cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 --2- 172.21.15.49:0/3270096205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af01058d0 0x7f9af0105cb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00098f0 tx=0x7f9ae0009c00 comp rx=0 tx=0).stop 2023-10-08T09:13:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 -- 172.21.15.49:0/3270096205 shutdown_connections 2023-10-08T09:13:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 --2- 172.21.15.49:0/3270096205 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af00ffb30 0x7f9af010d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 --2- 172.21.15.49:0/3270096205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af01058d0 0x7f9af0105cb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 --2- 172.21.15.49:0/3270096205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9af00ff190 0x7f9af00ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.974 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 -- 172.21.15.49:0/3270096205 >> 172.21.15.49:0/3270096205 conn(0x7f9af00749b0 msgr2=0x7f9af0074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 -- 172.21.15.49:0/3270096205 shutdown_connections 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.968+0000 7f9af5521700 1 -- 172.21.15.49:0/3270096205 wait complete. 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 Processor -- start 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 -- start start 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af00ff190 0x7f9af0100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af00ffb30 0x7f9af01008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9af01058d0 0x7f9af0100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af01a4fd0 con 0x7f9af00ffb30 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af01a5140 con 0x7f9af00ff190 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.969+0000 7f9af5521700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af01a5300 con 0x7f9af01058d0 2023-10-08T09:13:59.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aee7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af00ffb30 0x7f9af01008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aee7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af00ffb30 0x7f9af01008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48694/0 (socket says 172.21.15.49:48694) 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aee7fc700 1 -- 172.21.15.49:0/1117800814 learned_addr learned my addr 172.21.15.49:0/1117800814 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aef7fe700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9af01058d0 0x7f9af0100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aeeffd700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af00ff190 0x7f9af0100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aeeffd700 1 -- 172.21.15.49:0/1117800814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9af01058d0 msgr2=0x7f9af0100df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aeeffd700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9af01058d0 0x7f9af0100df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aeeffd700 1 -- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af00ffb30 msgr2=0x7f9af01008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aeeffd700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af00ffb30 0x7f9af01008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aeeffd700 1 -- 172.21.15.49:0/1117800814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009d00 con 0x7f9af00ff190 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aee7fc700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9af00ffb30 0x7f9af01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aef7fe700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9af01058d0 0x7f9af0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:13:59.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.970+0000 7f9aeeffd700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af00ff190 0x7f9af0100370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009230 tx=0x7f9ae000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.971+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae001c070 con 0x7f9af00ff190 2023-10-08T09:13:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.971+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae0009520 con 0x7f9af00ff190 2023-10-08T09:13:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.971+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ae000ba20 con 0x7f9af00ff190 2023-10-08T09:13:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.971+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00038e0 con 0x7f9af00ff190 2023-10-08T09:13:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.971+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af01a5860 con 0x7f9af00ff190 2023-10-08T09:13:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.972+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9af01109c0 con 0x7f9af00ff190 2023-10-08T09:13:59.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.973+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9ae0020070 con 0x7f9af00ff190 2023-10-08T09:13:59.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.974+0000 7f9ad7fff700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ad80a1590 0x7f9ad80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.975+0000 7f9aee7fc700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ad80a1590 0x7f9ad80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.975+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9ae00cdd60 con 0x7f9af00ff190 2023-10-08T09:13:59.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.975+0000 7f9ad7fff700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] conn(0x7f9ad80ac5b0 0x7f9ad80aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:13:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.975+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 --> [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ad80af120 con 0x7f9ad80ac5b0 2023-10-08T09:13:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.975+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae00ce170 con 0x7f9af00ff190 2023-10-08T09:13:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.975+0000 7f9aee7fc700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ad80a1590 0x7f9ad80a3a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9af0101a50 tx=0x7f9ae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.975+0000 7f9aef7fe700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] conn(0x7f9ad80ac5b0 0x7f9ad80aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:13:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.976+0000 7f9aef7fe700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] conn(0x7f9ad80ac5b0 0x7f9ad80aea50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:13:59.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.977+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== osd.10 v2:172.21.15.132:6824/3556600534 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9ad80af120 con 0x7f9ad80ac5b0 2023-10-08T09:13:59.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.994+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 --> [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9af0068920 con 0x7f9ad80ac5b0 2023-10-08T09:13:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.995+0000 7f9ad7fff700 1 -- 172.21.15.49:0/1117800814 <== osd.10 v2:172.21.15.132:6824/3556600534 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9af0068920 con 0x7f9ad80ac5b0 2023-10-08T09:13:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.995+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] conn(0x7f9ad80ac5b0 msgr2=0x7f9ad80aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.995+0000 7f9af5521700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] conn(0x7f9ad80ac5b0 0x7f9ad80aea50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:13:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ad80a1590 msgr2=0x7f9ad80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ad80a1590 0x7f9ad80a3a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9af0101a50 tx=0x7f9ae4009260 comp rx=0 tx=0).stop 2023-10-08T09:13:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af00ff190 msgr2=0x7f9af0100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:13:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 --2- 172.21.15.49:0/1117800814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9af00ff190 0x7f9af0100370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009230 tx=0x7f9ae000fa00 comp rx=0 tx=0).stop 2023-10-08T09:13:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9aef7fe700 1 -- 172.21.15.49:0/1117800814 reap_dead start 2023-10-08T09:13:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 shutdown_connections 2023-10-08T09:13:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 >> 172.21.15.49:0/1117800814 conn(0x7f9af00749b0 msgr2=0x7f9af00fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:13:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 shutdown_connections 2023-10-08T09:13:59.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.996+0000 7f9af5521700 1 -- 172.21.15.49:0/1117800814 wait complete. 2023-10-08T09:14:00.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.864+0000 7ff21fa67700 1 -- 172.21.15.49:0/2216426766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100110 msgr2=0x7ff21810c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.864+0000 7ff21fa67700 1 --2- 172.21.15.49:0/2216426766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100110 0x7ff21810c580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff21000b0a0 tx=0x7ff21000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.864+0000 7ff21fa67700 1 -- 172.21.15.49:0/2216426766 shutdown_connections 2023-10-08T09:14:00.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.864+0000 7ff21fa67700 1 --2- 172.21.15.49:0/2216426766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2181056b0 0x7ff218105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.864+0000 7ff21fa67700 1 --2- 172.21.15.49:0/2216426766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff218100110 0x7ff21810c580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.864+0000 7ff21fa67700 1 --2- 172.21.15.49:0/2216426766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2180ff750 0x7ff2180ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.864+0000 7ff21fa67700 1 -- 172.21.15.49:0/2216426766 >> 172.21.15.49:0/2216426766 conn(0x7ff218075790 msgr2=0x7ff218075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.865+0000 7ff21fa67700 1 -- 172.21.15.49:0/2216426766 shutdown_connections 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.865+0000 7ff21fa67700 1 -- 172.21.15.49:0/2216426766 wait complete. 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.865+0000 7ff21fa67700 1 Processor -- start 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.865+0000 7ff21fa67700 1 -- start start 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21fa67700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2180ff750 0x7ff21819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21fa67700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218100110 0x7ff21819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21fa67700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2181056b0 0x7ff218196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21fa67700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218197230 con 0x7ff2181056b0 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21fa67700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181973a0 con 0x7ff218100110 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21fa67700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff218197560 con 0x7ff2180ff750 2023-10-08T09:14:00.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21e004700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2181056b0 0x7ff218196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21e004700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2181056b0 0x7ff218196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48680/0 (socket says 172.21.15.49:48680) 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21e004700 1 -- 172.21.15.49:0/191281298 learned_addr learned my addr 172.21.15.49:0/191281298 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21d803700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2180ff750 0x7ff21819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21d803700 1 -- 172.21.15.49:0/191281298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218100110 msgr2=0x7ff21819cf80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21d803700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff218100110 0x7ff21819cf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21d803700 1 -- 172.21.15.49:0/191281298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2181056b0 msgr2=0x7ff218196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21d803700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2181056b0 0x7ff218196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.866+0000 7ff21d803700 1 -- 172.21.15.49:0/191281298 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff210009d00 con 0x7ff2180ff750 2023-10-08T09:14:00.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.867+0000 7ff21d803700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2180ff750 0x7ff21819ca40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff214009c50 tx=0x7ff214009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:00.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.867+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214015070 con 0x7ff2180ff750 2023-10-08T09:14:00.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.867+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff218197840 con 0x7ff2180ff750 2023-10-08T09:14:00.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.867+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff218101380 con 0x7ff2180ff750 2023-10-08T09:14:00.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.867+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff214019410 con 0x7ff2180ff750 2023-10-08T09:14:00.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.867+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214022610 con 0x7ff2180ff750 2023-10-08T09:14:00.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.868+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1fc000fc0 con 0x7ff2180ff750 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.869+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff21400bb00 con 0x7ff2180ff750 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.870+0000 7ff20effd700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2040a1590 0x7ff2040a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.870+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2140d0a00 con 0x7ff2180ff750 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.870+0000 7ff20effd700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] conn(0x7ff2040ac5b0 0x7ff2040aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.870+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 --> [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2040af120 con 0x7ff2040ac5b0 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.870+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2140d0e10 con 0x7ff2180ff750 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.871+0000 7ff21e004700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] conn(0x7ff2040ac5b0 0x7ff2040aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.871+0000 7ff21e004700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] conn(0x7ff2040ac5b0 0x7ff2040aea50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:00.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.872+0000 7ff21d002700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2040a1590 0x7ff2040a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.872+0000 7ff21d002700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2040a1590 0x7ff2040a3a50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff218198220 tx=0x7ff208009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:00.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.873+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== osd.11 v2:172.21.15.184:6824/613863047 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7ff2040af120 con 0x7ff2040ac5b0 2023-10-08T09:14:00.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.891+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 --> [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff1fc002d40 con 0x7ff2040ac5b0 2023-10-08T09:14:00.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.891+0000 7ff20effd700 1 -- 172.21.15.49:0/191281298 <== osd.11 v2:172.21.15.184:6824/613863047 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff1fc002d40 con 0x7ff2040ac5b0 2023-10-08T09:14:00.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] conn(0x7ff2040ac5b0 msgr2=0x7ff2040aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] conn(0x7ff2040ac5b0 0x7ff2040aea50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2040a1590 msgr2=0x7ff2040a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2040a1590 0x7ff2040a3a50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff218198220 tx=0x7ff208009260 comp rx=0 tx=0).stop 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2180ff750 msgr2=0x7ff21819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 --2- 172.21.15.49:0/191281298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2180ff750 0x7ff21819ca40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff214009c50 tx=0x7ff214009f60 comp rx=0 tx=0).stop 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21e004700 1 -- 172.21.15.49:0/191281298 reap_dead start 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 shutdown_connections 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 >> 172.21.15.49:0/191281298 conn(0x7ff218075790 msgr2=0x7ff218068530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 shutdown_connections 2023-10-08T09:14:00.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:13:59.892+0000 7ff21fa67700 1 -- 172.21.15.49:0/191281298 wait complete. 2023-10-08T09:14:00.394 INFO:teuthology.orchestra.run.smithi049.stdout:77309411388 2023-10-08T09:14:00.394 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 -- 172.21.15.49:0/3427990063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40ff950 msgr2=0x7f0af4105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 --2- 172.21.15.49:0/3427990063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40ff950 0x7f0af4105490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009990 tx=0x7f0ae8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 -- 172.21.15.49:0/3427990063 shutdown_connections 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 --2- 172.21.15.49:0/3427990063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4105b70 0x7f0af410d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 --2- 172.21.15.49:0/3427990063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0af40fefa0 0x7f0af40ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 --2- 172.21.15.49:0/3427990063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40ff950 0x7f0af4105490 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 -- 172.21.15.49:0/3427990063 >> 172.21.15.49:0/3427990063 conn(0x7f0af4075cc0 msgr2=0x7f0af40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 -- 172.21.15.49:0/3427990063 shutdown_connections 2023-10-08T09:14:00.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.762+0000 7f0afa02b700 1 -- 172.21.15.49:0/3427990063 wait complete. 2023-10-08T09:14:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 Processor -- start 2023-10-08T09:14:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 -- start start 2023-10-08T09:14:00.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40fefa0 0x7f0af419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0af40ff950 0x7f0af419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4105b70 0x7f0af4196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41972c0 con 0x7f0af40fefa0 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4197430 con 0x7f0af40ff950 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0afa02b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41975f0 con 0x7f0af4105b70 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40fefa0 0x7f0af419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40fefa0 0x7f0af419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48708/0 (socket says 172.21.15.49:48708) 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.763+0000 7f0af37fe700 1 -- 172.21.15.49:0/4224580059 learned_addr learned my addr 172.21.15.49:0/4224580059 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af3fff700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4105b70 0x7f0af4196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af2ffd700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0af40ff950 0x7f0af419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af37fe700 1 -- 172.21.15.49:0/4224580059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4105b70 msgr2=0x7f0af4196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af37fe700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0af4105b70 0x7f0af4196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af37fe700 1 -- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0af40ff950 msgr2=0x7f0af419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af37fe700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0af40ff950 0x7f0af419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af37fe700 1 -- 172.21.15.49:0/4224580059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80095f0 con 0x7f0af40fefa0 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af2ffd700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0af40ff950 0x7f0af419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af37fe700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40fefa0 0x7f0af419ca60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400d950 tx=0x7f0ae400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4009940 con 0x7f0af40fefa0 2023-10-08T09:14:00.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af41978d0 con 0x7f0af40fefa0 2023-10-08T09:14:00.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae4010460 con 0x7f0af40fefa0 2023-10-08T09:14:00.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae400f620 con 0x7f0af40fefa0 2023-10-08T09:14:00.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.764+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af41a39e0 con 0x7f0af40fefa0 2023-10-08T09:14:00.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.765+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0af41109b0 con 0x7f0af40fefa0 2023-10-08T09:14:00.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.766+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0ae40108d0 con 0x7f0af40fefa0 2023-10-08T09:14:00.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.767+0000 7f0af0ff9700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0adc0a1510 0x7f0adc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.767+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0ae40cd520 con 0x7f0af40fefa0 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.767+0000 7f0af2ffd700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0adc0a1510 0x7f0adc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.767+0000 7f0af0ff9700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] conn(0x7f0adc0ac530 0x7f0adc0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.767+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 --> [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0adc0af0a0 con 0x7f0adc0ac530 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.767+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae40cd930 con 0x7f0af40fefa0 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.768+0000 7f0af2ffd700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0adc0a1510 0x7f0adc0a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0af4198280 tx=0x7f0ae801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.768+0000 7f0af3fff700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] conn(0x7f0adc0ac530 0x7f0adc0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.768+0000 7f0af3fff700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] conn(0x7f0adc0ac530 0x7f0adc0ae9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:00.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.769+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== osd.4 v2:172.21.15.132:6808/2969628103 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0adc0af0a0 con 0x7f0adc0ac530 2023-10-08T09:14:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.785+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 --> [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0af4060e70 con 0x7f0adc0ac530 2023-10-08T09:14:00.786 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.786+0000 7f0af0ff9700 1 -- 172.21.15.49:0/4224580059 <== osd.4 v2:172.21.15.132:6808/2969628103 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0af4060e70 con 0x7f0adc0ac530 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.786+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] conn(0x7f0adc0ac530 msgr2=0x7f0adc0ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.786+0000 7f0afa02b700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] conn(0x7f0adc0ac530 0x7f0adc0ae9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.786+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0adc0a1510 msgr2=0x7f0adc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.786+0000 7f0afa02b700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0adc0a1510 0x7f0adc0a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0af4198280 tx=0x7f0ae801a040 comp rx=0 tx=0).stop 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.786+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40fefa0 msgr2=0x7f0af419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.786+0000 7f0afa02b700 1 --2- 172.21.15.49:0/4224580059 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0af40fefa0 0x7f0af419ca60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400d950 tx=0x7f0ae400dc60 comp rx=0 tx=0).stop 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.787+0000 7f0af3fff700 1 -- 172.21.15.49:0/4224580059 reap_dead start 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.787+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 shutdown_connections 2023-10-08T09:14:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.787+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 >> 172.21.15.49:0/4224580059 conn(0x7f0af4075cc0 msgr2=0x7f0af40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.787+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 shutdown_connections 2023-10-08T09:14:00.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:00.787+0000 7f0afa02b700 1 -- 172.21.15.49:0/4224580059 wait complete. 2023-10-08T09:14:00.935 INFO:teuthology.orchestra.run.smithi049.stdout:240518168612 2023-10-08T09:14:00.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-08T09:14:01.253 INFO:teuthology.orchestra.run.smithi049.stdout:107374182457 2023-10-08T09:14:01.254 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-08T09:14:01.454 INFO:teuthology.orchestra.run.smithi049.stdout:55834574913 2023-10-08T09:14:01.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-08T09:14:01.458 INFO:teuthology.orchestra.run.smithi049.stdout:158913790000 2023-10-08T09:14:01.458 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-08T09:14:01.502 INFO:teuthology.orchestra.run.smithi049.stdout:296352743451 2023-10-08T09:14:01.503 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-08T09:14:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:01 smithi184 ceph-mon[103675]: pgmap v300: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-08T09:14:01.795 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:01 smithi049 ceph-mon[99485]: pgmap v300: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-08T09:14:01.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:01 smithi132 ceph-mon[103389]: pgmap v300: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2023-10-08T09:14:01.998 INFO:teuthology.orchestra.run.smithi049.stdout:326417514519 2023-10-08T09:14:01.998 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-08T09:14:02.017 INFO:teuthology.orchestra.run.smithi049.stdout:137438953524 2023-10-08T09:14:02.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-08T09:14:02.147 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:02.547 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:02.584 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:03.533 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:03 smithi049 ceph-mon[99485]: pgmap v301: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2023-10-08T09:14:03.533 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:03 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:14:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:03 smithi184 ceph-mon[103675]: pgmap v301: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2023-10-08T09:14:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:03 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:14:03.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:03 smithi132 ceph-mon[103389]: pgmap v301: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2023-10-08T09:14:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:03 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:14:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 -- 172.21.15.49:0/1964758095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78105700 msgr2=0x7efd78105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/1964758095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78105700 0x7efd78105ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efd680099c0 tx=0x7efd68009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 -- 172.21.15.49:0/1964758095 shutdown_connections 2023-10-08T09:14:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/1964758095 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd78100110 0x7efd7810c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/1964758095 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78105700 0x7efd78105ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/1964758095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd780ff750 0x7efd780ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 -- 172.21.15.49:0/1964758095 >> 172.21.15.49:0/1964758095 conn(0x7efd78075790 msgr2=0x7efd78075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:05.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 -- 172.21.15.49:0/1964758095 shutdown_connections 2023-10-08T09:14:05.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.326+0000 7efd7e9a9700 1 -- 172.21.15.49:0/1964758095 wait complete. 2023-10-08T09:14:05.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.327+0000 7efd7e9a9700 1 Processor -- start 2023-10-08T09:14:05.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.327+0000 7efd7e9a9700 1 -- start start 2023-10-08T09:14:05.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd780ff750 0x7efd78197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78100110 0x7efd78197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd78105700 0x7efd781a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd78198820 con 0x7efd78100110 2023-10-08T09:14:05.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd781a2fe0 con 0x7efd78105700 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd781a3120 con 0x7efd780ff750 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78100110 0x7efd78197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78100110 0x7efd78197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48726/0 (socket says 172.21.15.49:48726) 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 -- 172.21.15.49:0/557793642 learned_addr learned my addr 172.21.15.49:0/557793642 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd77fff700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd780ff750 0x7efd78197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7cf46700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd78105700 0x7efd781a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 -- 172.21.15.49:0/557793642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd780ff750 msgr2=0x7efd78197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd780ff750 0x7efd78197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 -- 172.21.15.49:0/557793642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd78105700 msgr2=0x7efd781a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd78105700 0x7efd781a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 -- 172.21.15.49:0/557793642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd680095f0 con 0x7efd78100110 2023-10-08T09:14:05.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7cf46700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd78105700 0x7efd781a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:05.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd77fff700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd780ff750 0x7efd78197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:05.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd777fe700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78100110 0x7efd78197f60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd6000d980 tx=0x7efd6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:05.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd757fa700 1 -- 172.21.15.49:0/557793642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60004220 con 0x7efd78100110 2023-10-08T09:14:05.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd781a33a0 con 0x7efd78100110 2023-10-08T09:14:05.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd757fa700 1 -- 172.21.15.49:0/557793642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd60007640 con 0x7efd78100110 2023-10-08T09:14:05.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd757fa700 1 -- 172.21.15.49:0/557793642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60005610 con 0x7efd78100110 2023-10-08T09:14:05.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.328+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd781a38f0 con 0x7efd78100110 2023-10-08T09:14:05.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.330+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd580052f0 con 0x7efd78100110 2023-10-08T09:14:05.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.330+0000 7efd757fa700 1 -- 172.21.15.49:0/557793642 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7efd6000b750 con 0x7efd78100110 2023-10-08T09:14:05.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.331+0000 7efd757fa700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd640a1590 0x7efd640a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.331+0000 7efd77fff700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd640a1590 0x7efd640a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.331+0000 7efd757fa700 1 -- 172.21.15.49:0/557793642 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efd600cce70 con 0x7efd78100110 2023-10-08T09:14:05.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.332+0000 7efd77fff700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd640a1590 0x7efd640a3a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd68009fd0 tx=0x7efd68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:05.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.336+0000 7efd757fa700 1 -- 172.21.15.49:0/557793642 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efd60096060 con 0x7efd78100110 2023-10-08T09:14:05.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.547+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7efd58005870 con 0x7efd78100110 2023-10-08T09:14:05.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.547+0000 7efd757fa700 1 -- 172.21.15.49:0/557793642 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd60016020 con 0x7efd78100110 2023-10-08T09:14:05.736 INFO:teuthology.orchestra.run.smithi049.stdout:210453397545 2023-10-08T09:14:05.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.549+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd640a1590 msgr2=0x7efd640a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.549+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd640a1590 0x7efd640a3a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efd68009fd0 tx=0x7efd68005fb0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.549+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78100110 msgr2=0x7efd78197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.549+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78100110 0x7efd78197f60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd6000d980 tx=0x7efd6000dc90 comp rx=0 tx=0).stop 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 shutdown_connections 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd640a1590 0x7efd640a3a50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd780ff750 0x7efd78197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd78105700 0x7efd781a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 --2- 172.21.15.49:0/557793642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd78100110 0x7efd78197f60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 >> 172.21.15.49:0/557793642 conn(0x7efd78075790 msgr2=0x7efd780fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 shutdown_connections 2023-10-08T09:14:05.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.550+0000 7efd7e9a9700 1 -- 172.21.15.49:0/557793642 wait complete. 2023-10-08T09:14:05.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:05 smithi184 ceph-mon[103675]: pgmap v302: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-08T09:14:05.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:05 smithi132 ceph-mon[103389]: pgmap v302: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-08T09:14:05.916 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:05 smithi049 ceph-mon[99485]: pgmap v302: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.840+0000 7f4bd1c63700 1 -- 172.21.15.49:0/2858282526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 msgr2=0x7f4bcc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.840+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/2858282526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc105a90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400b0a0 tx=0x7f4bc400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.840+0000 7f4bd1c63700 1 -- 172.21.15.49:0/2858282526 shutdown_connections 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.840+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/2858282526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc100110 0x7f4bcc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.840+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/2858282526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc105a90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.840+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/2858282526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bcc0ff750 0x7f4bcc0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.840+0000 7f4bd1c63700 1 -- 172.21.15.49:0/2858282526 >> 172.21.15.49:0/2858282526 conn(0x7f4bcc075790 msgr2=0x7f4bcc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.841+0000 7f4bd1c63700 1 -- 172.21.15.49:0/2858282526 shutdown_connections 2023-10-08T09:14:05.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.841+0000 7f4bd1c63700 1 -- 172.21.15.49:0/2858282526 wait complete. 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.841+0000 7f4bd1c63700 1 Processor -- start 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.841+0000 7f4bd1c63700 1 -- start start 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bd1c63700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bcc0ff750 0x7f4bcc195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bd1c63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc100110 0x7f4bcc196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bd1c63700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bd1c63700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc19ab60 con 0x7f4bcc1056b0 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bd1c63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc19acd0 con 0x7f4bcc100110 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bd1c63700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc19ae90 con 0x7f4bcc0ff750 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48740/0 (socket says 172.21.15.49:48740) 2023-10-08T09:14:05.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 -- 172.21.15.49:0/1121405298 learned_addr learned my addr 172.21.15.49:0/1121405298 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcaffd700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc100110 0x7f4bcc196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcb7fe700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bcc0ff750 0x7f4bcc195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 -- 172.21.15.49:0/1121405298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bcc0ff750 msgr2=0x7f4bcc195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bcc0ff750 0x7f4bcc195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 -- 172.21.15.49:0/1121405298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc100110 msgr2=0x7f4bcc196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc100110 0x7f4bcc196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 -- 172.21.15.49:0/1121405298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4009d00 con 0x7f4bcc1056b0 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcaffd700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc100110 0x7f4bcc196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcb7fe700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bcc0ff750 0x7f4bcc195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bcbfff700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc19a550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00e940 tx=0x7f4bbc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.842+0000 7f4bc8ff9700 1 -- 172.21.15.49:0/1121405298 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0097b0 con 0x7f4bcc1056b0 2023-10-08T09:14:05.967 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.843+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc19b170 con 0x7f4bcc1056b0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.843+0000 7f4bc8ff9700 1 -- 172.21.15.49:0/1121405298 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4bbc005020 con 0x7f4bcc1056b0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.843+0000 7f4bc8ff9700 1 -- 172.21.15.49:0/1121405298 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00cc40 con 0x7f4bcc1056b0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.843+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc1a29f0 con 0x7f4bcc1056b0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.844+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc107cb0 con 0x7f4bcc1056b0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.845+0000 7f4bc8ff9700 1 -- 172.21.15.49:0/1121405298 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4bbc005400 con 0x7f4bcc1056b0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.846+0000 7f4bc8ff9700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bb40a1590 0x7f4bb40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.846+0000 7f4bcb7fe700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bb40a1590 0x7f4bb40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.846+0000 7f4bc8ff9700 1 -- 172.21.15.49:0/1121405298 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4bbc014070 con 0x7f4bcc1056b0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.847+0000 7f4bcb7fe700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bb40a1590 0x7f4bb40a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009fd0 tx=0x7f4bc4006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:05.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:05.853+0000 7f4bc8ff9700 1 -- 172.21.15.49:0/1121405298 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4bbc096420 con 0x7f4bcc1056b0 2023-10-08T09:14:06.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.062+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f4bcc196f70 con 0x7f4bcc1056b0 2023-10-08T09:14:06.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.062+0000 7f4bc8ff9700 1 -- 172.21.15.49:0/1121405298 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bbc005190 con 0x7f4bcc1056b0 2023-10-08T09:14:06.064 INFO:teuthology.orchestra.run.smithi049.stdout:274877906976 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bb40a1590 msgr2=0x7f4bb40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bb40a1590 0x7f4bb40a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009fd0 tx=0x7f4bc4006210 comp rx=0 tx=0).stop 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 msgr2=0x7f4bcc19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc19a550 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00e940 tx=0x7f4bbc00ec50 comp rx=0 tx=0).stop 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 shutdown_connections 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4bb40a1590 0x7f4bb40a3a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4bcc0ff750 0x7f4bcc195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:06.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bcc1056b0 0x7f4bcc19a550 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 --2- 172.21.15.49:0/1121405298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bcc100110 0x7f4bcc196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 >> 172.21.15.49:0/1121405298 conn(0x7f4bcc075790 msgr2=0x7f4bcc0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 shutdown_connections 2023-10-08T09:14:06.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:06.064+0000 7f4bd1c63700 1 -- 172.21.15.49:0/1121405298 wait complete. 2023-10-08T09:14:06.353 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:06.689 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:06.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:06 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/557793642' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T09:14:06.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:06 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:06 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:06 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:06 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.709 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:06 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1121405298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T09:14:06.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397544 got 210453397545 for osd.7 2023-10-08T09:14:06.720 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:06.770 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:06 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/557793642' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T09:14:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:06 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:06 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:06 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:06 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:06 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1121405298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T09:14:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:06 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/557793642' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-08T09:14:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:06 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:06 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:06 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:06 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:06.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:06 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1121405298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-08T09:14:07.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 -- 172.21.15.49:0/2192210663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101056a0 msgr2=0x7f6710105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:07.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 --2- 172.21.15.49:0/2192210663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101056a0 0x7f6710105a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f670c009990 tx=0x7f670c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 -- 172.21.15.49:0/2192210663 shutdown_connections 2023-10-08T09:14:07.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 --2- 172.21.15.49:0/2192210663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6710101260 0x7f671010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 --2- 172.21.15.49:0/2192210663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101056a0 0x7f6710105a80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 --2- 172.21.15.49:0/2192210663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67101008a0 0x7f6710100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 -- 172.21.15.49:0/2192210663 >> 172.21.15.49:0/2192210663 conn(0x7f6710077950 msgr2=0x7f6710077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 -- 172.21.15.49:0/2192210663 shutdown_connections 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 -- 172.21.15.49:0/2192210663 wait complete. 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.624+0000 7f6717ffc700 1 Processor -- start 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6717ffc700 1 -- start start 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6717ffc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101008a0 0x7f6710072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6717ffc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6710101260 0x7f6710073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:07.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6717ffc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67101056a0 0x7f6710073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6717ffc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67101a4f90 con 0x7f67101008a0 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6717ffc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67101a5100 con 0x7f6710101260 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6717ffc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67101a52c0 con 0x7f67101056a0 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101008a0 0x7f6710072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101008a0 0x7f6710072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48750/0 (socket says 172.21.15.49:48750) 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 -- 172.21.15.49:0/3821338040 learned_addr learned my addr 172.21.15.49:0/3821338040 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715597700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6710101260 0x7f6710073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:07.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6716599700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67101056a0 0x7f6710073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 -- 172.21.15.49:0/3821338040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67101056a0 msgr2=0x7f6710073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67101056a0 0x7f6710073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 -- 172.21.15.49:0/3821338040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6710101260 msgr2=0x7f6710073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6710101260 0x7f6710073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 -- 172.21.15.49:0/3821338040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0095f0 con 0x7f67101008a0 2023-10-08T09:14:07.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6716599700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67101056a0 0x7f6710073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.625+0000 7f6715d98700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101008a0 0x7f6710072af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f670c000c00 tx=0x7f670c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.626+0000 7f6706ffd700 1 -- 172.21.15.49:0/3821338040 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c01d070 con 0x7f67101008a0 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.626+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67101a5540 con 0x7f67101008a0 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.626+0000 7f6706ffd700 1 -- 172.21.15.49:0/3821338040 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f670c022470 con 0x7f67101008a0 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.626+0000 7f6706ffd700 1 -- 172.21.15.49:0/3821338040 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c017650 con 0x7f67101008a0 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.626+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67101a59e0 con 0x7f67101008a0 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.626+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6710108d10 con 0x7f67101008a0 2023-10-08T09:14:07.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.628+0000 7f6706ffd700 1 -- 172.21.15.49:0/3821338040 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f670c0225e0 con 0x7f67101008a0 2023-10-08T09:14:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.628+0000 7f6706ffd700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66fc0a1510 0x7f66fc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.628+0000 7f6715597700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66fc0a1510 0x7f66fc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.628+0000 7f6706ffd700 1 -- 172.21.15.49:0/3821338040 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f670c0cea20 con 0x7f67101008a0 2023-10-08T09:14:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.629+0000 7f6715597700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66fc0a1510 0x7f66fc0a39d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6710074220 tx=0x7f670000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:07.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.632+0000 7f6706ffd700 1 -- 172.21.15.49:0/3821338040 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f670c097dc0 con 0x7f67101008a0 2023-10-08T09:14:07.735 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906974 got 274877906976 for osd.9 2023-10-08T09:14:07.735 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:07.760 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:07 smithi049 ceph-mon[99485]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2023-10-08T09:14:07.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:07 smithi132 ceph-mon[103389]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2023-10-08T09:14:07.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.882+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f6710101ec0 con 0x7f67101008a0 2023-10-08T09:14:07.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.883+0000 7f6706ffd700 1 -- 172.21.15.49:0/3821338040 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f670c09b3e0 con 0x7f67101008a0 2023-10-08T09:14:07.884 INFO:teuthology.orchestra.run.smithi049.stdout:240518168613 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66fc0a1510 msgr2=0x7f66fc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66fc0a1510 0x7f66fc0a39d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6710074220 tx=0x7f670000b410 comp rx=0 tx=0).stop 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101008a0 msgr2=0x7f6710072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101008a0 0x7f6710072af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f670c000c00 tx=0x7f670c004730 comp rx=0 tx=0).stop 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 shutdown_connections 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66fc0a1510 0x7f66fc0a39d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67101056a0 0x7f6710073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f67101008a0 0x7f6710072af0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 --2- 172.21.15.49:0/3821338040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6710101260 0x7f6710073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:07.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.885+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 >> 172.21.15.49:0/3821338040 conn(0x7f6710077950 msgr2=0x7f67100fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:07.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.886+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 shutdown_connections 2023-10-08T09:14:07.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:07.886+0000 7f6717ffc700 1 -- 172.21.15.49:0/3821338040 wait complete. 2023-10-08T09:14:07.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:07 smithi184 ceph-mon[103675]: pgmap v303: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2023-10-08T09:14:08.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.101+0000 7fc51eef9700 1 -- 172.21.15.49:0/804838836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc518068190 msgr2=0x7fc518068570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.101+0000 7fc51eef9700 1 --2- 172.21.15.49:0/804838836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc518068190 0x7fc518068570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009990 tx=0x7fc50c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.102+0000 7fc51eef9700 1 -- 172.21.15.49:0/804838836 shutdown_connections 2023-10-08T09:14:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.102+0000 7fc51eef9700 1 --2- 172.21.15.49:0/804838836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc51810d300 0x7fc51810f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.102+0000 7fc51eef9700 1 --2- 172.21.15.49:0/804838836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc518068190 0x7fc518068570 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.102+0000 7fc51eef9700 1 --2- 172.21.15.49:0/804838836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc518068ab0 0x7fc51810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.102+0000 7fc51eef9700 1 -- 172.21.15.49:0/804838836 >> 172.21.15.49:0/804838836 conn(0x7fc518075790 msgr2=0x7fc518075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.102+0000 7fc51eef9700 1 -- 172.21.15.49:0/804838836 shutdown_connections 2023-10-08T09:14:08.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.102+0000 7fc51eef9700 1 -- 172.21.15.49:0/804838836 wait complete. 2023-10-08T09:14:08.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 Processor -- start 2023-10-08T09:14:08.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 -- start start 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc518068190 0x7fc518198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc518068ab0 0x7fc518199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc51810d300 0x7fc51819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51819dd30 con 0x7fc51810d300 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51819dea0 con 0x7fc518068190 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51eef9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51819e060 con 0x7fc518068ab0 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51d496700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc51810d300 0x7fc51819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51d496700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc51810d300 0x7fc51819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48780/0 (socket says 172.21.15.49:48780) 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51d496700 1 -- 172.21.15.49:0/573417247 learned_addr learned my addr 172.21.15.49:0/573417247 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc517fff700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc518068ab0 0x7fc518199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.103+0000 7fc51cc95700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc518068190 0x7fc518198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51d496700 1 -- 172.21.15.49:0/573417247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc518068ab0 msgr2=0x7fc518199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51d496700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc518068ab0 0x7fc518199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51d496700 1 -- 172.21.15.49:0/573417247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc518068190 msgr2=0x7fc518198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51d496700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc518068190 0x7fc518198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51d496700 1 -- 172.21.15.49:0/573417247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c0095f0 con 0x7fc51810d300 2023-10-08T09:14:08.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51d496700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc51810d300 0x7fc51819d720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc51000ef90 tx=0x7fc51000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 -- 172.21.15.49:0/733371850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068190 msgr2=0x7f08300685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 --2- 172.21.15.49:0/733371850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068190 0x7f08300685f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0820009990 tx=0x7f0820009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 -- 172.21.15.49:0/733371850 shutdown_connections 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 --2- 172.21.15.49:0/733371850 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08301067a0 0x7f0830106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 --2- 172.21.15.49:0/733371850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0830068b30 0x7f083010d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 --2- 172.21.15.49:0/733371850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068190 0x7f08300685f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 -- 172.21.15.49:0/733371850 >> 172.21.15.49:0/733371850 conn(0x7f0830075790 msgr2=0x7f0830075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.035+0000 7f0837a0a700 1 -- 172.21.15.49:0/733371850 shutdown_connections 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 -- 172.21.15.49:0/733371850 wait complete. 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 Processor -- start 2023-10-08T09:14:08.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 -- start start 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0830068190 0x7f083019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068b30 0x7f083019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08301067a0 0x7f0830196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830197240 con 0x7f0830068b30 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08301973b0 con 0x7f0830068190 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0837a0a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830197570 con 0x7f08301067a0 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.036+0000 7f0834fa5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068b30 0x7f083019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068b30 0x7f083019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48764/0 (socket says 172.21.15.49:48764) 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 -- 172.21.15.49:0/181728311 learned_addr learned my addr 172.21.15.49:0/181728311 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f08357a6700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0830068190 0x7f083019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0835fa7700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08301067a0 0x7f0830196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 -- 172.21.15.49:0/181728311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08301067a0 msgr2=0x7f0830196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08301067a0 0x7f0830196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 -- 172.21.15.49:0/181728311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0830068190 msgr2=0x7f083019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0830068190 0x7f083019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 -- 172.21.15.49:0/181728311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200095f0 con 0x7f0830068b30 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0834fa5700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068b30 0x7f083019cf20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f08200098e0 tx=0x7f082000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f08267fc700 1 -- 172.21.15.49:0/181728311 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082001d070 con 0x7f0830068b30 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f08267fc700 1 -- 172.21.15.49:0/181728311 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0820022470 con 0x7f0830068b30 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08301977f0 con 0x7f0830068b30 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f08267fc700 1 -- 172.21.15.49:0/181728311 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820017650 con 0x7f0830068b30 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.037+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08301a3960 con 0x7f0830068b30 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.038+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0830108d50 con 0x7f0830068b30 2023-10-08T09:14:08.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.039+0000 7f08267fc700 1 -- 172.21.15.49:0/181728311 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f08200225e0 con 0x7f0830068b30 2023-10-08T09:14:08.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.040+0000 7f08267fc700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f081c0a1590 0x7f081c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.040+0000 7f08267fc700 1 -- 172.21.15.49:0/181728311 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f08200ce470 con 0x7f0830068b30 2023-10-08T09:14:08.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.040+0000 7f08357a6700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f081c0a1590 0x7f081c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.040+0000 7f08357a6700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f081c0a1590 0x7f081c0a3a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f082c005950 tx=0x7f082c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:08.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.048+0000 7f08267fc700 1 -- 172.21.15.49:0/181728311 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f08200976f0 con 0x7f0830068b30 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc515ffb700 1 -- 172.21.15.49:0/573417247 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510018070 con 0x7fc51810d300 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc515ffb700 1 -- 172.21.15.49:0/573417247 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc51000f040 con 0x7fc51810d300 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc515ffb700 1 -- 172.21.15.49:0/573417247 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000cda0 con 0x7fc51810d300 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51819e2e0 con 0x7fc51810d300 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.104+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5181a5ba0 con 0x7fc51810d300 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.106+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc518108520 con 0x7fc51810d300 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.107+0000 7fc515ffb700 1 -- 172.21.15.49:0/573417247 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc510003a00 con 0x7fc51810d300 2023-10-08T09:14:08.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.108+0000 7fc515ffb700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc5080a1510 0x7fc5080a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:08.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.108+0000 7fc51cc95700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc5080a1510 0x7fc5080a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:08.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.108+0000 7fc515ffb700 1 -- 172.21.15.49:0/573417247 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5100cd640 con 0x7fc51810d300 2023-10-08T09:14:08.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.108+0000 7fc51cc95700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc5080a1510 0x7fc5080a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00b2e0 tx=0x7fc50c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:08.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.111+0000 7fc515ffb700 1 -- 172.21.15.49:0/573417247 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc5100968c0 con 0x7fc51810d300 2023-10-08T09:14:08.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.261+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0830197f70 con 0x7f0830068b30 2023-10-08T09:14:08.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.261+0000 7f08267fc700 1 -- 172.21.15.49:0/181728311 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f082009ad10 con 0x7f0830068b30 2023-10-08T09:14:08.263 INFO:teuthology.orchestra.run.smithi049.stdout:188978561070 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f081c0a1590 msgr2=0x7f081c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f081c0a1590 0x7f081c0a3a50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f082c005950 tx=0x7f082c00b410 comp rx=0 tx=0).stop 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068b30 msgr2=0x7f083019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068b30 0x7f083019cf20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f08200098e0 tx=0x7f082000f670 comp rx=0 tx=0).stop 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 shutdown_connections 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f081c0a1590 0x7f081c0a3a50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08301067a0 0x7f0830196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0830068190 0x7f083019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 --2- 172.21.15.49:0/181728311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0830068b30 0x7f083019cf20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 >> 172.21.15.49:0/181728311 conn(0x7f0830075790 msgr2=0x7f08300fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 shutdown_connections 2023-10-08T09:14:08.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.263+0000 7f0837a0a700 1 -- 172.21.15.49:0/181728311 wait complete. 2023-10-08T09:14:08.327 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.326+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc5180ffd30 con 0x7fc51810d300 2023-10-08T09:14:08.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.326+0000 7fc515ffb700 1 -- 172.21.15.49:0/573417247 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc510099ee0 con 0x7fc51810d300 2023-10-08T09:14:08.328 INFO:teuthology.orchestra.run.smithi049.stdout:34359738439 2023-10-08T09:14:08.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc5080a1510 msgr2=0x7fc5080a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc5080a1510 0x7fc5080a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00b2e0 tx=0x7fc50c005960 comp rx=0 tx=0).stop 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc51810d300 msgr2=0x7fc51819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc51810d300 0x7fc51819d720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc51000ef90 tx=0x7fc51000c3a0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 shutdown_connections 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc5080a1510 0x7fc5080a39d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc51810d300 0x7fc51819d720 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc518068190 0x7fc518198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 --2- 172.21.15.49:0/573417247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc518068ab0 0x7fc518199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:08.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 >> 172.21.15.49:0/573417247 conn(0x7fc518075790 msgr2=0x7fc51810e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:08.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 shutdown_connections 2023-10-08T09:14:08.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:08.328+0000 7fc51eef9700 1 -- 172.21.15.49:0/573417247 wait complete. 2023-10-08T09:14:08.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:08 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3821338040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T09:14:08.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:08 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/181728311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T09:14:08.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:08 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/573417247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T09:14:08.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:08 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3821338040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T09:14:08.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:08 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/181728311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T09:14:08.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:08 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/573417247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T09:14:08.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:08 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3821338040' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-08T09:14:08.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:08 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/181728311' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-08T09:14:08.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:08 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/573417247' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-08T09:14:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 -- 172.21.15.49:0/701856063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42040ff6e0 msgr2=0x7f42040ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 --2- 172.21.15.49:0/701856063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42040ff6e0 0x7f42040ffb60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009990 tx=0x7f41ec009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 -- 172.21.15.49:0/701856063 shutdown_connections 2023-10-08T09:14:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 --2- 172.21.15.49:0/701856063 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42041000a0 0x7f420410c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 --2- 172.21.15.49:0/701856063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42041045b0 0x7f4204104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 --2- 172.21.15.49:0/701856063 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42040ff6e0 0x7f42040ffb60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 -- 172.21.15.49:0/701856063 >> 172.21.15.49:0/701856063 conn(0x7f42040766c0 msgr2=0x7f4204076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 -- 172.21.15.49:0/701856063 shutdown_connections 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.104+0000 7f4209155700 1 -- 172.21.15.49:0/701856063 wait complete. 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 Processor -- start 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 -- start start 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42040ff6e0 0x7f4204197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42041000a0 0x7f4204197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42041045b0 0x7f42041a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4204198870 con 0x7f42041045b0 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42041a3030 con 0x7f42041000a0 2023-10-08T09:14:09.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4209155700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42041a3170 con 0x7f42040ff6e0 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f420359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42041045b0 0x7f42041a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f420359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42041045b0 0x7f42041a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48794/0 (socket says 172.21.15.49:48794) 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f4202d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42040ff6e0 0x7f4204197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f420359e700 1 -- 172.21.15.49:0/1851232303 learned_addr learned my addr 172.21.15.49:0/1851232303 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.105+0000 7f420259c700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42041000a0 0x7f4204197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f420359e700 1 -- 172.21.15.49:0/1851232303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42040ff6e0 msgr2=0x7f4204197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f420359e700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42040ff6e0 0x7f4204197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f420359e700 1 -- 172.21.15.49:0/1851232303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42041000a0 msgr2=0x7f4204197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f420359e700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42041000a0 0x7f4204197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f420359e700 1 -- 172.21.15.49:0/1851232303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0095f0 con 0x7f42041045b0 2023-10-08T09:14:09.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f420359e700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42041045b0 0x7f42041a2af0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00ef30 tx=0x7f41fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f41fbfff700 1 -- 172.21.15.49:0/1851232303 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc018070 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42041a3390 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f41fbfff700 1 -- 172.21.15.49:0/1851232303 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41fc00f040 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f41fbfff700 1 -- 172.21.15.49:0/1851232303 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00cd00 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.106+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42041a3860 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.107+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e40052f0 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.108+0000 7f41fbfff700 1 -- 172.21.15.49:0/1851232303 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f41fc004ad0 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.109+0000 7f41fbfff700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f41f00a1510 0x7f41f00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.109+0000 7f4202d9d700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f41f00a1510 0x7f41f00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.109+0000 7f41fbfff700 1 -- 172.21.15.49:0/1851232303 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f41fc0cd5a0 con 0x7f42041045b0 2023-10-08T09:14:09.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.109+0000 7f4202d9d700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f41f00a1510 0x7f41f00a39d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f41f4007b60 tx=0x7f41f40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:09.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.113+0000 7f41fbfff700 1 -- 172.21.15.49:0/1851232303 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f41fc096820 con 0x7f42041045b0 2023-10-08T09:14:09.308 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738438 got 34359738439 for osd.0 2023-10-08T09:14:09.309 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.327+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f41e4005870 con 0x7f42041045b0 2023-10-08T09:14:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.327+0000 7f41fbfff700 1 -- 172.21.15.49:0/1851232303 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f41fc099e40 con 0x7f42041045b0 2023-10-08T09:14:09.329 INFO:teuthology.orchestra.run.smithi049.stdout:55834574915 2023-10-08T09:14:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.329+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f41f00a1510 msgr2=0x7f41f00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.329+0000 7f4209155700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f41f00a1510 0x7f41f00a39d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f41f4007b60 tx=0x7f41f40073d0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42041045b0 msgr2=0x7f42041a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42041045b0 0x7f42041a2af0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00ef30 tx=0x7f41fc00c3a0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 shutdown_connections 2023-10-08T09:14:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f41f00a1510 0x7f41f00a39d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42040ff6e0 0x7f4204197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f42041045b0 0x7f42041a2af0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 --2- 172.21.15.49:0/1851232303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42041000a0 0x7f4204197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 >> 172.21.15.49:0/1851232303 conn(0x7f42040766c0 msgr2=0x7f42040fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 shutdown_connections 2023-10-08T09:14:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:09.330+0000 7f4209155700 1 -- 172.21.15.49:0/1851232303 wait complete. 2023-10-08T09:14:09.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561068 got 188978561070 for osd.6 2023-10-08T09:14:09.700 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:09.701 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168612 got 240518168613 for osd.8 2023-10-08T09:14:09.701 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:09.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:09 smithi049 ceph-mon[99485]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T09:14:09.759 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:09 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1851232303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T09:14:09.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:09 smithi132 ceph-mon[103389]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T09:14:09.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:09 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1851232303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T09:14:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:09 smithi184 ceph-mon[103675]: pgmap v304: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T09:14:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:09 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1851232303' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-08T09:14:10.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 -- 172.21.15.49:0/531675124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 msgr2=0x7f0fa8100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 --2- 172.21.15.49:0/531675124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa8100720 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009990 tx=0x7f0f9c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 -- 172.21.15.49:0/531675124 shutdown_connections 2023-10-08T09:14:10.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 --2- 172.21.15.49:0/531675124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8104720 0x7f0fa8104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 --2- 172.21.15.49:0/531675124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8100c60 0x7f0fa810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 --2- 172.21.15.49:0/531675124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa8100720 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 -- 172.21.15.49:0/531675124 >> 172.21.15.49:0/531675124 conn(0x7f0fa80749b0 msgr2=0x7f0fa8074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:10.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 -- 172.21.15.49:0/531675124 shutdown_connections 2023-10-08T09:14:10.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.069+0000 7f0fae734700 1 -- 172.21.15.49:0/531675124 wait complete. 2023-10-08T09:14:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.070+0000 7f0fae734700 1 Processor -- start 2023-10-08T09:14:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.070+0000 7f0fae734700 1 -- start start 2023-10-08T09:14:10.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fae734700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fae734700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8100c60 0x7f0fa8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fae734700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8104720 0x7f0fa81a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fae734700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81987b0 con 0x7f0fa81002a0 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fae734700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81a2f70 con 0x7f0fa8104720 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fae734700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81a30b0 con 0x7f0fa8100c60 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa81979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48804/0 (socket says 172.21.15.49:48804) 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 -- 172.21.15.49:0/228252341 learned_addr learned my addr 172.21.15.49:0/228252341 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:10.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa77fe700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8100c60 0x7f0fa8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0faccd1700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8104720 0x7f0fa81a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 -- 172.21.15.49:0/228252341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8100c60 msgr2=0x7f0fa8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8100c60 0x7f0fa8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 -- 172.21.15.49:0/228252341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8104720 msgr2=0x7f0fa81a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8104720 0x7f0fa81a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa7fff700 1 -- 172.21.15.49:0/228252341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0095f0 con 0x7f0fa81002a0 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.071+0000 7f0fa77fe700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8100c60 0x7f0fa8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:10.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.072+0000 7f0fa7fff700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa81979b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800c9b0 tx=0x7f0f9800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:10.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.072+0000 7f0fa57fa700 1 -- 172.21.15.49:0/228252341 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98007b00 con 0x7f0fa81002a0 2023-10-08T09:14:10.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.072+0000 7f0fa57fa700 1 -- 172.21.15.49:0/228252341 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f98007c60 con 0x7f0fa81002a0 2023-10-08T09:14:10.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.072+0000 7f0fa57fa700 1 -- 172.21.15.49:0/228252341 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98018660 con 0x7f0fa81002a0 2023-10-08T09:14:10.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.072+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa8195800 con 0x7f0fa81002a0 2023-10-08T09:14:10.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.073+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa8195cb0 con 0x7f0fa81002a0 2023-10-08T09:14:10.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.073+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa8068a00 con 0x7f0fa81002a0 2023-10-08T09:14:10.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.075+0000 7f0fa57fa700 1 -- 172.21.15.49:0/228252341 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0f9800f450 con 0x7f0fa81002a0 2023-10-08T09:14:10.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.076+0000 7f0fa57fa700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f900a1590 0x7f0f900a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.076+0000 7f0fa57fa700 1 -- 172.21.15.49:0/228252341 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f980cd300 con 0x7f0fa81002a0 2023-10-08T09:14:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.076+0000 7f0fa77fe700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f900a1590 0x7f0f900a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.077+0000 7f0fa77fe700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f900a1590 0x7f0f900a3a50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00b5c0 tx=0x7f0f9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:10.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.079+0000 7f0fa57fa700 1 -- 172.21.15.49:0/228252341 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0f98096600 con 0x7f0fa81002a0 2023-10-08T09:14:10.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.295+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0fa810f6f0 con 0x7f0fa81002a0 2023-10-08T09:14:10.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.296+0000 7f0fa57fa700 1 -- 172.21.15.49:0/228252341 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f98099c20 con 0x7f0fa81002a0 2023-10-08T09:14:10.297 INFO:teuthology.orchestra.run.smithi049.stdout:158913790002 2023-10-08T09:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f900a1590 msgr2=0x7f0f900a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f900a1590 0x7f0f900a3a50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00b5c0 tx=0x7f0f9c01a040 comp rx=0 tx=0).stop 2023-10-08T09:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 msgr2=0x7f0fa81979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa81979b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800c9b0 tx=0x7f0f9800ccc0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 shutdown_connections 2023-10-08T09:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f900a1590 0x7f0f900a3a50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0fa8104720 0x7f0fa81a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0fa81002a0 0x7f0fa81979b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 --2- 172.21.15.49:0/228252341 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fa8100c60 0x7f0fa8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 >> 172.21.15.49:0/228252341 conn(0x7f0fa80749b0 msgr2=0x7f0fa80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 shutdown_connections 2023-10-08T09:14:10.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.299+0000 7f0fae734700 1 -- 172.21.15.49:0/228252341 wait complete. 2023-10-08T09:14:10.312 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574913 got 55834574915 for osd.1 2023-10-08T09:14:10.312 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:10.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.390+0000 7f0ab1f96700 1 -- 172.21.15.49:0/384842557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac1057c0 msgr2=0x7f0aac105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.390+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/384842557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac1057c0 0x7f0aac105ba0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009990 tx=0x7f0a9c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.391+0000 7f0ab1f96700 1 -- 172.21.15.49:0/384842557 shutdown_connections 2023-10-08T09:14:10.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.391+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/384842557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aac101260 0x7f0aac10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.391+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/384842557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac1057c0 0x7f0aac105ba0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.391+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/384842557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac1008a0 0x7f0aac100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.391+0000 7f0ab1f96700 1 -- 172.21.15.49:0/384842557 >> 172.21.15.49:0/384842557 conn(0x7f0aac0779e0 msgr2=0x7f0aac077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:10.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.391+0000 7f0ab1f96700 1 -- 172.21.15.49:0/384842557 shutdown_connections 2023-10-08T09:14:10.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.391+0000 7f0ab1f96700 1 -- 172.21.15.49:0/384842557 wait complete. 2023-10-08T09:14:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.392+0000 7f0ab1f96700 1 Processor -- start 2023-10-08T09:14:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.392+0000 7f0ab1f96700 1 -- start start 2023-10-08T09:14:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0ab1f96700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac1008a0 0x7f0aac072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0ab1f96700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac101260 0x7f0aac073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0ab1f96700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aac1057c0 0x7f0aac073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0ab1f96700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac1a4f60 con 0x7f0aac101260 2023-10-08T09:14:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0ab1f96700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac1a50d0 con 0x7f0aac1008a0 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0ab1f96700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac1a5290 con 0x7f0aac1057c0 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac101260 0x7f0aac073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac101260 0x7f0aac073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48820/0 (socket says 172.21.15.49:48820) 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 -- 172.21.15.49:0/3265764548 learned_addr learned my addr 172.21.15.49:0/3265764548 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aab7fe700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac1008a0 0x7f0aac072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aabfff700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aac1057c0 0x7f0aac073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 -- 172.21.15.49:0/3265764548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aac1057c0 msgr2=0x7f0aac073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aac1057c0 0x7f0aac073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 -- 172.21.15.49:0/3265764548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac1008a0 msgr2=0x7f0aac072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac1008a0 0x7f0aac072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aaaffd700 1 -- 172.21.15.49:0/3265764548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0095f0 con 0x7f0aac101260 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.393+0000 7f0aabfff700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aac1057c0 0x7f0aac073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.394+0000 7f0aaaffd700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac101260 0x7f0aac073030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d950 tx=0x7f0a9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.394+0000 7f0aa8ff9700 1 -- 172.21.15.49:0/3265764548 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94009940 con 0x7f0aac101260 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.394+0000 7f0aa8ff9700 1 -- 172.21.15.49:0/3265764548 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a94010460 con 0x7f0aac101260 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.394+0000 7f0aa8ff9700 1 -- 172.21.15.49:0/3265764548 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400f620 con 0x7f0aac101260 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.394+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac1a5570 con 0x7f0aac101260 2023-10-08T09:14:10.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.394+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac1a5a20 con 0x7f0aac101260 2023-10-08T09:14:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.395+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac108d80 con 0x7f0aac101260 2023-10-08T09:14:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.396+0000 7f0aa8ff9700 1 -- 172.21.15.49:0/3265764548 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0a940108d0 con 0x7f0aac101260 2023-10-08T09:14:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.397+0000 7f0aa8ff9700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a980a1510 0x7f0a980a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.397+0000 7f0aa8ff9700 1 -- 172.21.15.49:0/3265764548 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0a940cd010 con 0x7f0aac101260 2023-10-08T09:14:10.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.397+0000 7f0aab7fe700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a980a1510 0x7f0a980a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.397+0000 7f0aab7fe700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a980a1510 0x7f0a980a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006010 tx=0x7f0a9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:10.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.401+0000 7f0aa8ff9700 1 -- 172.21.15.49:0/3265764548 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a94096200 con 0x7f0aac101260 2023-10-08T09:14:10.513 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:10.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.627+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0aac073fd0 con 0x7f0aac101260 2023-10-08T09:14:10.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.628+0000 7f0aa8ff9700 1 -- 172.21.15.49:0/3265764548 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a94099820 con 0x7f0aac101260 2023-10-08T09:14:10.630 INFO:teuthology.orchestra.run.smithi049.stdout:77309411390 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a980a1510 msgr2=0x7f0a980a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a980a1510 0x7f0a980a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006010 tx=0x7f0a9c005960 comp rx=0 tx=0).stop 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac101260 msgr2=0x7f0aac073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac101260 0x7f0aac073030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d950 tx=0x7f0a9400dc60 comp rx=0 tx=0).stop 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 shutdown_connections 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0a980a1510 0x7f0a980a39d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aac1057c0 0x7f0aac073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0aac1008a0 0x7f0aac072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 --2- 172.21.15.49:0/3265764548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0aac101260 0x7f0aac073030 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 >> 172.21.15.49:0/3265764548 conn(0x7f0aac0779e0 msgr2=0x7f0aac0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:10.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 shutdown_connections 2023-10-08T09:14:10.633 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.630+0000 7f0ab1f96700 1 -- 172.21.15.49:0/3265764548 wait complete. 2023-10-08T09:14:10.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:10 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/228252341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T09:14:10.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:10 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/228252341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T09:14:10.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.855+0000 7f4e62c62700 1 -- 172.21.15.49:0/18858648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c1056b0 msgr2=0x7f4e5c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.855+0000 7f4e62c62700 1 --2- 172.21.15.49:0/18858648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c1056b0 0x7f4e5c105a90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009990 tx=0x7f4e44009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- 172.21.15.49:0/18858648 shutdown_connections 2023-10-08T09:14:10.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 --2- 172.21.15.49:0/18858648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e5c100110 0x7f4e5c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 --2- 172.21.15.49:0/18858648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c1056b0 0x7f4e5c105a90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 --2- 172.21.15.49:0/18858648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c0ff750 0x7f4e5c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- 172.21.15.49:0/18858648 >> 172.21.15.49:0/18858648 conn(0x7f4e5c075790 msgr2=0x7f4e5c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:10.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- 172.21.15.49:0/18858648 shutdown_connections 2023-10-08T09:14:10.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- 172.21.15.49:0/18858648 wait complete. 2023-10-08T09:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 Processor -- start 2023-10-08T09:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- start start 2023-10-08T09:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c0ff750 0x7f4e5c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c100110 0x7f4e5c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e5c1056b0 0x7f4e5c195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c195f70 con 0x7f4e5c0ff750 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c1960e0 con 0x7f4e5c1056b0 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.856+0000 7f4e62c62700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c1962a0 con 0x7f4e5c100110 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c0ff750 0x7f4e5c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c0ff750 0x7f4e5c19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48832/0 (socket says 172.21.15.49:48832) 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 -- 172.21.15.49:0/2619346391 learned_addr learned my addr 172.21.15.49:0/2619346391 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e5bfff700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c100110 0x7f4e5c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e611ff700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e5c1056b0 0x7f4e5c195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 -- 172.21.15.49:0/2619346391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c100110 msgr2=0x7f4e5c19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c100110 0x7f4e5c19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 -- 172.21.15.49:0/2619346391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e5c1056b0 msgr2=0x7f4e5c195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e5c1056b0 0x7f4e5c195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 -- 172.21.15.49:0/2619346391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e440095f0 con 0x7f4e5c0ff750 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e609fe700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c0ff750 0x7f4e5c19b7d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4e440098e0 tx=0x7f4e4400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e59ffb700 1 -- 172.21.15.49:0/2619346391 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4401d070 con 0x7f4e5c0ff750 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e59ffb700 1 -- 172.21.15.49:0/2619346391 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e44022470 con 0x7f4e5c0ff750 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e59ffb700 1 -- 172.21.15.49:0/2619346391 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44017650 con 0x7f4e5c0ff750 2023-10-08T09:14:10.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.857+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c196520 con 0x7f4e5c0ff750 2023-10-08T09:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.858+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1a27c0 con 0x7f4e5c0ff750 2023-10-08T09:14:10.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.859+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c107c40 con 0x7f4e5c0ff750 2023-10-08T09:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.860+0000 7f4e59ffb700 1 -- 172.21.15.49:0/2619346391 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4e440225e0 con 0x7f4e5c0ff750 2023-10-08T09:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.860+0000 7f4e59ffb700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e480a1510 0x7f4e480a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.861+0000 7f4e5bfff700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e480a1510 0x7f4e480a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.861+0000 7f4e59ffb700 1 -- 172.21.15.49:0/2619346391 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4e440ce190 con 0x7f4e5c0ff750 2023-10-08T09:14:10.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.861+0000 7f4e5bfff700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e480a1510 0x7f4e480a39d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c196ad0 tx=0x7f4e4c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:10.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:10.865+0000 7f4e59ffb700 1 -- 172.21.15.49:0/2619346391 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4e44097410 con 0x7f4e5c0ff750 2023-10-08T09:14:10.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:10 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/228252341' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-08T09:14:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.072+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f4e5c100df0 con 0x7f4e5c0ff750 2023-10-08T09:14:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.072+0000 7f4e59ffb700 1 -- 172.21.15.49:0/2619346391 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f4e4409aa30 con 0x7f4e5c0ff750 2023-10-08T09:14:11.074 INFO:teuthology.orchestra.run.smithi049.stdout:296352743453 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e480a1510 msgr2=0x7f4e480a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e480a1510 0x7f4e480a39d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c196ad0 tx=0x7f4e4c00a300 comp rx=0 tx=0).stop 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c0ff750 msgr2=0x7f4e5c19b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c0ff750 0x7f4e5c19b7d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4e440098e0 tx=0x7f4e4400f670 comp rx=0 tx=0).stop 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 shutdown_connections 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4e480a1510 0x7f4e480a39d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e5c1056b0 0x7f4e5c195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4e5c0ff750 0x7f4e5c19b7d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 --2- 172.21.15.49:0/2619346391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e5c100110 0x7f4e5c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 >> 172.21.15.49:0/2619346391 conn(0x7f4e5c075790 msgr2=0x7f4e5c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 shutdown_connections 2023-10-08T09:14:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:11.074+0000 7f4e62c62700 1 -- 172.21.15.49:0/2619346391 wait complete. 2023-10-08T09:14:11.342 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411388 got 77309411390 for osd.2 2023-10-08T09:14:11.342 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:11 smithi049 ceph-mon[99485]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T09:14:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:11 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3265764548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T09:14:11.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:11 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2619346391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T09:14:11.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:11 smithi132 ceph-mon[103389]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T09:14:11.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:11 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3265764548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T09:14:11.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:11 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2619346391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T09:14:11.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790000 got 158913790002 for osd.5 2023-10-08T09:14:11.970 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:11 smithi184 ceph-mon[103675]: pgmap v305: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-08T09:14:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:11 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3265764548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-08T09:14:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:11 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2619346391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-08T09:14:12.247 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743451 got 296352743453 for osd.10 2023-10-08T09:14:12.247 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:12.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 -- 172.21.15.49:0/3198886152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a81008a0 msgr2=0x7ff8a8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:12.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/3198886152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a81008a0 0x7ff8a8100d20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009990 tx=0x7ff8a4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:12.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 -- 172.21.15.49:0/3198886152 shutdown_connections 2023-10-08T09:14:12.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/3198886152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8a81057c0 0x7ff8a8105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:12.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/3198886152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8a8101260 0x7ff8a810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:12.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/3198886152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a81008a0 0x7ff8a8100d20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:12.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 -- 172.21.15.49:0/3198886152 >> 172.21.15.49:0/3198886152 conn(0x7ff8a80779e0 msgr2=0x7ff8a8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:12.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 -- 172.21.15.49:0/3198886152 shutdown_connections 2023-10-08T09:14:12.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.981+0000 7ff8afa0b700 1 -- 172.21.15.49:0/3198886152 wait complete. 2023-10-08T09:14:12.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 Processor -- start 2023-10-08T09:14:12.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 -- start start 2023-10-08T09:14:12.986 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8a81008a0 0x7ff8a8072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101260 0x7ff8a8073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8a81057c0 0x7ff8a8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a81a4f50 con 0x7ff8a8101260 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a81a50c0 con 0x7ff8a81057c0 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8afa0b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a81a5280 con 0x7ff8a81008a0 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8acfa6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101260 0x7ff8a8073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8acfa6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101260 0x7ff8a8073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48848/0 (socket says 172.21.15.49:48848) 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8acfa6700 1 -- 172.21.15.49:0/2653283677 learned_addr learned my addr 172.21.15.49:0/2653283677 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.982+0000 7ff8ad7a7700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8a81008a0 0x7ff8a8072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8adfa8700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8a81057c0 0x7ff8a8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:12.987 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8acfa6700 1 -- 172.21.15.49:0/2653283677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8a81008a0 msgr2=0x7ff8a8072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8acfa6700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8a81008a0 0x7ff8a8072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8acfa6700 1 -- 172.21.15.49:0/2653283677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8a81057c0 msgr2=0x7ff8a8073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8acfa6700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8a81057c0 0x7ff8a8073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8acfa6700 1 -- 172.21.15.49:0/2653283677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40095f0 con 0x7ff8a8101260 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8acfa6700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101260 0x7ff8a8073030 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a400f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff89e7fc700 1 -- 172.21.15.49:0/2653283677 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a401d070 con 0x7ff8a8101260 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a81a5500 con 0x7ff8a8101260 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff89e7fc700 1 -- 172.21.15.49:0/2653283677 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8a400fbf0 con 0x7ff8a8101260 2023-10-08T09:14:12.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff89e7fc700 1 -- 172.21.15.49:0/2653283677 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40176e0 con 0x7ff8a8101260 2023-10-08T09:14:12.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.983+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a81a5970 con 0x7ff8a8101260 2023-10-08T09:14:12.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.984+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a8101f30 con 0x7ff8a8101260 2023-10-08T09:14:12.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.985+0000 7ff89e7fc700 1 -- 172.21.15.49:0/2653283677 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff8a40229c0 con 0x7ff8a8101260 2023-10-08T09:14:12.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.986+0000 7ff89e7fc700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff8940a1510 0x7ff8940a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:12.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.986+0000 7ff8ad7a7700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff8940a1510 0x7ff8940a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:12.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.986+0000 7ff89e7fc700 1 -- 172.21.15.49:0/2653283677 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff8a40cecd0 con 0x7ff8a8101260 2023-10-08T09:14:12.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.986+0000 7ff8ad7a7700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff8940a1510 0x7ff8940a39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8101930 tx=0x7ff89800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:12.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:12.994+0000 7ff89e7fc700 1 -- 172.21.15.49:0/2653283677 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff8a4097f50 con 0x7ff8a8101260 2023-10-08T09:14:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.090+0000 7f432136d700 1 -- 172.21.15.49:0/957240605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068b30 msgr2=0x7f431c10d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:13.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.090+0000 7f432136d700 1 --2- 172.21.15.49:0/957240605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068b30 0x7f431c10d670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f431400b0a0 tx=0x7f431400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.090+0000 7f432136d700 1 -- 172.21.15.49:0/957240605 shutdown_connections 2023-10-08T09:14:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 --2- 172.21.15.49:0/957240605 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068b30 0x7f431c10d670 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 --2- 172.21.15.49:0/957240605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c1067a0 0x7f431c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 --2- 172.21.15.49:0/957240605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f431c068190 0x7f431c0685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 -- 172.21.15.49:0/957240605 >> 172.21.15.49:0/957240605 conn(0x7f431c075790 msgr2=0x7f431c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:13.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 -- 172.21.15.49:0/957240605 shutdown_connections 2023-10-08T09:14:13.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 -- 172.21.15.49:0/957240605 wait complete. 2023-10-08T09:14:13.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 Processor -- start 2023-10-08T09:14:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 -- start start 2023-10-08T09:14:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068190 0x7f431c0ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c068b30 0x7f431c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f431c1067a0 0x7f431c1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:13.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a50c0 con 0x7f431c068190 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a5230 con 0x7f431c068b30 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.091+0000 7f432136d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a53f0 con 0x7f431c1067a0 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068190 0x7f431c0ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068190 0x7f431c0ff820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48860/0 (socket says 172.21.15.49:48860) 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 -- 172.21.15.49:0/3204856428 learned_addr learned my addr 172.21.15.49:0/3204856428 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431b7fe700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f431c1067a0 0x7f431c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f4313fff700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c068b30 0x7f431c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 -- 172.21.15.49:0/3204856428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f431c1067a0 msgr2=0x7f431c1002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f431c1067a0 0x7f431c1002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 -- 172.21.15.49:0/3204856428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c068b30 msgr2=0x7f431c0ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:13.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c068b30 0x7f431c0ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 -- 172.21.15.49:0/3204856428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009d00 con 0x7f431c068190 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431b7fe700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f431c1067a0 0x7f431c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f431affd700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068190 0x7f431c0ff820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f430c00d980 tx=0x7f430c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f4318ff9700 1 -- 172.21.15.49:0/3204856428 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c004220 con 0x7f431c068190 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f4318ff9700 1 -- 172.21.15.49:0/3204856428 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f430c007640 con 0x7f431c068190 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c1a56d0 con 0x7f431c068190 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f4318ff9700 1 -- 172.21.15.49:0/3204856428 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c005690 con 0x7f431c068190 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.092+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1a5bd0 con 0x7f431c068190 2023-10-08T09:14:13.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.093+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c108d50 con 0x7f431c068190 2023-10-08T09:14:13.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.095+0000 7f4318ff9700 1 -- 172.21.15.49:0/3204856428 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f430c004750 con 0x7f431c068190 2023-10-08T09:14:13.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.095+0000 7f4318ff9700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43080a1510 0x7f43080a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:13.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.095+0000 7f4313fff700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43080a1510 0x7f43080a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:13.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.095+0000 7f4318ff9700 1 -- 172.21.15.49:0/3204856428 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f430c0cd420 con 0x7f431c068190 2023-10-08T09:14:13.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.096+0000 7f4313fff700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43080a1510 0x7f43080a39d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f431c100f50 tx=0x7f4304009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:13.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.099+0000 7f4318ff9700 1 -- 172.21.15.49:0/3204856428 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f430c096610 con 0x7f431c068190 2023-10-08T09:14:13.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.245+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff8a8073fd0 con 0x7ff8a8101260 2023-10-08T09:14:13.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.245+0000 7ff89e7fc700 1 -- 172.21.15.49:0/2653283677 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8a409b570 con 0x7ff8a8101260 2023-10-08T09:14:13.247 INFO:teuthology.orchestra.run.smithi049.stdout:137438953526 2023-10-08T09:14:13.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff8940a1510 msgr2=0x7ff8940a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff8940a1510 0x7ff8940a39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8101930 tx=0x7ff89800a3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101260 msgr2=0x7ff8a8073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101260 0x7ff8a8073030 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a400f700 comp rx=0 tx=0).stop 2023-10-08T09:14:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 shutdown_connections 2023-10-08T09:14:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff8940a1510 0x7ff8940a39d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8a81008a0 0x7ff8a8072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8a81057c0 0x7ff8a8073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 --2- 172.21.15.49:0/2653283677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8a8101260 0x7ff8a8073030 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 >> 172.21.15.49:0/2653283677 conn(0x7ff8a80779e0 msgr2=0x7ff8a80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 shutdown_connections 2023-10-08T09:14:13.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.249+0000 7ff8afa0b700 1 -- 172.21.15.49:0/2653283677 wait complete. 2023-10-08T09:14:13.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.380+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f431c101ee0 con 0x7f431c068190 2023-10-08T09:14:13.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.381+0000 7f4318ff9700 1 -- 172.21.15.49:0/3204856428 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f430c016090 con 0x7f431c068190 2023-10-08T09:14:13.382 INFO:teuthology.orchestra.run.smithi049.stdout:107374182460 2023-10-08T09:14:13.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43080a1510 msgr2=0x7f43080a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:13.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43080a1510 0x7f43080a39d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f431c100f50 tx=0x7f4304009260 comp rx=0 tx=0).stop 2023-10-08T09:14:13.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068190 msgr2=0x7f431c0ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:13.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068190 0x7f431c0ff820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f430c00d980 tx=0x7f430c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:14:13.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 shutdown_connections 2023-10-08T09:14:13.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f43080a1510 0x7f43080a39d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f431c1067a0 0x7f431c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f431c068190 0x7f431c0ff820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 --2- 172.21.15.49:0/3204856428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f431c068b30 0x7f431c0ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:13.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 >> 172.21.15.49:0/3204856428 conn(0x7f431c075790 msgr2=0x7f431c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:13.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 shutdown_connections 2023-10-08T09:14:13.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:13.383+0000 7f432136d700 1 -- 172.21.15.49:0/3204856428 wait complete. 2023-10-08T09:14:13.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:13 smithi049 ceph-mon[99485]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T09:14:13.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:13 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2653283677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T09:14:13.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:13 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3204856428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T09:14:13.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953524 got 137438953526 for osd.4 2023-10-08T09:14:13.749 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:13 smithi132 ceph-mon[103389]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T09:14:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:13 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2653283677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T09:14:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:13 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3204856428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T09:14:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:13 smithi184 ceph-mon[103675]: pgmap v306: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T09:14:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:13 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2653283677' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-08T09:14:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:13 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3204856428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-08T09:14:14.184 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182457 got 107374182460 for osd.3 2023-10-08T09:14:14.185 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:14.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 -- 172.21.15.49:0/965141855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0100840 msgr2=0x7f0de0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 --2- 172.21.15.49:0/965141855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0100840 0x7f0de0100cc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009990 tx=0x7f0dc8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 -- 172.21.15.49:0/965141855 shutdown_connections 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 --2- 172.21.15.49:0/965141855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0101200 0x7f0de010d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 --2- 172.21.15.49:0/965141855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0de01057a0 0x7f0de0105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 --2- 172.21.15.49:0/965141855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0100840 0x7f0de0100cc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 -- 172.21.15.49:0/965141855 >> 172.21.15.49:0/965141855 conn(0x7f0de0077950 msgr2=0x7f0de0077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 -- 172.21.15.49:0/965141855 shutdown_connections 2023-10-08T09:14:14.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.483+0000 7f0de5687700 1 -- 172.21.15.49:0/965141855 wait complete. 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 Processor -- start 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 -- start start 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0100840 0x7f0de0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0101200 0x7f0de0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0de01057a0 0x7f0de0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de01a4fd0 con 0x7f0de0101200 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de01a5140 con 0x7f0de01057a0 2023-10-08T09:14:14.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0de5687700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de01a5300 con 0x7f0de0100840 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0dde7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0101200 0x7f0de0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0dde7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0101200 0x7f0de0073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48874/0 (socket says 172.21.15.49:48874) 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0dde7fc700 1 -- 172.21.15.49:0/2098321757 learned_addr learned my addr 172.21.15.49:0/2098321757 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0ddf7fe700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0de01057a0 0x7f0de0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.484+0000 7f0ddeffd700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0100840 0x7f0de0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dde7fc700 1 -- 172.21.15.49:0/2098321757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0100840 msgr2=0x7f0de0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dde7fc700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0100840 0x7f0de0072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dde7fc700 1 -- 172.21.15.49:0/2098321757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0de01057a0 msgr2=0x7f0de0073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dde7fc700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0de01057a0 0x7f0de0073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dde7fc700 1 -- 172.21.15.49:0/2098321757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc80095f0 con 0x7f0de0101200 2023-10-08T09:14:14.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dde7fc700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0101200 0x7f0de0073030 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8000c00 tx=0x7f0dc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dd7fff700 1 -- 172.21.15.49:0/2098321757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc801d070 con 0x7f0de0101200 2023-10-08T09:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de01a5580 con 0x7f0de0101200 2023-10-08T09:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dd7fff700 1 -- 172.21.15.49:0/2098321757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dc8022470 con 0x7f0de0101200 2023-10-08T09:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0dd7fff700 1 -- 172.21.15.49:0/2098321757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8017650 con 0x7f0de0101200 2023-10-08T09:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.485+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de01a59a0 con 0x7f0de0101200 2023-10-08T09:14:14.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.486+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de0108d80 con 0x7f0de0101200 2023-10-08T09:14:14.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.488+0000 7f0dd7fff700 1 -- 172.21.15.49:0/2098321757 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0dc80225e0 con 0x7f0de0101200 2023-10-08T09:14:14.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.489+0000 7f0dd7fff700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0dcc0a1510 0x7f0dcc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:14.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.489+0000 7f0dd7fff700 1 -- 172.21.15.49:0/2098321757 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0dc80cec80 con 0x7f0de0101200 2023-10-08T09:14:14.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.489+0000 7f0ddeffd700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0dcc0a1510 0x7f0dcc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:14.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.489+0000 7f0ddeffd700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0dcc0a1510 0x7f0dcc0a39d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000f540 tx=0x7f0dd000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:14.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.492+0000 7f0dd7fff700 1 -- 172.21.15.49:0/2098321757 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0dc8097ea0 con 0x7f0de0101200 2023-10-08T09:14:14.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.716+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f0de0073fd0 con 0x7f0de0101200 2023-10-08T09:14:14.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.716+0000 7f0dd7fff700 1 -- 172.21.15.49:0/2098321757 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f0dc809b4c0 con 0x7f0de0101200 2023-10-08T09:14:14.718 INFO:teuthology.orchestra.run.smithi049.stdout:326417514522 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0dcc0a1510 msgr2=0x7f0dcc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0dcc0a1510 0x7f0dcc0a39d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000f540 tx=0x7f0dd000b040 comp rx=0 tx=0).stop 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0101200 msgr2=0x7f0de0073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0101200 0x7f0de0073030 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8000c00 tx=0x7f0dc8004730 comp rx=0 tx=0).stop 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 shutdown_connections 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0dcc0a1510 0x7f0dcc0a39d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0de0100840 0x7f0de0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0de01057a0 0x7f0de0073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 --2- 172.21.15.49:0/2098321757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0de0101200 0x7f0de0073030 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:14.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.718+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 >> 172.21.15.49:0/2098321757 conn(0x7f0de0077950 msgr2=0x7f0de00fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:14.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.719+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 shutdown_connections 2023-10-08T09:14:14.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:14.719+0000 7f0de5687700 1 -- 172.21.15.49:0/2098321757 wait complete. 2023-10-08T09:14:15.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514519 got 326417514522 for osd.11 2023-10-08T09:14:15.059 DEBUG:teuthology.parallel:result is None 2023-10-08T09:14:15.059 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-08T09:14:15.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-08T09:14:15.635 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:15 smithi049 ceph-mon[99485]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T09:14:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:15 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2098321757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T09:14:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:14:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:14:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:15 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:15 smithi184 ceph-mon[103675]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T09:14:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:15 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2098321757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T09:14:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:14:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:14:15.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:15 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:15 smithi132 ceph-mon[103389]: pgmap v307: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 8 op/s 2023-10-08T09:14:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:15 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2098321757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-08T09:14:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:14:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:14:16.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:15 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:14:16.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 -- 172.21.15.49:0/2547580858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 msgr2=0x7f197810d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 --2- 172.21.15.49:0/2547580858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f197810d670 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f197000b0a0 tx=0x7f197000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 -- 172.21.15.49:0/2547580858 shutdown_connections 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 --2- 172.21.15.49:0/2547580858 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f197810d670 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 --2- 172.21.15.49:0/2547580858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19781067a0 0x7f1978106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 --2- 172.21.15.49:0/2547580858 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1978068190 0x7f19780685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 -- 172.21.15.49:0/2547580858 >> 172.21.15.49:0/2547580858 conn(0x7f1978075790 msgr2=0x7f1978075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 -- 172.21.15.49:0/2547580858 shutdown_connections 2023-10-08T09:14:16.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.939+0000 7f197ecfa700 1 -- 172.21.15.49:0/2547580858 wait complete. 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 Processor -- start 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 -- start start 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1978068190 0x7f19780ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f19780ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19781067a0 0x7f19781002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a5100 con 0x7f1978068b30 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a5270 con 0x7f19781067a0 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ecfa700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a5430 con 0x7f1978068190 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f19780ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:16.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f19780ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48888/0 (socket says 172.21.15.49:48888) 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 -- 172.21.15.49:0/3779943230 learned_addr learned my addr 172.21.15.49:0/3779943230 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197ca96700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1978068190 0x7f19780ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f197d297700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19781067a0 0x7f19781002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 -- 172.21.15.49:0/3779943230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1978068190 msgr2=0x7f19780ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1978068190 0x7f19780ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 -- 172.21.15.49:0/3779943230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19781067a0 msgr2=0x7f19781002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19781067a0 0x7f19781002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 -- 172.21.15.49:0/3779943230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970009d00 con 0x7f1978068b30 2023-10-08T09:14:16.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.940+0000 7f1977fff700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f19780ffd60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f196400c9b0 tx=0x7f196400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.941+0000 7f1975ffb700 1 -- 172.21.15.49:0/3779943230 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964007b00 con 0x7f1978068b30 2023-10-08T09:14:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.941+0000 7f1975ffb700 1 -- 172.21.15.49:0/3779943230 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1964007c60 con 0x7f1978068b30 2023-10-08T09:14:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.941+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19781a5710 con 0x7f1978068b30 2023-10-08T09:14:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.941+0000 7f1975ffb700 1 -- 172.21.15.49:0/3779943230 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964018660 con 0x7f1978068b30 2023-10-08T09:14:16.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.941+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781a5c10 con 0x7f1978068b30 2023-10-08T09:14:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.942+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1978108d50 con 0x7f1978068b30 2023-10-08T09:14:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.944+0000 7f1975ffb700 1 -- 172.21.15.49:0/3779943230 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f196400f450 con 0x7f1978068b30 2023-10-08T09:14:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.944+0000 7f1975ffb700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f19680a1510 0x7f19680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.944+0000 7f1975ffb700 1 -- 172.21.15.49:0/3779943230 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f19640cd440 con 0x7f1978068b30 2023-10-08T09:14:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.944+0000 7f197ca96700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f19680a1510 0x7f19680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:16.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.945+0000 7f197ca96700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f19680a1510 0x7f19680a39d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f196c009a10 tx=0x7f196c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:16.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:16.948+0000 7f1975ffb700 1 -- 172.21.15.49:0/3779943230 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f19640966c0 con 0x7f1978068b30 2023-10-08T09:14:17.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.142+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1978100cb0 con 0x7f19680a1510 2023-10-08T09:14:17.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.158+0000 7f1975ffb700 1 -- 172.21.15.49:0/3779943230 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426735 (secure 0 0 0) 0x7f1978100cb0 con 0x7f19680a1510 2023-10-08T09:14:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f19680a1510 msgr2=0x7f19680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f19680a1510 0x7f19680a39d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f196c009a10 tx=0x7f196c009190 comp rx=0 tx=0).stop 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 msgr2=0x7f19780ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f19780ffd60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f196400c9b0 tx=0x7f196400ccc0 comp rx=0 tx=0).stop 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 shutdown_connections 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f19680a1510 0x7f19680a39d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1978068190 0x7f19780ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:17.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19781067a0 0x7f19781002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 --2- 172.21.15.49:0/3779943230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1978068b30 0x7f19780ffd60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.161+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 >> 172.21.15.49:0/3779943230 conn(0x7f1978075790 msgr2=0x7f19780fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.162+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 shutdown_connections 2023-10-08T09:14:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:17.162+0000 7f197ecfa700 1 -- 172.21.15.49:0/3779943230 wait complete. 2023-10-08T09:14:17.169 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-10-08T09:14:17.601 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-10-08T09:14:15.871719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656399,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":590,"num_read_kb":412,"num_write":418,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180859,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336164,"kb_used_data":11900,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605660,"statfs":{"total":1151940427776,"available":1151596195840,"internally_reserved":0,"allocated":12185600,"data_stored":7793088,"data_compressed":319707,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":74,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003593"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441045+0000","last_change":"2023-10-08T09:13:02.831436+0000","last_active":"2023-10-08T09:13:36.441045+0000","last_peered":"2023-10-08T09:13:36.441045+0000","last_clean":"2023-10-08T09:13:36.441045+0000","last_became_active":"2023-10-08T09:13:02.831123+0000","last_became_peered":"2023-10-08T09:13:02.831123+0000","last_unstale":"2023-10-08T09:13:36.441045+0000","last_undegraded":"2023-10-08T09:13:36.441045+0000","last_fullsized":"2023-10-08T09:13:36.441045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:33.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437087+0000","last_change":"2023-10-08T09:13:02.832286+0000","last_active":"2023-10-08T09:13:36.437087+0000","last_peered":"2023-10-08T09:13:36.437087+0000","last_clean":"2023-10-08T09:13:36.437087+0000","last_became_active":"2023-10-08T09:13:02.831881+0000","last_became_peered":"2023-10-08T09:13:02.831881+0000","last_unstale":"2023-10-08T09:13:36.437087+0000","last_undegraded":"2023-10-08T09:13:36.437087+0000","last_fullsized":"2023-10-08T09:13:36.437087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:31.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436447+0000","last_change":"2023-10-08T09:13:02.831454+0000","last_active":"2023-10-08T09:13:36.436447+0000","last_peered":"2023-10-08T09:13:36.436447+0000","last_clean":"2023-10-08T09:13:36.436447+0000","last_became_active":"2023-10-08T09:13:02.831040+0000","last_became_peered":"2023-10-08T09:13:02.831040+0000","last_unstale":"2023-10-08T09:13:36.436447+0000","last_undegraded":"2023-10-08T09:13:36.436447+0000","last_fullsized":"2023-10-08T09:13:36.436447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:16:02.788279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439965+0000","last_change":"2023-10-08T09:13:02.832699+0000","last_active":"2023-10-08T09:13:36.439965+0000","last_peered":"2023-10-08T09:13:36.439965+0000","last_clean":"2023-10-08T09:13:36.439965+0000","last_became_active":"2023-10-08T09:13:02.832345+0000","last_became_peered":"2023-10-08T09:13:02.832345+0000","last_unstale":"2023-10-08T09:13:36.439965+0000","last_undegraded":"2023-10-08T09:13:36.439965+0000","last_fullsized":"2023-10-08T09:13:36.439965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:01.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436354+0000","last_change":"2023-10-08T09:13:02.827767+0000","last_active":"2023-10-08T09:13:36.436354+0000","last_peered":"2023-10-08T09:13:36.436354+0000","last_clean":"2023-10-08T09:13:36.436354+0000","last_became_active":"2023-10-08T09:13:02.827454+0000","last_became_peered":"2023-10-08T09:13:02.827454+0000","last_unstale":"2023-10-08T09:13:36.436354+0000","last_undegraded":"2023-10-08T09:13:36.436354+0000","last_fullsized":"2023-10-08T09:13:36.436354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:43.122347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.452701+0000","last_change":"2023-10-08T09:13:37.452701+0000","last_active":"2023-10-08T09:13:37.452701+0000","last_peered":"2023-10-08T09:13:37.452701+0000","last_clean":"2023-10-08T09:13:37.452701+0000","last_became_active":"2023-10-08T09:13:37.452261+0000","last_became_peered":"2023-10-08T09:13:37.452261+0000","last_unstale":"2023-10-08T09:13:37.452701+0000","last_undegraded":"2023-10-08T09:13:37.452701+0000","last_fullsized":"2023-10-08T09:13:37.452701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:56.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436726+0000","last_change":"2023-10-08T09:13:02.831765+0000","last_active":"2023-10-08T09:13:36.436726+0000","last_peered":"2023-10-08T09:13:36.436726+0000","last_clean":"2023-10-08T09:13:36.436726+0000","last_became_active":"2023-10-08T09:13:02.831446+0000","last_became_peered":"2023-10-08T09:13:02.831446+0000","last_unstale":"2023-10-08T09:13:36.436726+0000","last_undegraded":"2023-10-08T09:13:36.436726+0000","last_fullsized":"2023-10-08T09:13:36.436726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:41.463161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441582+0000","last_change":"2023-10-08T09:13:02.817586+0000","last_active":"2023-10-08T09:13:36.441582+0000","last_peered":"2023-10-08T09:13:36.441582+0000","last_clean":"2023-10-08T09:13:36.441582+0000","last_became_active":"2023-10-08T09:13:02.817231+0000","last_became_peered":"2023-10-08T09:13:02.817231+0000","last_unstale":"2023-10-08T09:13:36.441582+0000","last_undegraded":"2023-10-08T09:13:36.441582+0000","last_fullsized":"2023-10-08T09:13:36.441582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:23.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443527+0000","last_change":"2023-10-08T09:13:02.831925+0000","last_active":"2023-10-08T09:13:36.443527+0000","last_peered":"2023-10-08T09:13:36.443527+0000","last_clean":"2023-10-08T09:13:36.443527+0000","last_became_active":"2023-10-08T09:13:02.831625+0000","last_became_peered":"2023-10-08T09:13:02.831625+0000","last_unstale":"2023-10-08T09:13:36.443527+0000","last_undegraded":"2023-10-08T09:13:36.443527+0000","last_fullsized":"2023-10-08T09:13:36.443527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:39.115740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443908+0000","last_change":"2023-10-08T09:13:02.831752+0000","last_active":"2023-10-08T09:13:36.443908+0000","last_peered":"2023-10-08T09:13:36.443908+0000","last_clean":"2023-10-08T09:13:36.443908+0000","last_became_active":"2023-10-08T09:13:02.831468+0000","last_became_peered":"2023-10-08T09:13:02.831468+0000","last_unstale":"2023-10-08T09:13:36.443908+0000","last_undegraded":"2023-10-08T09:13:36.443908+0000","last_fullsized":"2023-10-08T09:13:36.443908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:05.758017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437153+0000","last_change":"2023-10-08T09:13:02.828838+0000","last_active":"2023-10-08T09:13:36.437153+0000","last_peered":"2023-10-08T09:13:36.437153+0000","last_clean":"2023-10-08T09:13:36.437153+0000","last_became_active":"2023-10-08T09:13:02.828598+0000","last_became_peered":"2023-10-08T09:13:02.828598+0000","last_unstale":"2023-10-08T09:13:36.437153+0000","last_undegraded":"2023-10-08T09:13:36.437153+0000","last_fullsized":"2023-10-08T09:13:36.437153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:04.317649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.450898+0000","last_change":"2023-10-08T09:13:37.450251+0000","last_active":"2023-10-08T09:13:37.450898+0000","last_peered":"2023-10-08T09:13:37.450898+0000","last_clean":"2023-10-08T09:13:37.450898+0000","last_became_active":"2023-10-08T09:13:37.449848+0000","last_became_peered":"2023-10-08T09:13:37.449848+0000","last_unstale":"2023-10-08T09:13:37.450898+0000","last_undegraded":"2023-10-08T09:13:37.450898+0000","last_fullsized":"2023-10-08T09:13:37.450898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:09:16.541785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.450381+0000","last_change":"2023-10-08T09:13:37.450381+0000","last_active":"2023-10-08T09:13:37.450381+0000","last_peered":"2023-10-08T09:13:37.450381+0000","last_clean":"2023-10-08T09:13:37.450381+0000","last_became_active":"2023-10-08T09:13:37.450044+0000","last_became_peered":"2023-10-08T09:13:37.450044+0000","last_unstale":"2023-10-08T09:13:37.450381+0000","last_undegraded":"2023-10-08T09:13:37.450381+0000","last_fullsized":"2023-10-08T09:13:37.450381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:03.203894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436269+0000","last_change":"2023-10-08T09:13:02.829027+0000","last_active":"2023-10-08T09:13:36.436269+0000","last_peered":"2023-10-08T09:13:36.436269+0000","last_clean":"2023-10-08T09:13:36.436269+0000","last_became_active":"2023-10-08T09:13:02.828714+0000","last_became_peered":"2023-10-08T09:13:02.828714+0000","last_unstale":"2023-10-08T09:13:36.436269+0000","last_undegraded":"2023-10-08T09:13:36.436269+0000","last_fullsized":"2023-10-08T09:13:36.436269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:13.121274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436239+0000","last_change":"2023-10-08T09:13:02.838640+0000","last_active":"2023-10-08T09:13:36.436239+0000","last_peered":"2023-10-08T09:13:36.436239+0000","last_clean":"2023-10-08T09:13:36.436239+0000","last_became_active":"2023-10-08T09:13:02.838302+0000","last_became_peered":"2023-10-08T09:13:02.838302+0000","last_unstale":"2023-10-08T09:13:36.436239+0000","last_undegraded":"2023-10-08T09:13:36.436239+0000","last_fullsized":"2023-10-08T09:13:36.436239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:54.268315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441384+0000","last_change":"2023-10-08T09:13:02.834027+0000","last_active":"2023-10-08T09:13:36.441384+0000","last_peered":"2023-10-08T09:13:36.441384+0000","last_clean":"2023-10-08T09:13:36.441384+0000","last_became_active":"2023-10-08T09:13:02.833733+0000","last_became_peered":"2023-10-08T09:13:02.833733+0000","last_unstale":"2023-10-08T09:13:36.441384+0000","last_undegraded":"2023-10-08T09:13:36.441384+0000","last_fullsized":"2023-10-08T09:13:36.441384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:03.644640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440991+0000","last_change":"2023-10-08T09:13:02.832040+0000","last_active":"2023-10-08T09:13:36.440991+0000","last_peered":"2023-10-08T09:13:36.440991+0000","last_clean":"2023-10-08T09:13:36.440991+0000","last_became_active":"2023-10-08T09:13:02.831752+0000","last_became_peered":"2023-10-08T09:13:02.831752+0000","last_unstale":"2023-10-08T09:13:36.440991+0000","last_undegraded":"2023-10-08T09:13:36.440991+0000","last_fullsized":"2023-10-08T09:13:36.440991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:38.938703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440912+0000","last_change":"2023-10-08T09:13:02.830535+0000","last_active":"2023-10-08T09:13:36.440912+0000","last_peered":"2023-10-08T09:13:36.440912+0000","last_clean":"2023-10-08T09:13:36.440912+0000","last_became_active":"2023-10-08T09:13:02.830144+0000","last_became_peered":"2023-10-08T09:13:02.830144+0000","last_unstale":"2023-10-08T09:13:36.440912+0000","last_undegraded":"2023-10-08T09:13:36.440912+0000","last_fullsized":"2023-10-08T09:13:36.440912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:33.850815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443550+0000","last_change":"2023-10-08T09:13:02.830938+0000","last_active":"2023-10-08T09:13:36.443550+0000","last_peered":"2023-10-08T09:13:36.443550+0000","last_clean":"2023-10-08T09:13:36.443550+0000","last_became_active":"2023-10-08T09:13:02.830551+0000","last_became_peered":"2023-10-08T09:13:02.830551+0000","last_unstale":"2023-10-08T09:13:36.443550+0000","last_undegraded":"2023-10-08T09:13:36.443550+0000","last_fullsized":"2023-10-08T09:13:36.443550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:26.894614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442755+0000","last_change":"2023-10-08T09:13:07.871322+0000","last_active":"2023-10-08T09:13:36.442755+0000","last_peered":"2023-10-08T09:13:36.442755+0000","last_clean":"2023-10-08T09:13:36.442755+0000","last_became_active":"2023-10-08T09:13:07.870828+0000","last_became_peered":"2023-10-08T09:13:07.870828+0000","last_unstale":"2023-10-08T09:13:36.442755+0000","last_undegraded":"2023-10-08T09:13:36.442755+0000","last_fullsized":"2023-10-08T09:13:36.442755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:37.145705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:12.496533+0000","last_change":"2023-10-08T09:13:02.373510+0000","last_active":"2023-10-08T09:14:12.496533+0000","last_peered":"2023-10-08T09:14:12.496533+0000","last_clean":"2023-10-08T09:14:12.496533+0000","last_became_active":"2023-10-08T09:13:02.373174+0000","last_became_peered":"2023-10-08T09:13:02.373174+0000","last_unstale":"2023-10-08T09:14:12.496533+0000","last_undegraded":"2023-10-08T09:14:12.496533+0000","last_fullsized":"2023-10-08T09:14:12.496533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442547+0000","last_change":"2023-10-08T09:13:02.835195+0000","last_active":"2023-10-08T09:13:36.442547+0000","last_peered":"2023-10-08T09:13:36.442547+0000","last_clean":"2023-10-08T09:13:36.442547+0000","last_became_active":"2023-10-08T09:13:02.834893+0000","last_became_peered":"2023-10-08T09:13:02.834893+0000","last_unstale":"2023-10-08T09:13:36.442547+0000","last_undegraded":"2023-10-08T09:13:36.442547+0000","last_fullsized":"2023-10-08T09:13:36.442547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:35.835017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439374+0000","last_change":"2023-10-08T09:13:07.871418+0000","last_active":"2023-10-08T09:13:36.439374+0000","last_peered":"2023-10-08T09:13:36.439374+0000","last_clean":"2023-10-08T09:13:36.439374+0000","last_became_active":"2023-10-08T09:13:07.870961+0000","last_became_peered":"2023-10-08T09:13:07.870961+0000","last_unstale":"2023-10-08T09:13:36.439374+0000","last_undegraded":"2023-10-08T09:13:36.439374+0000","last_fullsized":"2023-10-08T09:13:36.439374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:31.723498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442572+0000","last_change":"2023-10-08T09:13:02.370728+0000","last_active":"2023-10-08T09:13:36.442572+0000","last_peered":"2023-10-08T09:13:36.442572+0000","last_clean":"2023-10-08T09:13:36.442572+0000","last_became_active":"2023-10-08T09:13:02.370396+0000","last_became_peered":"2023-10-08T09:13:02.370396+0000","last_unstale":"2023-10-08T09:13:36.442572+0000","last_undegraded":"2023-10-08T09:13:36.442572+0000","last_fullsized":"2023-10-08T09:13:36.442572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:32.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436716+0000","last_change":"2023-10-08T09:13:02.827823+0000","last_active":"2023-10-08T09:13:36.436716+0000","last_peered":"2023-10-08T09:13:36.436716+0000","last_clean":"2023-10-08T09:13:36.436716+0000","last_became_active":"2023-10-08T09:13:02.827417+0000","last_became_peered":"2023-10-08T09:13:02.827417+0000","last_unstale":"2023-10-08T09:13:36.436716+0000","last_undegraded":"2023-10-08T09:13:36.436716+0000","last_fullsized":"2023-10-08T09:13:36.436716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:38.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.455325+0000","last_change":"2023-10-08T09:13:37.455325+0000","last_active":"2023-10-08T09:13:37.455325+0000","last_peered":"2023-10-08T09:13:37.455325+0000","last_clean":"2023-10-08T09:13:37.455325+0000","last_became_active":"2023-10-08T09:13:37.454985+0000","last_became_peered":"2023-10-08T09:13:37.454985+0000","last_unstale":"2023-10-08T09:13:37.455325+0000","last_undegraded":"2023-10-08T09:13:37.455325+0000","last_fullsized":"2023-10-08T09:13:37.455325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:13.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436867+0000","last_change":"2023-10-08T09:13:02.375244+0000","last_active":"2023-10-08T09:13:36.436867+0000","last_peered":"2023-10-08T09:13:36.436867+0000","last_clean":"2023-10-08T09:13:36.436867+0000","last_became_active":"2023-10-08T09:13:02.374526+0000","last_became_peered":"2023-10-08T09:13:02.374526+0000","last_unstale":"2023-10-08T09:13:36.436867+0000","last_undegraded":"2023-10-08T09:13:36.436867+0000","last_fullsized":"2023-10-08T09:13:36.436867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:36.380491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443641+0000","last_change":"2023-10-08T09:13:02.838373+0000","last_active":"2023-10-08T09:13:36.443641+0000","last_peered":"2023-10-08T09:13:36.443641+0000","last_clean":"2023-10-08T09:13:36.443641+0000","last_became_active":"2023-10-08T09:13:02.838100+0000","last_became_peered":"2023-10-08T09:13:02.838100+0000","last_unstale":"2023-10-08T09:13:36.443641+0000","last_undegraded":"2023-10-08T09:13:36.443641+0000","last_fullsized":"2023-10-08T09:13:36.443641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:59.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439494+0000","last_change":"2023-10-08T09:13:07.868929+0000","last_active":"2023-10-08T09:13:36.439494+0000","last_peered":"2023-10-08T09:13:36.439494+0000","last_clean":"2023-10-08T09:13:36.439494+0000","last_became_active":"2023-10-08T09:13:07.868451+0000","last_became_peered":"2023-10-08T09:13:07.868451+0000","last_unstale":"2023-10-08T09:13:36.439494+0000","last_undegraded":"2023-10-08T09:13:36.439494+0000","last_fullsized":"2023-10-08T09:13:36.439494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:36.540319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436511+0000","last_change":"2023-10-08T09:13:02.366404+0000","last_active":"2023-10-08T09:13:36.436511+0000","last_peered":"2023-10-08T09:13:36.436511+0000","last_clean":"2023-10-08T09:13:36.436511+0000","last_became_active":"2023-10-08T09:13:02.366025+0000","last_became_peered":"2023-10-08T09:13:02.366025+0000","last_unstale":"2023-10-08T09:13:36.436511+0000","last_undegraded":"2023-10-08T09:13:36.436511+0000","last_fullsized":"2023-10-08T09:13:36.436511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:51.916565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443379+0000","last_change":"2023-10-08T09:13:02.832852+0000","last_active":"2023-10-08T09:13:36.443379+0000","last_peered":"2023-10-08T09:13:36.443379+0000","last_clean":"2023-10-08T09:13:36.443379+0000","last_became_active":"2023-10-08T09:13:02.832530+0000","last_became_peered":"2023-10-08T09:13:02.832530+0000","last_unstale":"2023-10-08T09:13:36.443379+0000","last_undegraded":"2023-10-08T09:13:36.443379+0000","last_fullsized":"2023-10-08T09:13:36.443379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:20.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443286+0000","last_change":"2023-10-08T09:13:07.863864+0000","last_active":"2023-10-08T09:13:36.443286+0000","last_peered":"2023-10-08T09:13:36.443286+0000","last_clean":"2023-10-08T09:13:36.443286+0000","last_became_active":"2023-10-08T09:13:07.863509+0000","last_became_peered":"2023-10-08T09:13:07.863509+0000","last_unstale":"2023-10-08T09:13:36.443286+0000","last_undegraded":"2023-10-08T09:13:36.443286+0000","last_fullsized":"2023-10-08T09:13:36.443286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:22.933327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:12.473899+0000","last_change":"2023-10-08T09:13:02.374215+0000","last_active":"2023-10-08T09:14:12.473899+0000","last_peered":"2023-10-08T09:14:12.473899+0000","last_clean":"2023-10-08T09:14:12.473899+0000","last_became_active":"2023-10-08T09:13:02.373912+0000","last_became_peered":"2023-10-08T09:13:02.373912+0000","last_unstale":"2023-10-08T09:14:12.473899+0000","last_undegraded":"2023-10-08T09:14:12.473899+0000","last_fullsized":"2023-10-08T09:14:12.473899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:18.315244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436499+0000","last_change":"2023-10-08T09:13:02.830651+0000","last_active":"2023-10-08T09:13:36.436499+0000","last_peered":"2023-10-08T09:13:36.436499+0000","last_clean":"2023-10-08T09:13:36.436499+0000","last_became_active":"2023-10-08T09:13:02.830324+0000","last_became_peered":"2023-10-08T09:13:02.830324+0000","last_unstale":"2023-10-08T09:13:36.436499+0000","last_undegraded":"2023-10-08T09:13:36.436499+0000","last_fullsized":"2023-10-08T09:13:36.436499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:02.143794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.454054+0000","last_change":"2023-10-08T09:13:37.454054+0000","last_active":"2023-10-08T09:13:37.454054+0000","last_peered":"2023-10-08T09:13:37.454054+0000","last_clean":"2023-10-08T09:13:37.454054+0000","last_became_active":"2023-10-08T09:13:37.453482+0000","last_became_peered":"2023-10-08T09:13:37.453482+0000","last_unstale":"2023-10-08T09:13:37.454054+0000","last_undegraded":"2023-10-08T09:13:37.454054+0000","last_fullsized":"2023-10-08T09:13:37.454054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:31.550837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443108+0000","last_change":"2023-10-08T09:13:02.370334+0000","last_active":"2023-10-08T09:13:36.443108+0000","last_peered":"2023-10-08T09:13:36.443108+0000","last_clean":"2023-10-08T09:13:36.443108+0000","last_became_active":"2023-10-08T09:13:02.370068+0000","last_became_peered":"2023-10-08T09:13:02.370068+0000","last_unstale":"2023-10-08T09:13:36.443108+0000","last_undegraded":"2023-10-08T09:13:36.443108+0000","last_fullsized":"2023-10-08T09:13:36.443108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.962779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445562+0000","last_change":"2023-10-08T09:13:02.832346+0000","last_active":"2023-10-08T09:13:36.445562+0000","last_peered":"2023-10-08T09:13:36.445562+0000","last_clean":"2023-10-08T09:13:36.445562+0000","last_became_active":"2023-10-08T09:13:02.832059+0000","last_became_peered":"2023-10-08T09:13:02.832059+0000","last_unstale":"2023-10-08T09:13:36.445562+0000","last_undegraded":"2023-10-08T09:13:36.445562+0000","last_fullsized":"2023-10-08T09:13:36.445562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:30.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440849+0000","last_change":"2023-10-08T09:13:07.863466+0000","last_active":"2023-10-08T09:13:36.440849+0000","last_peered":"2023-10-08T09:13:36.440849+0000","last_clean":"2023-10-08T09:13:36.440849+0000","last_became_active":"2023-10-08T09:13:07.863015+0000","last_became_peered":"2023-10-08T09:13:07.863015+0000","last_unstale":"2023-10-08T09:13:36.440849+0000","last_undegraded":"2023-10-08T09:13:36.440849+0000","last_fullsized":"2023-10-08T09:13:36.440849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:30.316625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445901+0000","last_change":"2023-10-08T09:13:02.369444+0000","last_active":"2023-10-08T09:13:36.445901+0000","last_peered":"2023-10-08T09:13:36.445901+0000","last_clean":"2023-10-08T09:13:36.445901+0000","last_became_active":"2023-10-08T09:13:02.369142+0000","last_became_peered":"2023-10-08T09:13:02.369142+0000","last_unstale":"2023-10-08T09:13:36.445901+0000","last_undegraded":"2023-10-08T09:13:36.445901+0000","last_fullsized":"2023-10-08T09:13:36.445901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:28.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441755+0000","last_change":"2023-10-08T09:13:02.836125+0000","last_active":"2023-10-08T09:13:36.441755+0000","last_peered":"2023-10-08T09:13:36.441755+0000","last_clean":"2023-10-08T09:13:36.441755+0000","last_became_active":"2023-10-08T09:13:02.835809+0000","last_became_peered":"2023-10-08T09:13:02.835809+0000","last_unstale":"2023-10-08T09:13:36.441755+0000","last_undegraded":"2023-10-08T09:13:36.441755+0000","last_fullsized":"2023-10-08T09:13:36.441755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:15.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436500+0000","last_change":"2023-10-08T09:13:07.861633+0000","last_active":"2023-10-08T09:13:36.436500+0000","last_peered":"2023-10-08T09:13:36.436500+0000","last_clean":"2023-10-08T09:13:36.436500+0000","last_became_active":"2023-10-08T09:13:07.861168+0000","last_became_peered":"2023-10-08T09:13:07.861168+0000","last_unstale":"2023-10-08T09:13:36.436500+0000","last_undegraded":"2023-10-08T09:13:36.436500+0000","last_fullsized":"2023-10-08T09:13:36.436500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:47.283892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443337+0000","last_change":"2023-10-08T09:13:02.365703+0000","last_active":"2023-10-08T09:13:36.443337+0000","last_peered":"2023-10-08T09:13:36.443337+0000","last_clean":"2023-10-08T09:13:36.443337+0000","last_became_active":"2023-10-08T09:13:02.365421+0000","last_became_peered":"2023-10-08T09:13:02.365421+0000","last_unstale":"2023-10-08T09:13:36.443337+0000","last_undegraded":"2023-10-08T09:13:36.443337+0000","last_fullsized":"2023-10-08T09:13:36.443337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:43.124511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439111+0000","last_change":"2023-10-08T09:13:02.840941+0000","last_active":"2023-10-08T09:13:36.439111+0000","last_peered":"2023-10-08T09:13:36.439111+0000","last_clean":"2023-10-08T09:13:36.439111+0000","last_became_active":"2023-10-08T09:13:02.840605+0000","last_became_peered":"2023-10-08T09:13:02.840605+0000","last_unstale":"2023-10-08T09:13:36.439111+0000","last_undegraded":"2023-10-08T09:13:36.439111+0000","last_fullsized":"2023-10-08T09:13:36.439111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:24:13.417028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442140+0000","last_change":"2023-10-08T09:13:07.868950+0000","last_active":"2023-10-08T09:13:36.442140+0000","last_peered":"2023-10-08T09:13:36.442140+0000","last_clean":"2023-10-08T09:13:36.442140+0000","last_became_active":"2023-10-08T09:13:07.868542+0000","last_became_peered":"2023-10-08T09:13:07.868542+0000","last_unstale":"2023-10-08T09:13:36.442140+0000","last_undegraded":"2023-10-08T09:13:36.442140+0000","last_fullsized":"2023-10-08T09:13:36.442140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:11.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:13.474769+0000","last_change":"2023-10-08T09:13:02.376188+0000","last_active":"2023-10-08T09:14:13.474769+0000","last_peered":"2023-10-08T09:14:13.474769+0000","last_clean":"2023-10-08T09:14:13.474769+0000","last_became_active":"2023-10-08T09:13:02.375914+0000","last_became_peered":"2023-10-08T09:13:02.375914+0000","last_unstale":"2023-10-08T09:14:13.474769+0000","last_undegraded":"2023-10-08T09:14:13.474769+0000","last_fullsized":"2023-10-08T09:14:13.474769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:10.744085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442281+0000","last_change":"2023-10-08T09:13:02.833429+0000","last_active":"2023-10-08T09:13:36.442281+0000","last_peered":"2023-10-08T09:13:36.442281+0000","last_clean":"2023-10-08T09:13:36.442281+0000","last_became_active":"2023-10-08T09:13:02.832792+0000","last_became_peered":"2023-10-08T09:13:02.832792+0000","last_unstale":"2023-10-08T09:13:36.442281+0000","last_undegraded":"2023-10-08T09:13:36.442281+0000","last_fullsized":"2023-10-08T09:13:36.442281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:24.563447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439055+0000","last_change":"2023-10-08T09:13:07.862688+0000","last_active":"2023-10-08T09:13:36.439055+0000","last_peered":"2023-10-08T09:13:36.439055+0000","last_clean":"2023-10-08T09:13:36.439055+0000","last_became_active":"2023-10-08T09:13:07.862197+0000","last_became_peered":"2023-10-08T09:13:07.862197+0000","last_unstale":"2023-10-08T09:13:36.439055+0000","last_undegraded":"2023-10-08T09:13:36.439055+0000","last_fullsized":"2023-10-08T09:13:36.439055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:56.792045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437118+0000","last_change":"2023-10-08T09:13:02.380283+0000","last_active":"2023-10-08T09:13:36.437118+0000","last_peered":"2023-10-08T09:13:36.437118+0000","last_clean":"2023-10-08T09:13:36.437118+0000","last_became_active":"2023-10-08T09:13:02.379844+0000","last_became_peered":"2023-10-08T09:13:02.379844+0000","last_unstale":"2023-10-08T09:13:36.437118+0000","last_undegraded":"2023-10-08T09:13:36.437118+0000","last_fullsized":"2023-10-08T09:13:36.437118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:18.162437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436786+0000","last_change":"2023-10-08T09:13:02.839211+0000","last_active":"2023-10-08T09:13:36.436786+0000","last_peered":"2023-10-08T09:13:36.436786+0000","last_clean":"2023-10-08T09:13:36.436786+0000","last_became_active":"2023-10-08T09:13:02.838968+0000","last_became_peered":"2023-10-08T09:13:02.838968+0000","last_unstale":"2023-10-08T09:13:36.436786+0000","last_undegraded":"2023-10-08T09:13:36.436786+0000","last_fullsized":"2023-10-08T09:13:36.436786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:24:02.541749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"86'1","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.457701+0000","last_change":"2023-10-08T09:13:37.457701+0000","last_active":"2023-10-08T09:13:37.457701+0000","last_peered":"2023-10-08T09:13:37.457701+0000","last_clean":"2023-10-08T09:13:37.457701+0000","last_became_active":"2023-10-08T09:13:37.452309+0000","last_became_peered":"2023-10-08T09:13:37.452309+0000","last_unstale":"2023-10-08T09:13:37.457701+0000","last_undegraded":"2023-10-08T09:13:37.457701+0000","last_fullsized":"2023-10-08T09:13:37.457701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:14.898542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443267+0000","last_change":"2023-10-08T09:13:02.377027+0000","last_active":"2023-10-08T09:13:36.443267+0000","last_peered":"2023-10-08T09:13:36.443267+0000","last_clean":"2023-10-08T09:13:36.443267+0000","last_became_active":"2023-10-08T09:13:02.376746+0000","last_became_peered":"2023-10-08T09:13:02.376746+0000","last_unstale":"2023-10-08T09:13:36.443267+0000","last_undegraded":"2023-10-08T09:13:36.443267+0000","last_fullsized":"2023-10-08T09:13:36.443267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:19.062296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443725+0000","last_change":"2023-10-08T09:13:02.837279+0000","last_active":"2023-10-08T09:13:36.443725+0000","last_peered":"2023-10-08T09:13:36.443725+0000","last_clean":"2023-10-08T09:13:36.443725+0000","last_became_active":"2023-10-08T09:13:02.836966+0000","last_became_peered":"2023-10-08T09:13:02.836966+0000","last_unstale":"2023-10-08T09:13:36.443725+0000","last_undegraded":"2023-10-08T09:13:36.443725+0000","last_fullsized":"2023-10-08T09:13:36.443725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:25.462835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439420+0000","last_change":"2023-10-08T09:13:07.865538+0000","last_active":"2023-10-08T09:13:36.439420+0000","last_peered":"2023-10-08T09:13:36.439420+0000","last_clean":"2023-10-08T09:13:36.439420+0000","last_became_active":"2023-10-08T09:13:07.865118+0000","last_became_peered":"2023-10-08T09:13:07.865118+0000","last_unstale":"2023-10-08T09:13:36.439420+0000","last_undegraded":"2023-10-08T09:13:36.439420+0000","last_fullsized":"2023-10-08T09:13:36.439420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:32.167439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":148,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:13.479496+0000","last_change":"2023-10-08T09:13:02.378173+0000","last_active":"2023-10-08T09:14:13.479496+0000","last_peered":"2023-10-08T09:14:13.479496+0000","last_clean":"2023-10-08T09:14:13.479496+0000","last_became_active":"2023-10-08T09:13:02.377880+0000","last_became_peered":"2023-10-08T09:13:02.377880+0000","last_unstale":"2023-10-08T09:14:13.479496+0000","last_undegraded":"2023-10-08T09:14:13.479496+0000","last_fullsized":"2023-10-08T09:14:13.479496+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58583,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":104,"num_write":36,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436695+0000","last_change":"2023-10-08T09:13:02.829874+0000","last_active":"2023-10-08T09:13:36.436695+0000","last_peered":"2023-10-08T09:13:36.436695+0000","last_clean":"2023-10-08T09:13:36.436695+0000","last_became_active":"2023-10-08T09:13:02.829552+0000","last_became_peered":"2023-10-08T09:13:02.829552+0000","last_unstale":"2023-10-08T09:13:36.436695+0000","last_undegraded":"2023-10-08T09:13:36.436695+0000","last_fullsized":"2023-10-08T09:13:36.436695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:29.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443746+0000","last_change":"2023-10-08T09:13:07.869661+0000","last_active":"2023-10-08T09:13:36.443746+0000","last_peered":"2023-10-08T09:13:36.443746+0000","last_clean":"2023-10-08T09:13:36.443746+0000","last_became_active":"2023-10-08T09:13:07.869236+0000","last_became_peered":"2023-10-08T09:13:07.869236+0000","last_unstale":"2023-10-08T09:13:36.443746+0000","last_undegraded":"2023-10-08T09:13:36.443746+0000","last_fullsized":"2023-10-08T09:13:36.443746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:26.354461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443292+0000","last_change":"2023-10-08T09:13:02.376941+0000","last_active":"2023-10-08T09:13:36.443292+0000","last_peered":"2023-10-08T09:13:36.443292+0000","last_clean":"2023-10-08T09:13:36.443292+0000","last_became_active":"2023-10-08T09:13:02.375810+0000","last_became_peered":"2023-10-08T09:13:02.375810+0000","last_unstale":"2023-10-08T09:13:36.443292+0000","last_undegraded":"2023-10-08T09:13:36.443292+0000","last_fullsized":"2023-10-08T09:13:36.443292+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:17.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445839+0000","last_change":"2023-10-08T09:13:02.836781+0000","last_active":"2023-10-08T09:13:36.445839+0000","last_peered":"2023-10-08T09:13:36.445839+0000","last_clean":"2023-10-08T09:13:36.445839+0000","last_became_active":"2023-10-08T09:13:02.836493+0000","last_became_peered":"2023-10-08T09:13:02.836493+0000","last_unstale":"2023-10-08T09:13:36.445839+0000","last_undegraded":"2023-10-08T09:13:36.445839+0000","last_fullsized":"2023-10-08T09:13:36.445839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:03.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440866+0000","last_change":"2023-10-08T09:13:07.859104+0000","last_active":"2023-10-08T09:13:36.440866+0000","last_peered":"2023-10-08T09:13:36.440866+0000","last_clean":"2023-10-08T09:13:36.440866+0000","last_became_active":"2023-10-08T09:13:07.858691+0000","last_became_peered":"2023-10-08T09:13:07.858691+0000","last_unstale":"2023-10-08T09:13:36.440866+0000","last_undegraded":"2023-10-08T09:13:36.440866+0000","last_fullsized":"2023-10-08T09:13:36.440866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:33.256044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443857+0000","last_change":"2023-10-08T09:13:02.364472+0000","last_active":"2023-10-08T09:13:36.443857+0000","last_peered":"2023-10-08T09:13:36.443857+0000","last_clean":"2023-10-08T09:13:36.443857+0000","last_became_active":"2023-10-08T09:13:02.364130+0000","last_became_peered":"2023-10-08T09:13:02.364130+0000","last_unstale":"2023-10-08T09:13:36.443857+0000","last_undegraded":"2023-10-08T09:13:36.443857+0000","last_fullsized":"2023-10-08T09:13:36.443857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:18.156186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437151+0000","last_change":"2023-10-08T09:13:02.836119+0000","last_active":"2023-10-08T09:13:36.437151+0000","last_peered":"2023-10-08T09:13:36.437151+0000","last_clean":"2023-10-08T09:13:36.437151+0000","last_became_active":"2023-10-08T09:13:02.835548+0000","last_became_peered":"2023-10-08T09:13:02.835548+0000","last_unstale":"2023-10-08T09:13:36.437151+0000","last_undegraded":"2023-10-08T09:13:36.437151+0000","last_fullsized":"2023-10-08T09:13:36.437151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:34.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440444+0000","last_change":"2023-10-08T09:13:07.871447+0000","last_active":"2023-10-08T09:13:36.440444+0000","last_peered":"2023-10-08T09:13:36.440444+0000","last_clean":"2023-10-08T09:13:36.440444+0000","last_became_active":"2023-10-08T09:13:07.871018+0000","last_became_peered":"2023-10-08T09:13:07.871018+0000","last_unstale":"2023-10-08T09:13:36.440444+0000","last_undegraded":"2023-10-08T09:13:36.440444+0000","last_fullsized":"2023-10-08T09:13:36.440444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:34.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:11.471867+0000","last_change":"2023-10-08T09:13:02.378954+0000","last_active":"2023-10-08T09:14:11.471867+0000","last_peered":"2023-10-08T09:14:11.471867+0000","last_clean":"2023-10-08T09:14:11.471867+0000","last_became_active":"2023-10-08T09:13:02.378635+0000","last_became_peered":"2023-10-08T09:13:02.378635+0000","last_unstale":"2023-10-08T09:14:11.471867+0000","last_undegraded":"2023-10-08T09:14:11.471867+0000","last_fullsized":"2023-10-08T09:14:11.471867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:09.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436789+0000","last_change":"2023-10-08T09:13:02.828732+0000","last_active":"2023-10-08T09:13:36.436789+0000","last_peered":"2023-10-08T09:13:36.436789+0000","last_clean":"2023-10-08T09:13:36.436789+0000","last_became_active":"2023-10-08T09:13:02.828449+0000","last_became_peered":"2023-10-08T09:13:02.828449+0000","last_unstale":"2023-10-08T09:13:36.436789+0000","last_undegraded":"2023-10-08T09:13:36.436789+0000","last_fullsized":"2023-10-08T09:13:36.436789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:09.883908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446290+0000","last_change":"2023-10-08T09:13:07.869610+0000","last_active":"2023-10-08T09:13:36.446290+0000","last_peered":"2023-10-08T09:13:36.446290+0000","last_clean":"2023-10-08T09:13:36.446290+0000","last_became_active":"2023-10-08T09:13:07.869225+0000","last_became_peered":"2023-10-08T09:13:07.869225+0000","last_unstale":"2023-10-08T09:13:36.446290+0000","last_undegraded":"2023-10-08T09:13:36.446290+0000","last_fullsized":"2023-10-08T09:13:36.446290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:35.225452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436233+0000","last_change":"2023-10-08T09:13:02.375658+0000","last_active":"2023-10-08T09:13:36.436233+0000","last_peered":"2023-10-08T09:13:36.436233+0000","last_clean":"2023-10-08T09:13:36.436233+0000","last_became_active":"2023-10-08T09:13:02.375383+0000","last_became_peered":"2023-10-08T09:13:02.375383+0000","last_unstale":"2023-10-08T09:13:36.436233+0000","last_undegraded":"2023-10-08T09:13:36.436233+0000","last_fullsized":"2023-10-08T09:13:36.436233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:52.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.455645+0000","last_change":"2023-10-08T09:13:37.455645+0000","last_active":"2023-10-08T09:13:37.455645+0000","last_peered":"2023-10-08T09:13:37.455645+0000","last_clean":"2023-10-08T09:13:37.455645+0000","last_became_active":"2023-10-08T09:13:37.455207+0000","last_became_peered":"2023-10-08T09:13:37.455207+0000","last_unstale":"2023-10-08T09:13:37.455645+0000","last_undegraded":"2023-10-08T09:13:37.455645+0000","last_fullsized":"2023-10-08T09:13:37.455645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:18.021608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445968+0000","last_change":"2023-10-08T09:13:07.871441+0000","last_active":"2023-10-08T09:13:36.445968+0000","last_peered":"2023-10-08T09:13:36.445968+0000","last_clean":"2023-10-08T09:13:36.445968+0000","last_became_active":"2023-10-08T09:13:07.870968+0000","last_became_peered":"2023-10-08T09:13:07.870968+0000","last_unstale":"2023-10-08T09:13:36.445968+0000","last_undegraded":"2023-10-08T09:13:36.445968+0000","last_fullsized":"2023-10-08T09:13:36.445968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:48.746147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436858+0000","last_change":"2023-10-08T09:13:02.372514+0000","last_active":"2023-10-08T09:13:36.436858+0000","last_peered":"2023-10-08T09:13:36.436858+0000","last_clean":"2023-10-08T09:13:36.436858+0000","last_became_active":"2023-10-08T09:13:02.372181+0000","last_became_peered":"2023-10-08T09:13:02.372181+0000","last_unstale":"2023-10-08T09:13:36.436858+0000","last_undegraded":"2023-10-08T09:13:36.436858+0000","last_fullsized":"2023-10-08T09:13:36.436858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:28.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440816+0000","last_change":"2023-10-08T09:13:02.835220+0000","last_active":"2023-10-08T09:13:36.440816+0000","last_peered":"2023-10-08T09:13:36.440816+0000","last_clean":"2023-10-08T09:13:36.440816+0000","last_became_active":"2023-10-08T09:13:02.834914+0000","last_became_peered":"2023-10-08T09:13:02.834914+0000","last_unstale":"2023-10-08T09:13:36.440816+0000","last_undegraded":"2023-10-08T09:13:36.440816+0000","last_fullsized":"2023-10-08T09:13:36.440816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:37.447418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436452+0000","last_change":"2023-10-08T09:13:08.096146+0000","last_active":"2023-10-08T09:13:36.436452+0000","last_peered":"2023-10-08T09:13:36.436452+0000","last_clean":"2023-10-08T09:13:36.436452+0000","last_became_active":"2023-10-08T09:13:08.095703+0000","last_became_peered":"2023-10-08T09:13:08.095703+0000","last_unstale":"2023-10-08T09:13:36.436452+0000","last_undegraded":"2023-10-08T09:13:36.436452+0000","last_fullsized":"2023-10-08T09:13:36.436452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:37.511094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440338+0000","last_change":"2023-10-08T09:13:02.372836+0000","last_active":"2023-10-08T09:13:36.440338+0000","last_peered":"2023-10-08T09:13:36.440338+0000","last_clean":"2023-10-08T09:13:36.440338+0000","last_became_active":"2023-10-08T09:13:02.372488+0000","last_became_peered":"2023-10-08T09:13:02.372488+0000","last_unstale":"2023-10-08T09:13:36.440338+0000","last_undegraded":"2023-10-08T09:13:36.440338+0000","last_fullsized":"2023-10-08T09:13:36.440338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:19.269539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443814+0000","last_change":"2023-10-08T09:13:02.827306+0000","last_active":"2023-10-08T09:13:36.443814+0000","last_peered":"2023-10-08T09:13:36.443814+0000","last_clean":"2023-10-08T09:13:36.443814+0000","last_became_active":"2023-10-08T09:13:02.826965+0000","last_became_peered":"2023-10-08T09:13:02.826965+0000","last_unstale":"2023-10-08T09:13:36.443814+0000","last_undegraded":"2023-10-08T09:13:36.443814+0000","last_fullsized":"2023-10-08T09:13:36.443814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:27.513860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446236+0000","last_change":"2023-10-08T09:13:07.862637+0000","last_active":"2023-10-08T09:13:36.446236+0000","last_peered":"2023-10-08T09:13:36.446236+0000","last_clean":"2023-10-08T09:13:36.446236+0000","last_became_active":"2023-10-08T09:13:07.862117+0000","last_became_peered":"2023-10-08T09:13:07.862117+0000","last_unstale":"2023-10-08T09:13:36.446236+0000","last_undegraded":"2023-10-08T09:13:36.446236+0000","last_fullsized":"2023-10-08T09:13:36.446236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:14.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:14.499880+0000","last_change":"2023-10-08T09:13:02.379585+0000","last_active":"2023-10-08T09:14:14.499880+0000","last_peered":"2023-10-08T09:14:14.499880+0000","last_clean":"2023-10-08T09:14:14.499880+0000","last_became_active":"2023-10-08T09:13:02.379324+0000","last_became_peered":"2023-10-08T09:13:02.379324+0000","last_unstale":"2023-10-08T09:14:14.499880+0000","last_undegraded":"2023-10-08T09:14:14.499880+0000","last_fullsized":"2023-10-08T09:14:14.499880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:31.609558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443748+0000","last_change":"2023-10-08T09:13:02.833416+0000","last_active":"2023-10-08T09:13:36.443748+0000","last_peered":"2023-10-08T09:13:36.443748+0000","last_clean":"2023-10-08T09:13:36.443748+0000","last_became_active":"2023-10-08T09:13:02.833040+0000","last_became_peered":"2023-10-08T09:13:02.833040+0000","last_unstale":"2023-10-08T09:13:36.443748+0000","last_undegraded":"2023-10-08T09:13:36.443748+0000","last_fullsized":"2023-10-08T09:13:36.443748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:44.480445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446160+0000","last_change":"2023-10-08T09:13:08.095723+0000","last_active":"2023-10-08T09:13:36.446160+0000","last_peered":"2023-10-08T09:13:36.446160+0000","last_clean":"2023-10-08T09:13:36.446160+0000","last_became_active":"2023-10-08T09:13:08.095271+0000","last_became_peered":"2023-10-08T09:13:08.095271+0000","last_unstale":"2023-10-08T09:13:36.446160+0000","last_undegraded":"2023-10-08T09:13:36.446160+0000","last_fullsized":"2023-10-08T09:13:36.446160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:04.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443625+0000","last_change":"2023-10-08T09:13:02.829663+0000","last_active":"2023-10-08T09:13:36.443625+0000","last_peered":"2023-10-08T09:13:36.443625+0000","last_clean":"2023-10-08T09:13:36.443625+0000","last_became_active":"2023-10-08T09:13:02.829354+0000","last_became_peered":"2023-10-08T09:13:02.829354+0000","last_unstale":"2023-10-08T09:13:36.443625+0000","last_undegraded":"2023-10-08T09:13:36.443625+0000","last_fullsized":"2023-10-08T09:13:36.443625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:55.332346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:13.473976+0000","last_change":"2023-10-08T09:13:02.372114+0000","last_active":"2023-10-08T09:14:13.473976+0000","last_peered":"2023-10-08T09:14:13.473976+0000","last_clean":"2023-10-08T09:14:13.473976+0000","last_became_active":"2023-10-08T09:13:02.371603+0000","last_became_peered":"2023-10-08T09:13:02.371603+0000","last_unstale":"2023-10-08T09:14:13.473976+0000","last_undegraded":"2023-10-08T09:14:13.473976+0000","last_fullsized":"2023-10-08T09:14:13.473976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:39.078307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437022+0000","last_change":"2023-10-08T09:13:07.863492+0000","last_active":"2023-10-08T09:13:36.437022+0000","last_peered":"2023-10-08T09:13:36.437022+0000","last_clean":"2023-10-08T09:13:36.437022+0000","last_became_active":"2023-10-08T09:13:07.862803+0000","last_became_peered":"2023-10-08T09:13:07.862803+0000","last_unstale":"2023-10-08T09:13:36.437022+0000","last_undegraded":"2023-10-08T09:13:36.437022+0000","last_fullsized":"2023-10-08T09:13:36.437022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:39.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442781+0000","last_change":"2023-10-08T09:13:02.836596+0000","last_active":"2023-10-08T09:13:36.442781+0000","last_peered":"2023-10-08T09:13:36.442781+0000","last_clean":"2023-10-08T09:13:36.442781+0000","last_became_active":"2023-10-08T09:13:02.836283+0000","last_became_peered":"2023-10-08T09:13:02.836283+0000","last_unstale":"2023-10-08T09:13:36.442781+0000","last_undegraded":"2023-10-08T09:13:36.442781+0000","last_fullsized":"2023-10-08T09:13:36.442781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:02.393575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440466+0000","last_change":"2023-10-08T09:13:02.379516+0000","last_active":"2023-10-08T09:13:36.440466+0000","last_peered":"2023-10-08T09:13:36.440466+0000","last_clean":"2023-10-08T09:13:36.440466+0000","last_became_active":"2023-10-08T09:13:02.379229+0000","last_became_peered":"2023-10-08T09:13:02.379229+0000","last_unstale":"2023-10-08T09:13:36.440466+0000","last_undegraded":"2023-10-08T09:13:36.440466+0000","last_fullsized":"2023-10-08T09:13:36.440466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:55.294651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":312,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436665+0000","last_change":"2023-10-08T09:12:02.620769+0000","last_active":"2023-10-08T09:13:36.436665+0000","last_peered":"2023-10-08T09:13:36.436665+0000","last_clean":"2023-10-08T09:13:36.436665+0000","last_became_active":"2023-10-08T09:12:02.620388+0000","last_became_peered":"2023-10-08T09:12:02.620388+0000","last_unstale":"2023-10-08T09:13:36.436665+0000","last_undegraded":"2023-10-08T09:13:36.436665+0000","last_fullsized":"2023-10-08T09:13:36.436665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_clean_scrub_stamp":"2023-10-08T09:09:10.746288+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:55:35.713060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.444776+0000","last_change":"2023-10-08T09:13:08.096318+0000","last_active":"2023-10-08T09:13:36.444776+0000","last_peered":"2023-10-08T09:13:36.444776+0000","last_clean":"2023-10-08T09:13:36.444776+0000","last_became_active":"2023-10-08T09:13:08.095774+0000","last_became_peered":"2023-10-08T09:13:08.095774+0000","last_unstale":"2023-10-08T09:13:36.444776+0000","last_undegraded":"2023-10-08T09:13:36.444776+0000","last_fullsized":"2023-10-08T09:13:36.444776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:53.089468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445283+0000","last_change":"2023-10-08T09:13:02.835167+0000","last_active":"2023-10-08T09:13:36.445283+0000","last_peered":"2023-10-08T09:13:36.445283+0000","last_clean":"2023-10-08T09:13:36.445283+0000","last_became_active":"2023-10-08T09:13:02.834850+0000","last_became_peered":"2023-10-08T09:13:02.834850+0000","last_unstale":"2023-10-08T09:13:36.445283+0000","last_undegraded":"2023-10-08T09:13:36.445283+0000","last_fullsized":"2023-10-08T09:13:36.445283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:19.422372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436727+0000","last_change":"2023-10-08T09:13:02.377746+0000","last_active":"2023-10-08T09:13:36.436727+0000","last_peered":"2023-10-08T09:13:36.436727+0000","last_clean":"2023-10-08T09:13:36.436727+0000","last_became_active":"2023-10-08T09:13:02.377422+0000","last_became_peered":"2023-10-08T09:13:02.377422+0000","last_unstale":"2023-10-08T09:13:36.436727+0000","last_undegraded":"2023-10-08T09:13:36.436727+0000","last_fullsized":"2023-10-08T09:13:36.436727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:13.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440674+0000","last_change":"2023-10-08T09:13:07.864678+0000","last_active":"2023-10-08T09:13:36.440674+0000","last_peered":"2023-10-08T09:13:36.440674+0000","last_clean":"2023-10-08T09:13:36.440674+0000","last_became_active":"2023-10-08T09:13:07.864226+0000","last_became_peered":"2023-10-08T09:13:07.864226+0000","last_unstale":"2023-10-08T09:13:36.440674+0000","last_undegraded":"2023-10-08T09:13:36.440674+0000","last_fullsized":"2023-10-08T09:13:36.440674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:43:51.179853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436798+0000","last_change":"2023-10-08T09:13:02.370014+0000","last_active":"2023-10-08T09:13:36.436798+0000","last_peered":"2023-10-08T09:13:36.436798+0000","last_clean":"2023-10-08T09:13:36.436798+0000","last_became_active":"2023-10-08T09:13:02.369704+0000","last_became_peered":"2023-10-08T09:13:02.369704+0000","last_unstale":"2023-10-08T09:13:36.436798+0000","last_undegraded":"2023-10-08T09:13:36.436798+0000","last_fullsized":"2023-10-08T09:13:36.436798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:28.902274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446349+0000","last_change":"2023-10-08T09:13:02.833859+0000","last_active":"2023-10-08T09:13:36.446349+0000","last_peered":"2023-10-08T09:13:36.446349+0000","last_clean":"2023-10-08T09:13:36.446349+0000","last_became_active":"2023-10-08T09:13:02.833546+0000","last_became_peered":"2023-10-08T09:13:02.833546+0000","last_unstale":"2023-10-08T09:13:36.446349+0000","last_undegraded":"2023-10-08T09:13:36.446349+0000","last_fullsized":"2023-10-08T09:13:36.446349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:35.534588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441671+0000","last_change":"2023-10-08T09:13:08.095622+0000","last_active":"2023-10-08T09:13:36.441671+0000","last_peered":"2023-10-08T09:13:36.441671+0000","last_clean":"2023-10-08T09:13:36.441671+0000","last_became_active":"2023-10-08T09:13:08.095143+0000","last_became_peered":"2023-10-08T09:13:08.095143+0000","last_unstale":"2023-10-08T09:13:36.441671+0000","last_undegraded":"2023-10-08T09:13:36.441671+0000","last_fullsized":"2023-10-08T09:13:36.441671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:46.738102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.459534+0000","last_change":"2023-10-08T09:13:37.459185+0000","last_active":"2023-10-08T09:13:37.459534+0000","last_peered":"2023-10-08T09:13:37.459534+0000","last_clean":"2023-10-08T09:13:37.459534+0000","last_became_active":"2023-10-08T09:13:37.451105+0000","last_became_peered":"2023-10-08T09:13:37.451105+0000","last_unstale":"2023-10-08T09:13:37.459534+0000","last_undegraded":"2023-10-08T09:13:37.459534+0000","last_fullsized":"2023-10-08T09:13:37.459534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.768269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442434+0000","last_change":"2023-10-08T09:13:02.834053+0000","last_active":"2023-10-08T09:13:36.442434+0000","last_peered":"2023-10-08T09:13:36.442434+0000","last_clean":"2023-10-08T09:13:36.442434+0000","last_became_active":"2023-10-08T09:13:02.833707+0000","last_became_peered":"2023-10-08T09:13:02.833707+0000","last_unstale":"2023-10-08T09:13:36.442434+0000","last_undegraded":"2023-10-08T09:13:36.442434+0000","last_fullsized":"2023-10-08T09:13:36.442434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:31.372003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442555+0000","last_change":"2023-10-08T09:13:07.855734+0000","last_active":"2023-10-08T09:13:36.442555+0000","last_peered":"2023-10-08T09:13:36.442555+0000","last_clean":"2023-10-08T09:13:36.442555+0000","last_became_active":"2023-10-08T09:13:07.855313+0000","last_became_peered":"2023-10-08T09:13:07.855313+0000","last_unstale":"2023-10-08T09:13:36.442555+0000","last_undegraded":"2023-10-08T09:13:36.442555+0000","last_fullsized":"2023-10-08T09:13:36.442555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:15.207337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440758+0000","last_change":"2023-10-08T09:13:02.371319+0000","last_active":"2023-10-08T09:13:36.440758+0000","last_peered":"2023-10-08T09:13:36.440758+0000","last_clean":"2023-10-08T09:13:36.440758+0000","last_became_active":"2023-10-08T09:13:02.371049+0000","last_became_peered":"2023-10-08T09:13:02.371049+0000","last_unstale":"2023-10-08T09:13:36.440758+0000","last_undegraded":"2023-10-08T09:13:36.440758+0000","last_fullsized":"2023-10-08T09:13:36.440758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:28.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437178+0000","last_change":"2023-10-08T09:13:02.838682+0000","last_active":"2023-10-08T09:13:36.437178+0000","last_peered":"2023-10-08T09:13:36.437178+0000","last_clean":"2023-10-08T09:13:36.437178+0000","last_became_active":"2023-10-08T09:13:02.838389+0000","last_became_peered":"2023-10-08T09:13:02.838389+0000","last_unstale":"2023-10-08T09:13:36.437178+0000","last_undegraded":"2023-10-08T09:13:36.437178+0000","last_fullsized":"2023-10-08T09:13:36.437178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:52.937723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440434+0000","last_change":"2023-10-08T09:13:08.098131+0000","last_active":"2023-10-08T09:13:36.440434+0000","last_peered":"2023-10-08T09:13:36.440434+0000","last_clean":"2023-10-08T09:13:36.440434+0000","last_became_active":"2023-10-08T09:13:08.097352+0000","last_became_peered":"2023-10-08T09:13:08.097352+0000","last_unstale":"2023-10-08T09:13:36.440434+0000","last_undegraded":"2023-10-08T09:13:36.440434+0000","last_fullsized":"2023-10-08T09:13:36.440434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:30.388875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.461067+0000","last_change":"2023-10-08T09:13:37.461067+0000","last_active":"2023-10-08T09:13:37.461067+0000","last_peered":"2023-10-08T09:13:37.461067+0000","last_clean":"2023-10-08T09:13:37.461067+0000","last_became_active":"2023-10-08T09:13:37.451739+0000","last_became_peered":"2023-10-08T09:13:37.451739+0000","last_unstale":"2023-10-08T09:13:37.461067+0000","last_undegraded":"2023-10-08T09:13:37.461067+0000","last_fullsized":"2023-10-08T09:13:37.461067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:26.603798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436259+0000","last_change":"2023-10-08T09:13:02.830017+0000","last_active":"2023-10-08T09:13:36.436259+0000","last_peered":"2023-10-08T09:13:36.436259+0000","last_clean":"2023-10-08T09:13:36.436259+0000","last_became_active":"2023-10-08T09:13:02.829700+0000","last_became_peered":"2023-10-08T09:13:02.829700+0000","last_unstale":"2023-10-08T09:13:36.436259+0000","last_undegraded":"2023-10-08T09:13:36.436259+0000","last_fullsized":"2023-10-08T09:13:36.436259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:31:31.387049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440739+0000","last_change":"2023-10-08T09:13:07.869097+0000","last_active":"2023-10-08T09:13:36.440739+0000","last_peered":"2023-10-08T09:13:36.440739+0000","last_clean":"2023-10-08T09:13:36.440739+0000","last_became_active":"2023-10-08T09:13:07.868710+0000","last_became_peered":"2023-10-08T09:13:07.868710+0000","last_unstale":"2023-10-08T09:13:36.440739+0000","last_undegraded":"2023-10-08T09:13:36.440739+0000","last_fullsized":"2023-10-08T09:13:36.440739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:37:27.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442264+0000","last_change":"2023-10-08T09:13:02.370694+0000","last_active":"2023-10-08T09:13:36.442264+0000","last_peered":"2023-10-08T09:13:36.442264+0000","last_clean":"2023-10-08T09:13:36.442264+0000","last_became_active":"2023-10-08T09:13:02.370161+0000","last_became_peered":"2023-10-08T09:13:02.370161+0000","last_unstale":"2023-10-08T09:13:36.442264+0000","last_undegraded":"2023-10-08T09:13:36.442264+0000","last_fullsized":"2023-10-08T09:13:36.442264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:04.809703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440513+0000","last_change":"2023-10-08T09:13:02.831026+0000","last_active":"2023-10-08T09:13:36.440513+0000","last_peered":"2023-10-08T09:13:36.440513+0000","last_clean":"2023-10-08T09:13:36.440513+0000","last_became_active":"2023-10-08T09:13:02.830715+0000","last_became_peered":"2023-10-08T09:13:02.830715+0000","last_unstale":"2023-10-08T09:13:36.440513+0000","last_undegraded":"2023-10-08T09:13:36.440513+0000","last_fullsized":"2023-10-08T09:13:36.440513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:47.709820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439763+0000","last_change":"2023-10-08T09:13:07.860464+0000","last_active":"2023-10-08T09:13:36.439763+0000","last_peered":"2023-10-08T09:13:36.439763+0000","last_clean":"2023-10-08T09:13:36.439763+0000","last_became_active":"2023-10-08T09:13:07.859962+0000","last_became_peered":"2023-10-08T09:13:07.859962+0000","last_unstale":"2023-10-08T09:13:36.439763+0000","last_undegraded":"2023-10-08T09:13:36.439763+0000","last_fullsized":"2023-10-08T09:13:36.439763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:04.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.708490+0000","last_change":"2023-10-08T09:13:02.371308+0000","last_active":"2023-10-08T09:13:36.708490+0000","last_peered":"2023-10-08T09:13:36.708490+0000","last_clean":"2023-10-08T09:13:36.708490+0000","last_became_active":"2023-10-08T09:13:02.370968+0000","last_became_peered":"2023-10-08T09:13:02.370968+0000","last_unstale":"2023-10-08T09:13:36.708490+0000","last_undegraded":"2023-10-08T09:13:36.708490+0000","last_fullsized":"2023-10-08T09:13:36.708490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:58.491112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442038+0000","last_change":"2023-10-08T09:13:02.826511+0000","last_active":"2023-10-08T09:13:36.442038+0000","last_peered":"2023-10-08T09:13:36.442038+0000","last_clean":"2023-10-08T09:13:36.442038+0000","last_became_active":"2023-10-08T09:13:02.826205+0000","last_became_peered":"2023-10-08T09:13:02.826205+0000","last_unstale":"2023-10-08T09:13:36.442038+0000","last_undegraded":"2023-10-08T09:13:36.442038+0000","last_fullsized":"2023-10-08T09:13:36.442038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:12.270176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440248+0000","last_change":"2023-10-08T09:13:08.096360+0000","last_active":"2023-10-08T09:13:36.440248+0000","last_peered":"2023-10-08T09:13:36.440248+0000","last_clean":"2023-10-08T09:13:36.440248+0000","last_became_active":"2023-10-08T09:13:08.095915+0000","last_became_peered":"2023-10-08T09:13:08.095915+0000","last_unstale":"2023-10-08T09:13:36.440248+0000","last_undegraded":"2023-10-08T09:13:36.440248+0000","last_fullsized":"2023-10-08T09:13:36.440248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:59.412471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436477+0000","last_change":"2023-10-08T09:13:02.375287+0000","last_active":"2023-10-08T09:13:36.436477+0000","last_peered":"2023-10-08T09:13:36.436477+0000","last_clean":"2023-10-08T09:13:36.436477+0000","last_became_active":"2023-10-08T09:13:02.374668+0000","last_became_peered":"2023-10-08T09:13:02.374668+0000","last_unstale":"2023-10-08T09:13:36.436477+0000","last_undegraded":"2023-10-08T09:13:36.436477+0000","last_fullsized":"2023-10-08T09:13:36.436477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:22.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436931+0000","last_change":"2023-10-08T09:13:02.828240+0000","last_active":"2023-10-08T09:13:36.436931+0000","last_peered":"2023-10-08T09:13:36.436931+0000","last_clean":"2023-10-08T09:13:36.436931+0000","last_became_active":"2023-10-08T09:13:02.827951+0000","last_became_peered":"2023-10-08T09:13:02.827951+0000","last_unstale":"2023-10-08T09:13:36.436931+0000","last_undegraded":"2023-10-08T09:13:36.436931+0000","last_fullsized":"2023-10-08T09:13:36.436931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:37.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443935+0000","last_change":"2023-10-08T09:13:07.865846+0000","last_active":"2023-10-08T09:13:36.443935+0000","last_peered":"2023-10-08T09:13:36.443935+0000","last_clean":"2023-10-08T09:13:36.443935+0000","last_became_active":"2023-10-08T09:13:07.865529+0000","last_became_peered":"2023-10-08T09:13:07.865529+0000","last_unstale":"2023-10-08T09:13:36.443935+0000","last_undegraded":"2023-10-08T09:13:36.443935+0000","last_fullsized":"2023-10-08T09:13:36.443935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:34:03.408036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:14.500688+0000","last_change":"2023-10-08T09:13:02.380128+0000","last_active":"2023-10-08T09:14:14.500688+0000","last_peered":"2023-10-08T09:14:14.500688+0000","last_clean":"2023-10-08T09:14:14.500688+0000","last_became_active":"2023-10-08T09:13:02.379800+0000","last_became_peered":"2023-10-08T09:13:02.379800+0000","last_unstale":"2023-10-08T09:14:14.500688+0000","last_undegraded":"2023-10-08T09:14:14.500688+0000","last_fullsized":"2023-10-08T09:14:14.500688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:01.546289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437151+0000","last_change":"2023-10-08T09:13:02.829474+0000","last_active":"2023-10-08T09:13:36.437151+0000","last_peered":"2023-10-08T09:13:36.437151+0000","last_clean":"2023-10-08T09:13:36.437151+0000","last_became_active":"2023-10-08T09:13:02.829173+0000","last_became_peered":"2023-10-08T09:13:02.829173+0000","last_unstale":"2023-10-08T09:13:36.437151+0000","last_undegraded":"2023-10-08T09:13:36.437151+0000","last_fullsized":"2023-10-08T09:13:36.437151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:02.469158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.453778+0000","last_change":"2023-10-08T09:13:37.453778+0000","last_active":"2023-10-08T09:13:37.453778+0000","last_peered":"2023-10-08T09:13:37.453778+0000","last_clean":"2023-10-08T09:13:37.453778+0000","last_became_active":"2023-10-08T09:13:37.453354+0000","last_became_peered":"2023-10-08T09:13:37.453354+0000","last_unstale":"2023-10-08T09:13:37.453778+0000","last_undegraded":"2023-10-08T09:13:37.453778+0000","last_fullsized":"2023-10-08T09:13:37.453778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:50.035588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443501+0000","last_change":"2023-10-08T09:13:02.376641+0000","last_active":"2023-10-08T09:13:36.443501+0000","last_peered":"2023-10-08T09:13:36.443501+0000","last_clean":"2023-10-08T09:13:36.443501+0000","last_became_active":"2023-10-08T09:13:02.376348+0000","last_became_peered":"2023-10-08T09:13:02.376348+0000","last_unstale":"2023-10-08T09:13:36.443501+0000","last_undegraded":"2023-10-08T09:13:36.443501+0000","last_fullsized":"2023-10-08T09:13:36.443501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:42.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437239+0000","last_change":"2023-10-08T09:13:02.836201+0000","last_active":"2023-10-08T09:13:36.437239+0000","last_peered":"2023-10-08T09:13:36.437239+0000","last_clean":"2023-10-08T09:13:36.437239+0000","last_became_active":"2023-10-08T09:13:02.835885+0000","last_became_peered":"2023-10-08T09:13:02.835885+0000","last_unstale":"2023-10-08T09:13:36.437239+0000","last_undegraded":"2023-10-08T09:13:36.437239+0000","last_fullsized":"2023-10-08T09:13:36.437239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:34.310854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443140+0000","last_change":"2023-10-08T09:13:07.863854+0000","last_active":"2023-10-08T09:13:36.443140+0000","last_peered":"2023-10-08T09:13:36.443140+0000","last_clean":"2023-10-08T09:13:36.443140+0000","last_became_active":"2023-10-08T09:13:07.863460+0000","last_became_peered":"2023-10-08T09:13:07.863460+0000","last_unstale":"2023-10-08T09:13:36.443140+0000","last_undegraded":"2023-10-08T09:13:36.443140+0000","last_fullsized":"2023-10-08T09:13:36.443140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:03.093858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446178+0000","last_change":"2023-10-08T09:13:02.372336+0000","last_active":"2023-10-08T09:13:36.446178+0000","last_peered":"2023-10-08T09:13:36.446178+0000","last_clean":"2023-10-08T09:13:36.446178+0000","last_became_active":"2023-10-08T09:13:02.372022+0000","last_became_peered":"2023-10-08T09:13:02.372022+0000","last_unstale":"2023-10-08T09:13:36.446178+0000","last_undegraded":"2023-10-08T09:13:36.446178+0000","last_fullsized":"2023-10-08T09:13:36.446178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:10.610332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437193+0000","last_change":"2023-10-08T09:13:02.832430+0000","last_active":"2023-10-08T09:13:36.437193+0000","last_peered":"2023-10-08T09:13:36.437193+0000","last_clean":"2023-10-08T09:13:36.437193+0000","last_became_active":"2023-10-08T09:13:02.832092+0000","last_became_peered":"2023-10-08T09:13:02.832092+0000","last_unstale":"2023-10-08T09:13:36.437193+0000","last_undegraded":"2023-10-08T09:13:36.437193+0000","last_fullsized":"2023-10-08T09:13:36.437193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:00.798900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440609+0000","last_change":"2023-10-08T09:13:02.833114+0000","last_active":"2023-10-08T09:13:36.440609+0000","last_peered":"2023-10-08T09:13:36.440609+0000","last_clean":"2023-10-08T09:13:36.440609+0000","last_became_active":"2023-10-08T09:13:02.830959+0000","last_became_peered":"2023-10-08T09:13:02.830959+0000","last_unstale":"2023-10-08T09:13:36.440609+0000","last_undegraded":"2023-10-08T09:13:36.440609+0000","last_fullsized":"2023-10-08T09:13:36.440609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:13.853585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442865+0000","last_change":"2023-10-08T09:13:02.836046+0000","last_active":"2023-10-08T09:13:36.442865+0000","last_peered":"2023-10-08T09:13:36.442865+0000","last_clean":"2023-10-08T09:13:36.442865+0000","last_became_active":"2023-10-08T09:13:02.835722+0000","last_became_peered":"2023-10-08T09:13:02.835722+0000","last_unstale":"2023-10-08T09:13:36.442865+0000","last_undegraded":"2023-10-08T09:13:36.442865+0000","last_fullsized":"2023-10-08T09:13:36.442865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:00.316259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442665+0000","last_change":"2023-10-08T09:13:02.830021+0000","last_active":"2023-10-08T09:13:36.442665+0000","last_peered":"2023-10-08T09:13:36.442665+0000","last_clean":"2023-10-08T09:13:36.442665+0000","last_became_active":"2023-10-08T09:13:02.829397+0000","last_became_peered":"2023-10-08T09:13:02.829397+0000","last_unstale":"2023-10-08T09:13:36.442665+0000","last_undegraded":"2023-10-08T09:13:36.442665+0000","last_fullsized":"2023-10-08T09:13:36.442665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:29.153457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436590+0000","last_change":"2023-10-08T09:13:02.832002+0000","last_active":"2023-10-08T09:13:36.436590+0000","last_peered":"2023-10-08T09:13:36.436590+0000","last_clean":"2023-10-08T09:13:36.436590+0000","last_became_active":"2023-10-08T09:13:02.831666+0000","last_became_peered":"2023-10-08T09:13:02.831666+0000","last_unstale":"2023-10-08T09:13:36.436590+0000","last_undegraded":"2023-10-08T09:13:36.436590+0000","last_fullsized":"2023-10-08T09:13:36.436590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:16.414127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446412+0000","last_change":"2023-10-08T09:13:02.835881+0000","last_active":"2023-10-08T09:13:36.446412+0000","last_peered":"2023-10-08T09:13:36.446412+0000","last_clean":"2023-10-08T09:13:36.446412+0000","last_became_active":"2023-10-08T09:13:02.835587+0000","last_became_peered":"2023-10-08T09:13:02.835587+0000","last_unstale":"2023-10-08T09:13:36.446412+0000","last_undegraded":"2023-10-08T09:13:36.446412+0000","last_fullsized":"2023-10-08T09:13:36.446412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:52.323234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440978+0000","last_change":"2023-10-08T09:13:02.826371+0000","last_active":"2023-10-08T09:13:36.440978+0000","last_peered":"2023-10-08T09:13:36.440978+0000","last_clean":"2023-10-08T09:13:36.440978+0000","last_became_active":"2023-10-08T09:13:02.826093+0000","last_became_peered":"2023-10-08T09:13:02.826093+0000","last_unstale":"2023-10-08T09:13:36.440978+0000","last_undegraded":"2023-10-08T09:13:36.440978+0000","last_fullsized":"2023-10-08T09:13:36.440978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:08.113739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443860+0000","last_change":"2023-10-08T09:13:02.829936+0000","last_active":"2023-10-08T09:13:36.443860+0000","last_peered":"2023-10-08T09:13:36.443860+0000","last_clean":"2023-10-08T09:13:36.443860+0000","last_became_active":"2023-10-08T09:13:02.829498+0000","last_became_peered":"2023-10-08T09:13:02.829498+0000","last_unstale":"2023-10-08T09:13:36.443860+0000","last_undegraded":"2023-10-08T09:13:36.443860+0000","last_fullsized":"2023-10-08T09:13:36.443860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:42.200791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442594+0000","last_change":"2023-10-08T09:13:02.840415+0000","last_active":"2023-10-08T09:13:36.442594+0000","last_peered":"2023-10-08T09:13:36.442594+0000","last_clean":"2023-10-08T09:13:36.442594+0000","last_became_active":"2023-10-08T09:13:02.840075+0000","last_became_peered":"2023-10-08T09:13:02.840075+0000","last_unstale":"2023-10-08T09:13:36.442594+0000","last_undegraded":"2023-10-08T09:13:36.442594+0000","last_fullsized":"2023-10-08T09:13:36.442594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:48.959372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443355+0000","last_change":"2023-10-08T09:13:02.828337+0000","last_active":"2023-10-08T09:13:36.443355+0000","last_peered":"2023-10-08T09:13:36.443355+0000","last_clean":"2023-10-08T09:13:36.443355+0000","last_became_active":"2023-10-08T09:13:02.827998+0000","last_became_peered":"2023-10-08T09:13:02.827998+0000","last_unstale":"2023-10-08T09:13:36.443355+0000","last_undegraded":"2023-10-08T09:13:36.443355+0000","last_fullsized":"2023-10-08T09:13:36.443355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:27.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442365+0000","last_change":"2023-10-08T09:13:02.833052+0000","last_active":"2023-10-08T09:13:36.442365+0000","last_peered":"2023-10-08T09:13:36.442365+0000","last_clean":"2023-10-08T09:13:36.442365+0000","last_became_active":"2023-10-08T09:13:02.832635+0000","last_became_peered":"2023-10-08T09:13:02.832635+0000","last_unstale":"2023-10-08T09:13:36.442365+0000","last_undegraded":"2023-10-08T09:13:36.442365+0000","last_fullsized":"2023-10-08T09:13:36.442365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:50.995768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443151+0000","last_change":"2023-10-08T09:13:02.840921+0000","last_active":"2023-10-08T09:13:36.443151+0000","last_peered":"2023-10-08T09:13:36.443151+0000","last_clean":"2023-10-08T09:13:36.443151+0000","last_became_active":"2023-10-08T09:13:02.840566+0000","last_became_peered":"2023-10-08T09:13:02.840566+0000","last_unstale":"2023-10-08T09:13:36.443151+0000","last_undegraded":"2023-10-08T09:13:36.443151+0000","last_fullsized":"2023-10-08T09:13:36.443151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:20.219581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446239+0000","last_change":"2023-10-08T09:13:02.834491+0000","last_active":"2023-10-08T09:13:36.446239+0000","last_peered":"2023-10-08T09:13:36.446239+0000","last_clean":"2023-10-08T09:13:36.446239+0000","last_became_active":"2023-10-08T09:13:02.834058+0000","last_became_peered":"2023-10-08T09:13:02.834058+0000","last_unstale":"2023-10-08T09:13:36.446239+0000","last_undegraded":"2023-10-08T09:13:36.446239+0000","last_fullsized":"2023-10-08T09:13:36.446239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:47.699423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440869+0000","last_change":"2023-10-08T09:13:02.836433+0000","last_active":"2023-10-08T09:13:36.440869+0000","last_peered":"2023-10-08T09:13:36.440869+0000","last_clean":"2023-10-08T09:13:36.440869+0000","last_became_active":"2023-10-08T09:13:02.835147+0000","last_became_peered":"2023-10-08T09:13:02.835147+0000","last_unstale":"2023-10-08T09:13:36.440869+0000","last_undegraded":"2023-10-08T09:13:36.440869+0000","last_fullsized":"2023-10-08T09:13:36.440869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:51.233662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247538,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65785,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":464,"num_read_kb":303,"num_write":235,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514522,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":488411,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.5},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.46100000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.64700000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.56000000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.624},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.54000000000000004}]},{"osd":7,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72899999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.81799999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.84499999999999997}]},{"osd":10,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71599999999999997}]}]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":1137516,"data_compressed":53691,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.502}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.61399999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73499999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83299999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.505,"5min":0.434,"15min":0.434},"max":{"1min":0.96199999999999997,"5min":1.01,"15min":1.01},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80700000000000005,"5min":1.115,"15min":1.115},"last":0.82799999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.441},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74299999999999999,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89300000000000002,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.125,"5min":1.534,"15min":1.534},"last":0.84699999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.879,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0840000000000001,"5min":1.649,"15min":1.649},"last":0.753}]},{"osd":11,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80800000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":487951,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.27300000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.78800000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80100000000000005,"5min":1.012,"15min":1.012},"last":0.55600000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.47799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74199999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76500000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.61399999999999999,"15min":0.628},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91400000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79500000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":1.165,"15min":1.165},"last":0.57099999999999995}]},{"osd":5,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.997,"5min":1.216,"15min":1.216},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.98199999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0449999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.83199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.056,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92800000000000005,"5min":1.337,"15min":1.337},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.69399999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.81200000000000006,"15min":0.82999999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96299999999999997,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79500000000000004,"5min":1.024,"15min":1.024},"last":0.64800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75900000000000001}]}]},{"osd":8,"up_from":56,"seq":240518168615,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":483902,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.32000000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.45000000000000001,"15min":0.436},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.33300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.33800000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.52700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0229999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.42199999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.51100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":0.999},"last":0.61199999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.93100000000000005,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.53900000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.57299999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.622},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67500000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.754,"15min":0.72899999999999998},"min":{"1min":0.76200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.69499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.65200000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.63200000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080729,"data_compressed":53691,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.41199999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54700000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.499},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.745},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83599999999999997,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.518,"15min":1.518},"last":0.76400000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81599999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.60899999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.505,"15min":0.47699999999999998},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.587,"15min":1.587},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":1.496,"15min":1.496},"last":0.79400000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.71399999999999997,"15min":0.68100000000000005},"min":{"1min":0.54300000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.71099999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66900000000000004,"15min":0.65900000000000003},"min":{"1min":0.496,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.80900000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90300000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68999999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91700000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63500000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88500000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":483845,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39000000000000001,"15min":0.371},"min":{"1min":0.32100000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.38300000000000001},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47899999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.46200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1.145,"15min":1.145},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46800000000000003,"15min":0.44},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.46200000000000002},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80000000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61499999999999999,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92400000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.46300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.0189999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69799999999999995,"15min":0.68000000000000005},"min":{"1min":0.51200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.622}]},{"osd":7,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.57599999999999996},{"interface":"front","average":{"1min":1.1879999999999999,"5min":0.81299999999999994,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.1229999999999993,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.63200000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.55700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.755},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92200000000000004,"5min":1.175,"15min":1.175},"last":0.60199999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.77400000000000002},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.075,"5min":1.274,"15min":1.274},"last":0.54000000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.61299999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.25,"5min":0.96999999999999997,"15min":0.92400000000000004},"min":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":9.1769999999999996,"5min":9.1769999999999996,"15min":9.1769999999999996},"last":0.67200000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":5.9610000000000003,"5min":5.9610000000000003,"15min":5.9610000000000003},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":547281,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91500000000000004,"5min":0.999,"15min":0.999},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.49099999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53100000000000003,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73199999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.439,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":0.751,"15min":0.751},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.42099999999999999},"min":{"1min":0.41799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.87}]},{"osd":6,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84399999999999997,"5min":1.034,"15min":1.034},"last":0.748},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.497,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.59299999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79200000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.89200000000000002}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.73599999999999999,"15min":0.71399999999999997},"min":{"1min":0.72899999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.91700000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.98399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":485301,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.45700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57699999999999996,"15min":0.52000000000000002},"min":{"1min":0.443,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.65100000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.59799999999999998,"15min":0.52800000000000002},"min":{"1min":0.53800000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.468,"5min":3.468,"15min":3.468},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.62,"15min":0.55400000000000005},"min":{"1min":0.52900000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.4409999999999998,"5min":3.4409999999999998,"15min":3.4409999999999998},"last":0.47899999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.435},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.45400000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.53800000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91800000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.47999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79100000000000004,"5min":1.133,"15min":1.133},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.095,"15min":1.095},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.30299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.75700000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0209999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.69299999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80700000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73199999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.79400000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.4569999999999999,"5min":3.4569999999999999,"15min":3.4569999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.58499999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90700000000000003,"5min":1.008,"15min":1.008},"last":0.746}]},{"osd":11,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547774,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80800000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.48699999999999999},"min":{"1min":0.33500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.41099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.318,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67900000000000005}]},{"osd":3,"last update":"Sun Oct 8 09:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88600000000000001,"5min":1.758,"15min":1.758},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59399999999999997,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.93100000000000005,"5min":1.772,"15min":1.772},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.47999999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96699999999999997,"5min":1.839,"15min":1.839},"last":0.63},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.50900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.014,"5min":1.492,"15min":1.492},"last":0.56899999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81000000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65000000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.755,"15min":0.747},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1040000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.72499999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.877,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.83099999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.755},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":482832,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39200000000000002,"15min":0.376},"min":{"1min":0.27400000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.60699999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.30399999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.442}]},{"osd":1,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.748,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72399999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.78800000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82199999999999995,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.49399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72399999999999998,"5min":1.494,"15min":1.494},"last":0.55100000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.89900000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.87},"last":0.622}]},{"osd":6,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95499999999999996,"5min":1.01,"15min":1.01},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57999999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.81499999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86099999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.74299999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.745,"15min":0.75800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.69199999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92500000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.81799999999999995,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":1.125,"15min":1.125},"last":0.76900000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.874},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80500000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":487529,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.499,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":1.466,"15min":1.466},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.36799999999999999},"min":{"1min":0.30199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.443}]},{"osd":2,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56799999999999995}]},{"osd":3,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.52700000000000002},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.89300000000000002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.52500000000000002},"min":{"1min":0.36899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77100000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71899999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57699999999999996,"15min":0.55400000000000005},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.78100000000000003,"5min":1.429,"15min":1.429},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93500000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79300000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78900000000000003,"5min":0.873,"15min":0.873},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.40999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.82299999999999995,"5min":1.353,"15min":1.353},"last":0.62}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.872,"5min":1.373,"15min":1.373},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.079,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.77700000000000002},"min":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92100000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.91300000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.48699999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.50900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.68400000000000005}]},{"osd":11,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.747}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1080017,"data_compressed":53691,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.502},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.627,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.998,"15min":0.998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.27400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.433}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63900000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.42199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.442,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.51900000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.52400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.016,"15min":1.016},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67900000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.99099999999999999,"5min":1.071,"15min":1.071},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.006},"last":0.79200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:14:17.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph pg dump --format=json 2023-10-08T09:14:17.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:17 smithi049 ceph-mon[99485]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-08T09:14:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:17 smithi184 ceph-mon[103675]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-08T09:14:18.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:17 smithi132 ceph-mon[103389]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2023-10-08T09:14:18.182 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:18.772 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:18 smithi049 ceph-mon[99485]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:14:18.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:18 smithi184 ceph-mon[103675]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:14:19.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:18 smithi132 ceph-mon[103389]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:14:19.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 -- 172.21.15.49:0/3455594836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c100110 msgr2=0x7f2f2c10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/3455594836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c100110 0x7f2f2c10c580 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400b0a0 tx=0x7f2f2400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 -- 172.21.15.49:0/3455594836 shutdown_connections 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/3455594836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c100110 0x7f2f2c10c580 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/3455594836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2c1056b0 0x7f2f2c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/3455594836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f2c0ff750 0x7f2f2c0ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 -- 172.21.15.49:0/3455594836 >> 172.21.15.49:0/3455594836 conn(0x7f2f2c075790 msgr2=0x7f2f2c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 -- 172.21.15.49:0/3455594836 shutdown_connections 2023-10-08T09:14:19.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 -- 172.21.15.49:0/3455594836 wait complete. 2023-10-08T09:14:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.506+0000 7f2f31dfa700 1 Processor -- start 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f31dfa700 1 -- start start 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f31dfa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2c0ff750 0x7f2f2c101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f31dfa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f2c100110 0x7f2f2c1017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f31dfa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c1056b0 0x7f2f2c101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f31dfa700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c1a5090 con 0x7f2f2c1056b0 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f31dfa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c1a5200 con 0x7f2f2c0ff750 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f31dfa700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c1a53c0 con 0x7f2f2c100110 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c1056b0 0x7f2f2c101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c1056b0 0x7f2f2c101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48904/0 (socket says 172.21.15.49:48904) 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 -- 172.21.15.49:0/256870099 learned_addr learned my addr 172.21.15.49:0/256870099 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2affd700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f2c100110 0x7f2f2c1017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2b7fe700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2c0ff750 0x7f2f2c101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 -- 172.21.15.49:0/256870099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f2c100110 msgr2=0x7f2f2c1017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f2c100110 0x7f2f2c1017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 -- 172.21.15.49:0/256870099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2c0ff750 msgr2=0x7f2f2c101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2c0ff750 0x7f2f2c101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2bfff700 1 -- 172.21.15.49:0/256870099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24009d00 con 0x7f2f2c1056b0 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.507+0000 7f2f2b7fe700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2c0ff750 0x7f2f2c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.508+0000 7f2f2bfff700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c1056b0 0x7f2f2c101d00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400b070 tx=0x7f2f24007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.508+0000 7f2f28ff9700 1 -- 172.21.15.49:0/256870099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400e040 con 0x7f2f2c1056b0 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.508+0000 7f2f28ff9700 1 -- 172.21.15.49:0/256870099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f2400be40 con 0x7f2f2c1056b0 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.508+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c1a5640 con 0x7f2f2c1056b0 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.508+0000 7f2f28ff9700 1 -- 172.21.15.49:0/256870099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f240052a0 con 0x7f2f2c1056b0 2023-10-08T09:14:19.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.508+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c1a5ab0 con 0x7f2f2c1056b0 2023-10-08T09:14:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.509+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c108ce0 con 0x7f2f2c1056b0 2023-10-08T09:14:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.510+0000 7f2f28ff9700 1 -- 172.21.15.49:0/256870099 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2f2401d470 con 0x7f2f2c1056b0 2023-10-08T09:14:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.510+0000 7f2f28ff9700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f140a1510 0x7f2f140a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.511+0000 7f2f28ff9700 1 -- 172.21.15.49:0/256870099 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f240ce5b0 con 0x7f2f2c1056b0 2023-10-08T09:14:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.511+0000 7f2f2b7fe700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f140a1510 0x7f2f140a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.511+0000 7f2f2b7fe700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f140a1510 0x7f2f140a39d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009c90 tx=0x7f2f1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:19.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.515+0000 7f2f28ff9700 1 -- 172.21.15.49:0/256870099 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2f24097830 con 0x7f2f2c1056b0 2023-10-08T09:14:19.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.713+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f2c102760 con 0x7f2f140a1510 2023-10-08T09:14:19.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.729+0000 7f2f28ff9700 1 -- 172.21.15.49:0/256870099 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426749 (secure 0 0 0) 0x7f2f2c102760 con 0x7f2f140a1510 2023-10-08T09:14:19.736 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:19.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f140a1510 msgr2=0x7f2f140a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:19.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f140a1510 0x7f2f140a39d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009c90 tx=0x7f2f1c009260 comp rx=0 tx=0).stop 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c1056b0 msgr2=0x7f2f2c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c1056b0 0x7f2f2c101d00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400b070 tx=0x7f2f24007930 comp rx=0 tx=0).stop 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 shutdown_connections 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f140a1510 0x7f2f140a39d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f2c1056b0 0x7f2f2c101d00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f2c0ff750 0x7f2f2c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 --2- 172.21.15.49:0/256870099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f2c100110 0x7f2f2c1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 >> 172.21.15.49:0/256870099 conn(0x7f2f2c075790 msgr2=0x7f2f2c068530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 shutdown_connections 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:19.732+0000 7f2f31dfa700 1 -- 172.21.15.49:0/256870099 wait complete. 2023-10-08T09:14:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-10-08T09:14:19.980 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:19 smithi049 ceph-mon[99485]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T09:14:19.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:19 smithi184 ceph-mon[103675]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T09:14:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:19 smithi132 ceph-mon[103389]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T09:14:20.240 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2023-10-08T09:14:17.872257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656399,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":595,"num_read_kb":412,"num_write":418,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180859,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336164,"kb_used_data":11900,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605660,"statfs":{"total":1151940427776,"available":1151596195840,"internally_reserved":0,"allocated":12185600,"data_stored":7793088,"data_compressed":319707,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003632"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441045+0000","last_change":"2023-10-08T09:13:02.831436+0000","last_active":"2023-10-08T09:13:36.441045+0000","last_peered":"2023-10-08T09:13:36.441045+0000","last_clean":"2023-10-08T09:13:36.441045+0000","last_became_active":"2023-10-08T09:13:02.831123+0000","last_became_peered":"2023-10-08T09:13:02.831123+0000","last_unstale":"2023-10-08T09:13:36.441045+0000","last_undegraded":"2023-10-08T09:13:36.441045+0000","last_fullsized":"2023-10-08T09:13:36.441045+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:33.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437087+0000","last_change":"2023-10-08T09:13:02.832286+0000","last_active":"2023-10-08T09:13:36.437087+0000","last_peered":"2023-10-08T09:13:36.437087+0000","last_clean":"2023-10-08T09:13:36.437087+0000","last_became_active":"2023-10-08T09:13:02.831881+0000","last_became_peered":"2023-10-08T09:13:02.831881+0000","last_unstale":"2023-10-08T09:13:36.437087+0000","last_undegraded":"2023-10-08T09:13:36.437087+0000","last_fullsized":"2023-10-08T09:13:36.437087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:31.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436447+0000","last_change":"2023-10-08T09:13:02.831454+0000","last_active":"2023-10-08T09:13:36.436447+0000","last_peered":"2023-10-08T09:13:36.436447+0000","last_clean":"2023-10-08T09:13:36.436447+0000","last_became_active":"2023-10-08T09:13:02.831040+0000","last_became_peered":"2023-10-08T09:13:02.831040+0000","last_unstale":"2023-10-08T09:13:36.436447+0000","last_undegraded":"2023-10-08T09:13:36.436447+0000","last_fullsized":"2023-10-08T09:13:36.436447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:16:02.788279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439965+0000","last_change":"2023-10-08T09:13:02.832699+0000","last_active":"2023-10-08T09:13:36.439965+0000","last_peered":"2023-10-08T09:13:36.439965+0000","last_clean":"2023-10-08T09:13:36.439965+0000","last_became_active":"2023-10-08T09:13:02.832345+0000","last_became_peered":"2023-10-08T09:13:02.832345+0000","last_unstale":"2023-10-08T09:13:36.439965+0000","last_undegraded":"2023-10-08T09:13:36.439965+0000","last_fullsized":"2023-10-08T09:13:36.439965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:01.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436354+0000","last_change":"2023-10-08T09:13:02.827767+0000","last_active":"2023-10-08T09:13:36.436354+0000","last_peered":"2023-10-08T09:13:36.436354+0000","last_clean":"2023-10-08T09:13:36.436354+0000","last_became_active":"2023-10-08T09:13:02.827454+0000","last_became_peered":"2023-10-08T09:13:02.827454+0000","last_unstale":"2023-10-08T09:13:36.436354+0000","last_undegraded":"2023-10-08T09:13:36.436354+0000","last_fullsized":"2023-10-08T09:13:36.436354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:43.122347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.452701+0000","last_change":"2023-10-08T09:13:37.452701+0000","last_active":"2023-10-08T09:13:37.452701+0000","last_peered":"2023-10-08T09:13:37.452701+0000","last_clean":"2023-10-08T09:13:37.452701+0000","last_became_active":"2023-10-08T09:13:37.452261+0000","last_became_peered":"2023-10-08T09:13:37.452261+0000","last_unstale":"2023-10-08T09:13:37.452701+0000","last_undegraded":"2023-10-08T09:13:37.452701+0000","last_fullsized":"2023-10-08T09:13:37.452701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:56.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436726+0000","last_change":"2023-10-08T09:13:02.831765+0000","last_active":"2023-10-08T09:13:36.436726+0000","last_peered":"2023-10-08T09:13:36.436726+0000","last_clean":"2023-10-08T09:13:36.436726+0000","last_became_active":"2023-10-08T09:13:02.831446+0000","last_became_peered":"2023-10-08T09:13:02.831446+0000","last_unstale":"2023-10-08T09:13:36.436726+0000","last_undegraded":"2023-10-08T09:13:36.436726+0000","last_fullsized":"2023-10-08T09:13:36.436726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:41.463161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441582+0000","last_change":"2023-10-08T09:13:02.817586+0000","last_active":"2023-10-08T09:13:36.441582+0000","last_peered":"2023-10-08T09:13:36.441582+0000","last_clean":"2023-10-08T09:13:36.441582+0000","last_became_active":"2023-10-08T09:13:02.817231+0000","last_became_peered":"2023-10-08T09:13:02.817231+0000","last_unstale":"2023-10-08T09:13:36.441582+0000","last_undegraded":"2023-10-08T09:13:36.441582+0000","last_fullsized":"2023-10-08T09:13:36.441582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:23.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443527+0000","last_change":"2023-10-08T09:13:02.831925+0000","last_active":"2023-10-08T09:13:36.443527+0000","last_peered":"2023-10-08T09:13:36.443527+0000","last_clean":"2023-10-08T09:13:36.443527+0000","last_became_active":"2023-10-08T09:13:02.831625+0000","last_became_peered":"2023-10-08T09:13:02.831625+0000","last_unstale":"2023-10-08T09:13:36.443527+0000","last_undegraded":"2023-10-08T09:13:36.443527+0000","last_fullsized":"2023-10-08T09:13:36.443527+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:39.115740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443908+0000","last_change":"2023-10-08T09:13:02.831752+0000","last_active":"2023-10-08T09:13:36.443908+0000","last_peered":"2023-10-08T09:13:36.443908+0000","last_clean":"2023-10-08T09:13:36.443908+0000","last_became_active":"2023-10-08T09:13:02.831468+0000","last_became_peered":"2023-10-08T09:13:02.831468+0000","last_unstale":"2023-10-08T09:13:36.443908+0000","last_undegraded":"2023-10-08T09:13:36.443908+0000","last_fullsized":"2023-10-08T09:13:36.443908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:05.758017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437153+0000","last_change":"2023-10-08T09:13:02.828838+0000","last_active":"2023-10-08T09:13:36.437153+0000","last_peered":"2023-10-08T09:13:36.437153+0000","last_clean":"2023-10-08T09:13:36.437153+0000","last_became_active":"2023-10-08T09:13:02.828598+0000","last_became_peered":"2023-10-08T09:13:02.828598+0000","last_unstale":"2023-10-08T09:13:36.437153+0000","last_undegraded":"2023-10-08T09:13:36.437153+0000","last_fullsized":"2023-10-08T09:13:36.437153+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:04.317649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.450898+0000","last_change":"2023-10-08T09:13:37.450251+0000","last_active":"2023-10-08T09:13:37.450898+0000","last_peered":"2023-10-08T09:13:37.450898+0000","last_clean":"2023-10-08T09:13:37.450898+0000","last_became_active":"2023-10-08T09:13:37.449848+0000","last_became_peered":"2023-10-08T09:13:37.449848+0000","last_unstale":"2023-10-08T09:13:37.450898+0000","last_undegraded":"2023-10-08T09:13:37.450898+0000","last_fullsized":"2023-10-08T09:13:37.450898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:09:16.541785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.450381+0000","last_change":"2023-10-08T09:13:37.450381+0000","last_active":"2023-10-08T09:13:37.450381+0000","last_peered":"2023-10-08T09:13:37.450381+0000","last_clean":"2023-10-08T09:13:37.450381+0000","last_became_active":"2023-10-08T09:13:37.450044+0000","last_became_peered":"2023-10-08T09:13:37.450044+0000","last_unstale":"2023-10-08T09:13:37.450381+0000","last_undegraded":"2023-10-08T09:13:37.450381+0000","last_fullsized":"2023-10-08T09:13:37.450381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:03.203894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436269+0000","last_change":"2023-10-08T09:13:02.829027+0000","last_active":"2023-10-08T09:13:36.436269+0000","last_peered":"2023-10-08T09:13:36.436269+0000","last_clean":"2023-10-08T09:13:36.436269+0000","last_became_active":"2023-10-08T09:13:02.828714+0000","last_became_peered":"2023-10-08T09:13:02.828714+0000","last_unstale":"2023-10-08T09:13:36.436269+0000","last_undegraded":"2023-10-08T09:13:36.436269+0000","last_fullsized":"2023-10-08T09:13:36.436269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:13.121274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436239+0000","last_change":"2023-10-08T09:13:02.838640+0000","last_active":"2023-10-08T09:13:36.436239+0000","last_peered":"2023-10-08T09:13:36.436239+0000","last_clean":"2023-10-08T09:13:36.436239+0000","last_became_active":"2023-10-08T09:13:02.838302+0000","last_became_peered":"2023-10-08T09:13:02.838302+0000","last_unstale":"2023-10-08T09:13:36.436239+0000","last_undegraded":"2023-10-08T09:13:36.436239+0000","last_fullsized":"2023-10-08T09:13:36.436239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:54.268315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441384+0000","last_change":"2023-10-08T09:13:02.834027+0000","last_active":"2023-10-08T09:13:36.441384+0000","last_peered":"2023-10-08T09:13:36.441384+0000","last_clean":"2023-10-08T09:13:36.441384+0000","last_became_active":"2023-10-08T09:13:02.833733+0000","last_became_peered":"2023-10-08T09:13:02.833733+0000","last_unstale":"2023-10-08T09:13:36.441384+0000","last_undegraded":"2023-10-08T09:13:36.441384+0000","last_fullsized":"2023-10-08T09:13:36.441384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:03.644640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440991+0000","last_change":"2023-10-08T09:13:02.832040+0000","last_active":"2023-10-08T09:13:36.440991+0000","last_peered":"2023-10-08T09:13:36.440991+0000","last_clean":"2023-10-08T09:13:36.440991+0000","last_became_active":"2023-10-08T09:13:02.831752+0000","last_became_peered":"2023-10-08T09:13:02.831752+0000","last_unstale":"2023-10-08T09:13:36.440991+0000","last_undegraded":"2023-10-08T09:13:36.440991+0000","last_fullsized":"2023-10-08T09:13:36.440991+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:38.938703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440912+0000","last_change":"2023-10-08T09:13:02.830535+0000","last_active":"2023-10-08T09:13:36.440912+0000","last_peered":"2023-10-08T09:13:36.440912+0000","last_clean":"2023-10-08T09:13:36.440912+0000","last_became_active":"2023-10-08T09:13:02.830144+0000","last_became_peered":"2023-10-08T09:13:02.830144+0000","last_unstale":"2023-10-08T09:13:36.440912+0000","last_undegraded":"2023-10-08T09:13:36.440912+0000","last_fullsized":"2023-10-08T09:13:36.440912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:33.850815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443550+0000","last_change":"2023-10-08T09:13:02.830938+0000","last_active":"2023-10-08T09:13:36.443550+0000","last_peered":"2023-10-08T09:13:36.443550+0000","last_clean":"2023-10-08T09:13:36.443550+0000","last_became_active":"2023-10-08T09:13:02.830551+0000","last_became_peered":"2023-10-08T09:13:02.830551+0000","last_unstale":"2023-10-08T09:13:36.443550+0000","last_undegraded":"2023-10-08T09:13:36.443550+0000","last_fullsized":"2023-10-08T09:13:36.443550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:26.894614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442755+0000","last_change":"2023-10-08T09:13:07.871322+0000","last_active":"2023-10-08T09:13:36.442755+0000","last_peered":"2023-10-08T09:13:36.442755+0000","last_clean":"2023-10-08T09:13:36.442755+0000","last_became_active":"2023-10-08T09:13:07.870828+0000","last_became_peered":"2023-10-08T09:13:07.870828+0000","last_unstale":"2023-10-08T09:13:36.442755+0000","last_undegraded":"2023-10-08T09:13:36.442755+0000","last_fullsized":"2023-10-08T09:13:36.442755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:37.145705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:12.496533+0000","last_change":"2023-10-08T09:13:02.373510+0000","last_active":"2023-10-08T09:14:12.496533+0000","last_peered":"2023-10-08T09:14:12.496533+0000","last_clean":"2023-10-08T09:14:12.496533+0000","last_became_active":"2023-10-08T09:13:02.373174+0000","last_became_peered":"2023-10-08T09:13:02.373174+0000","last_unstale":"2023-10-08T09:14:12.496533+0000","last_undegraded":"2023-10-08T09:14:12.496533+0000","last_fullsized":"2023-10-08T09:14:12.496533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442547+0000","last_change":"2023-10-08T09:13:02.835195+0000","last_active":"2023-10-08T09:13:36.442547+0000","last_peered":"2023-10-08T09:13:36.442547+0000","last_clean":"2023-10-08T09:13:36.442547+0000","last_became_active":"2023-10-08T09:13:02.834893+0000","last_became_peered":"2023-10-08T09:13:02.834893+0000","last_unstale":"2023-10-08T09:13:36.442547+0000","last_undegraded":"2023-10-08T09:13:36.442547+0000","last_fullsized":"2023-10-08T09:13:36.442547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:35.835017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439374+0000","last_change":"2023-10-08T09:13:07.871418+0000","last_active":"2023-10-08T09:13:36.439374+0000","last_peered":"2023-10-08T09:13:36.439374+0000","last_clean":"2023-10-08T09:13:36.439374+0000","last_became_active":"2023-10-08T09:13:07.870961+0000","last_became_peered":"2023-10-08T09:13:07.870961+0000","last_unstale":"2023-10-08T09:13:36.439374+0000","last_undegraded":"2023-10-08T09:13:36.439374+0000","last_fullsized":"2023-10-08T09:13:36.439374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:31.723498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442572+0000","last_change":"2023-10-08T09:13:02.370728+0000","last_active":"2023-10-08T09:13:36.442572+0000","last_peered":"2023-10-08T09:13:36.442572+0000","last_clean":"2023-10-08T09:13:36.442572+0000","last_became_active":"2023-10-08T09:13:02.370396+0000","last_became_peered":"2023-10-08T09:13:02.370396+0000","last_unstale":"2023-10-08T09:13:36.442572+0000","last_undegraded":"2023-10-08T09:13:36.442572+0000","last_fullsized":"2023-10-08T09:13:36.442572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:32.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436716+0000","last_change":"2023-10-08T09:13:02.827823+0000","last_active":"2023-10-08T09:13:36.436716+0000","last_peered":"2023-10-08T09:13:36.436716+0000","last_clean":"2023-10-08T09:13:36.436716+0000","last_became_active":"2023-10-08T09:13:02.827417+0000","last_became_peered":"2023-10-08T09:13:02.827417+0000","last_unstale":"2023-10-08T09:13:36.436716+0000","last_undegraded":"2023-10-08T09:13:36.436716+0000","last_fullsized":"2023-10-08T09:13:36.436716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:38.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.455325+0000","last_change":"2023-10-08T09:13:37.455325+0000","last_active":"2023-10-08T09:13:37.455325+0000","last_peered":"2023-10-08T09:13:37.455325+0000","last_clean":"2023-10-08T09:13:37.455325+0000","last_became_active":"2023-10-08T09:13:37.454985+0000","last_became_peered":"2023-10-08T09:13:37.454985+0000","last_unstale":"2023-10-08T09:13:37.455325+0000","last_undegraded":"2023-10-08T09:13:37.455325+0000","last_fullsized":"2023-10-08T09:13:37.455325+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:13.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436867+0000","last_change":"2023-10-08T09:13:02.375244+0000","last_active":"2023-10-08T09:13:36.436867+0000","last_peered":"2023-10-08T09:13:36.436867+0000","last_clean":"2023-10-08T09:13:36.436867+0000","last_became_active":"2023-10-08T09:13:02.374526+0000","last_became_peered":"2023-10-08T09:13:02.374526+0000","last_unstale":"2023-10-08T09:13:36.436867+0000","last_undegraded":"2023-10-08T09:13:36.436867+0000","last_fullsized":"2023-10-08T09:13:36.436867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:36.380491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443641+0000","last_change":"2023-10-08T09:13:02.838373+0000","last_active":"2023-10-08T09:13:36.443641+0000","last_peered":"2023-10-08T09:13:36.443641+0000","last_clean":"2023-10-08T09:13:36.443641+0000","last_became_active":"2023-10-08T09:13:02.838100+0000","last_became_peered":"2023-10-08T09:13:02.838100+0000","last_unstale":"2023-10-08T09:13:36.443641+0000","last_undegraded":"2023-10-08T09:13:36.443641+0000","last_fullsized":"2023-10-08T09:13:36.443641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:59.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439494+0000","last_change":"2023-10-08T09:13:07.868929+0000","last_active":"2023-10-08T09:13:36.439494+0000","last_peered":"2023-10-08T09:13:36.439494+0000","last_clean":"2023-10-08T09:13:36.439494+0000","last_became_active":"2023-10-08T09:13:07.868451+0000","last_became_peered":"2023-10-08T09:13:07.868451+0000","last_unstale":"2023-10-08T09:13:36.439494+0000","last_undegraded":"2023-10-08T09:13:36.439494+0000","last_fullsized":"2023-10-08T09:13:36.439494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:36.540319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436511+0000","last_change":"2023-10-08T09:13:02.366404+0000","last_active":"2023-10-08T09:13:36.436511+0000","last_peered":"2023-10-08T09:13:36.436511+0000","last_clean":"2023-10-08T09:13:36.436511+0000","last_became_active":"2023-10-08T09:13:02.366025+0000","last_became_peered":"2023-10-08T09:13:02.366025+0000","last_unstale":"2023-10-08T09:13:36.436511+0000","last_undegraded":"2023-10-08T09:13:36.436511+0000","last_fullsized":"2023-10-08T09:13:36.436511+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:51.916565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443379+0000","last_change":"2023-10-08T09:13:02.832852+0000","last_active":"2023-10-08T09:13:36.443379+0000","last_peered":"2023-10-08T09:13:36.443379+0000","last_clean":"2023-10-08T09:13:36.443379+0000","last_became_active":"2023-10-08T09:13:02.832530+0000","last_became_peered":"2023-10-08T09:13:02.832530+0000","last_unstale":"2023-10-08T09:13:36.443379+0000","last_undegraded":"2023-10-08T09:13:36.443379+0000","last_fullsized":"2023-10-08T09:13:36.443379+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:20.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443286+0000","last_change":"2023-10-08T09:13:07.863864+0000","last_active":"2023-10-08T09:13:36.443286+0000","last_peered":"2023-10-08T09:13:36.443286+0000","last_clean":"2023-10-08T09:13:36.443286+0000","last_became_active":"2023-10-08T09:13:07.863509+0000","last_became_peered":"2023-10-08T09:13:07.863509+0000","last_unstale":"2023-10-08T09:13:36.443286+0000","last_undegraded":"2023-10-08T09:13:36.443286+0000","last_fullsized":"2023-10-08T09:13:36.443286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:22.933327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:12.473899+0000","last_change":"2023-10-08T09:13:02.374215+0000","last_active":"2023-10-08T09:14:12.473899+0000","last_peered":"2023-10-08T09:14:12.473899+0000","last_clean":"2023-10-08T09:14:12.473899+0000","last_became_active":"2023-10-08T09:13:02.373912+0000","last_became_peered":"2023-10-08T09:13:02.373912+0000","last_unstale":"2023-10-08T09:14:12.473899+0000","last_undegraded":"2023-10-08T09:14:12.473899+0000","last_fullsized":"2023-10-08T09:14:12.473899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:18.315244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436499+0000","last_change":"2023-10-08T09:13:02.830651+0000","last_active":"2023-10-08T09:13:36.436499+0000","last_peered":"2023-10-08T09:13:36.436499+0000","last_clean":"2023-10-08T09:13:36.436499+0000","last_became_active":"2023-10-08T09:13:02.830324+0000","last_became_peered":"2023-10-08T09:13:02.830324+0000","last_unstale":"2023-10-08T09:13:36.436499+0000","last_undegraded":"2023-10-08T09:13:36.436499+0000","last_fullsized":"2023-10-08T09:13:36.436499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:02.143794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.454054+0000","last_change":"2023-10-08T09:13:37.454054+0000","last_active":"2023-10-08T09:13:37.454054+0000","last_peered":"2023-10-08T09:13:37.454054+0000","last_clean":"2023-10-08T09:13:37.454054+0000","last_became_active":"2023-10-08T09:13:37.453482+0000","last_became_peered":"2023-10-08T09:13:37.453482+0000","last_unstale":"2023-10-08T09:13:37.454054+0000","last_undegraded":"2023-10-08T09:13:37.454054+0000","last_fullsized":"2023-10-08T09:13:37.454054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:31.550837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443108+0000","last_change":"2023-10-08T09:13:02.370334+0000","last_active":"2023-10-08T09:13:36.443108+0000","last_peered":"2023-10-08T09:13:36.443108+0000","last_clean":"2023-10-08T09:13:36.443108+0000","last_became_active":"2023-10-08T09:13:02.370068+0000","last_became_peered":"2023-10-08T09:13:02.370068+0000","last_unstale":"2023-10-08T09:13:36.443108+0000","last_undegraded":"2023-10-08T09:13:36.443108+0000","last_fullsized":"2023-10-08T09:13:36.443108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.962779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445562+0000","last_change":"2023-10-08T09:13:02.832346+0000","last_active":"2023-10-08T09:13:36.445562+0000","last_peered":"2023-10-08T09:13:36.445562+0000","last_clean":"2023-10-08T09:13:36.445562+0000","last_became_active":"2023-10-08T09:13:02.832059+0000","last_became_peered":"2023-10-08T09:13:02.832059+0000","last_unstale":"2023-10-08T09:13:36.445562+0000","last_undegraded":"2023-10-08T09:13:36.445562+0000","last_fullsized":"2023-10-08T09:13:36.445562+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:30.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440849+0000","last_change":"2023-10-08T09:13:07.863466+0000","last_active":"2023-10-08T09:13:36.440849+0000","last_peered":"2023-10-08T09:13:36.440849+0000","last_clean":"2023-10-08T09:13:36.440849+0000","last_became_active":"2023-10-08T09:13:07.863015+0000","last_became_peered":"2023-10-08T09:13:07.863015+0000","last_unstale":"2023-10-08T09:13:36.440849+0000","last_undegraded":"2023-10-08T09:13:36.440849+0000","last_fullsized":"2023-10-08T09:13:36.440849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:30.316625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445901+0000","last_change":"2023-10-08T09:13:02.369444+0000","last_active":"2023-10-08T09:13:36.445901+0000","last_peered":"2023-10-08T09:13:36.445901+0000","last_clean":"2023-10-08T09:13:36.445901+0000","last_became_active":"2023-10-08T09:13:02.369142+0000","last_became_peered":"2023-10-08T09:13:02.369142+0000","last_unstale":"2023-10-08T09:13:36.445901+0000","last_undegraded":"2023-10-08T09:13:36.445901+0000","last_fullsized":"2023-10-08T09:13:36.445901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:28.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441755+0000","last_change":"2023-10-08T09:13:02.836125+0000","last_active":"2023-10-08T09:13:36.441755+0000","last_peered":"2023-10-08T09:13:36.441755+0000","last_clean":"2023-10-08T09:13:36.441755+0000","last_became_active":"2023-10-08T09:13:02.835809+0000","last_became_peered":"2023-10-08T09:13:02.835809+0000","last_unstale":"2023-10-08T09:13:36.441755+0000","last_undegraded":"2023-10-08T09:13:36.441755+0000","last_fullsized":"2023-10-08T09:13:36.441755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:15.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436500+0000","last_change":"2023-10-08T09:13:07.861633+0000","last_active":"2023-10-08T09:13:36.436500+0000","last_peered":"2023-10-08T09:13:36.436500+0000","last_clean":"2023-10-08T09:13:36.436500+0000","last_became_active":"2023-10-08T09:13:07.861168+0000","last_became_peered":"2023-10-08T09:13:07.861168+0000","last_unstale":"2023-10-08T09:13:36.436500+0000","last_undegraded":"2023-10-08T09:13:36.436500+0000","last_fullsized":"2023-10-08T09:13:36.436500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:47.283892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443337+0000","last_change":"2023-10-08T09:13:02.365703+0000","last_active":"2023-10-08T09:13:36.443337+0000","last_peered":"2023-10-08T09:13:36.443337+0000","last_clean":"2023-10-08T09:13:36.443337+0000","last_became_active":"2023-10-08T09:13:02.365421+0000","last_became_peered":"2023-10-08T09:13:02.365421+0000","last_unstale":"2023-10-08T09:13:36.443337+0000","last_undegraded":"2023-10-08T09:13:36.443337+0000","last_fullsized":"2023-10-08T09:13:36.443337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:43.124511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439111+0000","last_change":"2023-10-08T09:13:02.840941+0000","last_active":"2023-10-08T09:13:36.439111+0000","last_peered":"2023-10-08T09:13:36.439111+0000","last_clean":"2023-10-08T09:13:36.439111+0000","last_became_active":"2023-10-08T09:13:02.840605+0000","last_became_peered":"2023-10-08T09:13:02.840605+0000","last_unstale":"2023-10-08T09:13:36.439111+0000","last_undegraded":"2023-10-08T09:13:36.439111+0000","last_fullsized":"2023-10-08T09:13:36.439111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:24:13.417028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442140+0000","last_change":"2023-10-08T09:13:07.868950+0000","last_active":"2023-10-08T09:13:36.442140+0000","last_peered":"2023-10-08T09:13:36.442140+0000","last_clean":"2023-10-08T09:13:36.442140+0000","last_became_active":"2023-10-08T09:13:07.868542+0000","last_became_peered":"2023-10-08T09:13:07.868542+0000","last_unstale":"2023-10-08T09:13:36.442140+0000","last_undegraded":"2023-10-08T09:13:36.442140+0000","last_fullsized":"2023-10-08T09:13:36.442140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:11.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:13.474769+0000","last_change":"2023-10-08T09:13:02.376188+0000","last_active":"2023-10-08T09:14:13.474769+0000","last_peered":"2023-10-08T09:14:13.474769+0000","last_clean":"2023-10-08T09:14:13.474769+0000","last_became_active":"2023-10-08T09:13:02.375914+0000","last_became_peered":"2023-10-08T09:13:02.375914+0000","last_unstale":"2023-10-08T09:14:13.474769+0000","last_undegraded":"2023-10-08T09:14:13.474769+0000","last_fullsized":"2023-10-08T09:14:13.474769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:10.744085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442281+0000","last_change":"2023-10-08T09:13:02.833429+0000","last_active":"2023-10-08T09:13:36.442281+0000","last_peered":"2023-10-08T09:13:36.442281+0000","last_clean":"2023-10-08T09:13:36.442281+0000","last_became_active":"2023-10-08T09:13:02.832792+0000","last_became_peered":"2023-10-08T09:13:02.832792+0000","last_unstale":"2023-10-08T09:13:36.442281+0000","last_undegraded":"2023-10-08T09:13:36.442281+0000","last_fullsized":"2023-10-08T09:13:36.442281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:24.563447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439055+0000","last_change":"2023-10-08T09:13:07.862688+0000","last_active":"2023-10-08T09:13:36.439055+0000","last_peered":"2023-10-08T09:13:36.439055+0000","last_clean":"2023-10-08T09:13:36.439055+0000","last_became_active":"2023-10-08T09:13:07.862197+0000","last_became_peered":"2023-10-08T09:13:07.862197+0000","last_unstale":"2023-10-08T09:13:36.439055+0000","last_undegraded":"2023-10-08T09:13:36.439055+0000","last_fullsized":"2023-10-08T09:13:36.439055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:56.792045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437118+0000","last_change":"2023-10-08T09:13:02.380283+0000","last_active":"2023-10-08T09:13:36.437118+0000","last_peered":"2023-10-08T09:13:36.437118+0000","last_clean":"2023-10-08T09:13:36.437118+0000","last_became_active":"2023-10-08T09:13:02.379844+0000","last_became_peered":"2023-10-08T09:13:02.379844+0000","last_unstale":"2023-10-08T09:13:36.437118+0000","last_undegraded":"2023-10-08T09:13:36.437118+0000","last_fullsized":"2023-10-08T09:13:36.437118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:18.162437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436786+0000","last_change":"2023-10-08T09:13:02.839211+0000","last_active":"2023-10-08T09:13:36.436786+0000","last_peered":"2023-10-08T09:13:36.436786+0000","last_clean":"2023-10-08T09:13:36.436786+0000","last_became_active":"2023-10-08T09:13:02.838968+0000","last_became_peered":"2023-10-08T09:13:02.838968+0000","last_unstale":"2023-10-08T09:13:36.436786+0000","last_undegraded":"2023-10-08T09:13:36.436786+0000","last_fullsized":"2023-10-08T09:13:36.436786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:24:02.541749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"86'1","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.457701+0000","last_change":"2023-10-08T09:13:37.457701+0000","last_active":"2023-10-08T09:13:37.457701+0000","last_peered":"2023-10-08T09:13:37.457701+0000","last_clean":"2023-10-08T09:13:37.457701+0000","last_became_active":"2023-10-08T09:13:37.452309+0000","last_became_peered":"2023-10-08T09:13:37.452309+0000","last_unstale":"2023-10-08T09:13:37.457701+0000","last_undegraded":"2023-10-08T09:13:37.457701+0000","last_fullsized":"2023-10-08T09:13:37.457701+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:14.898542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443267+0000","last_change":"2023-10-08T09:13:02.377027+0000","last_active":"2023-10-08T09:13:36.443267+0000","last_peered":"2023-10-08T09:13:36.443267+0000","last_clean":"2023-10-08T09:13:36.443267+0000","last_became_active":"2023-10-08T09:13:02.376746+0000","last_became_peered":"2023-10-08T09:13:02.376746+0000","last_unstale":"2023-10-08T09:13:36.443267+0000","last_undegraded":"2023-10-08T09:13:36.443267+0000","last_fullsized":"2023-10-08T09:13:36.443267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:19.062296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443725+0000","last_change":"2023-10-08T09:13:02.837279+0000","last_active":"2023-10-08T09:13:36.443725+0000","last_peered":"2023-10-08T09:13:36.443725+0000","last_clean":"2023-10-08T09:13:36.443725+0000","last_became_active":"2023-10-08T09:13:02.836966+0000","last_became_peered":"2023-10-08T09:13:02.836966+0000","last_unstale":"2023-10-08T09:13:36.443725+0000","last_undegraded":"2023-10-08T09:13:36.443725+0000","last_fullsized":"2023-10-08T09:13:36.443725+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:25.462835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439420+0000","last_change":"2023-10-08T09:13:07.865538+0000","last_active":"2023-10-08T09:13:36.439420+0000","last_peered":"2023-10-08T09:13:36.439420+0000","last_clean":"2023-10-08T09:13:36.439420+0000","last_became_active":"2023-10-08T09:13:07.865118+0000","last_became_peered":"2023-10-08T09:13:07.865118+0000","last_unstale":"2023-10-08T09:13:36.439420+0000","last_undegraded":"2023-10-08T09:13:36.439420+0000","last_fullsized":"2023-10-08T09:13:36.439420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:32.167439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":148,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:13.479496+0000","last_change":"2023-10-08T09:13:02.378173+0000","last_active":"2023-10-08T09:14:13.479496+0000","last_peered":"2023-10-08T09:14:13.479496+0000","last_clean":"2023-10-08T09:14:13.479496+0000","last_became_active":"2023-10-08T09:13:02.377880+0000","last_became_peered":"2023-10-08T09:13:02.377880+0000","last_unstale":"2023-10-08T09:14:13.479496+0000","last_undegraded":"2023-10-08T09:14:13.479496+0000","last_fullsized":"2023-10-08T09:14:13.479496+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58583,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":104,"num_write":36,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436695+0000","last_change":"2023-10-08T09:13:02.829874+0000","last_active":"2023-10-08T09:13:36.436695+0000","last_peered":"2023-10-08T09:13:36.436695+0000","last_clean":"2023-10-08T09:13:36.436695+0000","last_became_active":"2023-10-08T09:13:02.829552+0000","last_became_peered":"2023-10-08T09:13:02.829552+0000","last_unstale":"2023-10-08T09:13:36.436695+0000","last_undegraded":"2023-10-08T09:13:36.436695+0000","last_fullsized":"2023-10-08T09:13:36.436695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:29.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443746+0000","last_change":"2023-10-08T09:13:07.869661+0000","last_active":"2023-10-08T09:13:36.443746+0000","last_peered":"2023-10-08T09:13:36.443746+0000","last_clean":"2023-10-08T09:13:36.443746+0000","last_became_active":"2023-10-08T09:13:07.869236+0000","last_became_peered":"2023-10-08T09:13:07.869236+0000","last_unstale":"2023-10-08T09:13:36.443746+0000","last_undegraded":"2023-10-08T09:13:36.443746+0000","last_fullsized":"2023-10-08T09:13:36.443746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:26.354461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443292+0000","last_change":"2023-10-08T09:13:02.376941+0000","last_active":"2023-10-08T09:13:36.443292+0000","last_peered":"2023-10-08T09:13:36.443292+0000","last_clean":"2023-10-08T09:13:36.443292+0000","last_became_active":"2023-10-08T09:13:02.375810+0000","last_became_peered":"2023-10-08T09:13:02.375810+0000","last_unstale":"2023-10-08T09:13:36.443292+0000","last_undegraded":"2023-10-08T09:13:36.443292+0000","last_fullsized":"2023-10-08T09:13:36.443292+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:17.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445839+0000","last_change":"2023-10-08T09:13:02.836781+0000","last_active":"2023-10-08T09:13:36.445839+0000","last_peered":"2023-10-08T09:13:36.445839+0000","last_clean":"2023-10-08T09:13:36.445839+0000","last_became_active":"2023-10-08T09:13:02.836493+0000","last_became_peered":"2023-10-08T09:13:02.836493+0000","last_unstale":"2023-10-08T09:13:36.445839+0000","last_undegraded":"2023-10-08T09:13:36.445839+0000","last_fullsized":"2023-10-08T09:13:36.445839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:03.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440866+0000","last_change":"2023-10-08T09:13:07.859104+0000","last_active":"2023-10-08T09:13:36.440866+0000","last_peered":"2023-10-08T09:13:36.440866+0000","last_clean":"2023-10-08T09:13:36.440866+0000","last_became_active":"2023-10-08T09:13:07.858691+0000","last_became_peered":"2023-10-08T09:13:07.858691+0000","last_unstale":"2023-10-08T09:13:36.440866+0000","last_undegraded":"2023-10-08T09:13:36.440866+0000","last_fullsized":"2023-10-08T09:13:36.440866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:33.256044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443857+0000","last_change":"2023-10-08T09:13:02.364472+0000","last_active":"2023-10-08T09:13:36.443857+0000","last_peered":"2023-10-08T09:13:36.443857+0000","last_clean":"2023-10-08T09:13:36.443857+0000","last_became_active":"2023-10-08T09:13:02.364130+0000","last_became_peered":"2023-10-08T09:13:02.364130+0000","last_unstale":"2023-10-08T09:13:36.443857+0000","last_undegraded":"2023-10-08T09:13:36.443857+0000","last_fullsized":"2023-10-08T09:13:36.443857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:18.156186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437151+0000","last_change":"2023-10-08T09:13:02.836119+0000","last_active":"2023-10-08T09:13:36.437151+0000","last_peered":"2023-10-08T09:13:36.437151+0000","last_clean":"2023-10-08T09:13:36.437151+0000","last_became_active":"2023-10-08T09:13:02.835548+0000","last_became_peered":"2023-10-08T09:13:02.835548+0000","last_unstale":"2023-10-08T09:13:36.437151+0000","last_undegraded":"2023-10-08T09:13:36.437151+0000","last_fullsized":"2023-10-08T09:13:36.437151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:34.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440444+0000","last_change":"2023-10-08T09:13:07.871447+0000","last_active":"2023-10-08T09:13:36.440444+0000","last_peered":"2023-10-08T09:13:36.440444+0000","last_clean":"2023-10-08T09:13:36.440444+0000","last_became_active":"2023-10-08T09:13:07.871018+0000","last_became_peered":"2023-10-08T09:13:07.871018+0000","last_unstale":"2023-10-08T09:13:36.440444+0000","last_undegraded":"2023-10-08T09:13:36.440444+0000","last_fullsized":"2023-10-08T09:13:36.440444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:34.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:16.479229+0000","last_change":"2023-10-08T09:13:02.378954+0000","last_active":"2023-10-08T09:14:16.479229+0000","last_peered":"2023-10-08T09:14:16.479229+0000","last_clean":"2023-10-08T09:14:16.479229+0000","last_became_active":"2023-10-08T09:13:02.378635+0000","last_became_peered":"2023-10-08T09:13:02.378635+0000","last_unstale":"2023-10-08T09:14:16.479229+0000","last_undegraded":"2023-10-08T09:14:16.479229+0000","last_fullsized":"2023-10-08T09:14:16.479229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:09.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436789+0000","last_change":"2023-10-08T09:13:02.828732+0000","last_active":"2023-10-08T09:13:36.436789+0000","last_peered":"2023-10-08T09:13:36.436789+0000","last_clean":"2023-10-08T09:13:36.436789+0000","last_became_active":"2023-10-08T09:13:02.828449+0000","last_became_peered":"2023-10-08T09:13:02.828449+0000","last_unstale":"2023-10-08T09:13:36.436789+0000","last_undegraded":"2023-10-08T09:13:36.436789+0000","last_fullsized":"2023-10-08T09:13:36.436789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:09.883908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446290+0000","last_change":"2023-10-08T09:13:07.869610+0000","last_active":"2023-10-08T09:13:36.446290+0000","last_peered":"2023-10-08T09:13:36.446290+0000","last_clean":"2023-10-08T09:13:36.446290+0000","last_became_active":"2023-10-08T09:13:07.869225+0000","last_became_peered":"2023-10-08T09:13:07.869225+0000","last_unstale":"2023-10-08T09:13:36.446290+0000","last_undegraded":"2023-10-08T09:13:36.446290+0000","last_fullsized":"2023-10-08T09:13:36.446290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:35.225452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436233+0000","last_change":"2023-10-08T09:13:02.375658+0000","last_active":"2023-10-08T09:13:36.436233+0000","last_peered":"2023-10-08T09:13:36.436233+0000","last_clean":"2023-10-08T09:13:36.436233+0000","last_became_active":"2023-10-08T09:13:02.375383+0000","last_became_peered":"2023-10-08T09:13:02.375383+0000","last_unstale":"2023-10-08T09:13:36.436233+0000","last_undegraded":"2023-10-08T09:13:36.436233+0000","last_fullsized":"2023-10-08T09:13:36.436233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:52.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.455645+0000","last_change":"2023-10-08T09:13:37.455645+0000","last_active":"2023-10-08T09:13:37.455645+0000","last_peered":"2023-10-08T09:13:37.455645+0000","last_clean":"2023-10-08T09:13:37.455645+0000","last_became_active":"2023-10-08T09:13:37.455207+0000","last_became_peered":"2023-10-08T09:13:37.455207+0000","last_unstale":"2023-10-08T09:13:37.455645+0000","last_undegraded":"2023-10-08T09:13:37.455645+0000","last_fullsized":"2023-10-08T09:13:37.455645+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:18.021608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445968+0000","last_change":"2023-10-08T09:13:07.871441+0000","last_active":"2023-10-08T09:13:36.445968+0000","last_peered":"2023-10-08T09:13:36.445968+0000","last_clean":"2023-10-08T09:13:36.445968+0000","last_became_active":"2023-10-08T09:13:07.870968+0000","last_became_peered":"2023-10-08T09:13:07.870968+0000","last_unstale":"2023-10-08T09:13:36.445968+0000","last_undegraded":"2023-10-08T09:13:36.445968+0000","last_fullsized":"2023-10-08T09:13:36.445968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:48.746147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436858+0000","last_change":"2023-10-08T09:13:02.372514+0000","last_active":"2023-10-08T09:13:36.436858+0000","last_peered":"2023-10-08T09:13:36.436858+0000","last_clean":"2023-10-08T09:13:36.436858+0000","last_became_active":"2023-10-08T09:13:02.372181+0000","last_became_peered":"2023-10-08T09:13:02.372181+0000","last_unstale":"2023-10-08T09:13:36.436858+0000","last_undegraded":"2023-10-08T09:13:36.436858+0000","last_fullsized":"2023-10-08T09:13:36.436858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:28.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440816+0000","last_change":"2023-10-08T09:13:02.835220+0000","last_active":"2023-10-08T09:13:36.440816+0000","last_peered":"2023-10-08T09:13:36.440816+0000","last_clean":"2023-10-08T09:13:36.440816+0000","last_became_active":"2023-10-08T09:13:02.834914+0000","last_became_peered":"2023-10-08T09:13:02.834914+0000","last_unstale":"2023-10-08T09:13:36.440816+0000","last_undegraded":"2023-10-08T09:13:36.440816+0000","last_fullsized":"2023-10-08T09:13:36.440816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:37.447418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436452+0000","last_change":"2023-10-08T09:13:08.096146+0000","last_active":"2023-10-08T09:13:36.436452+0000","last_peered":"2023-10-08T09:13:36.436452+0000","last_clean":"2023-10-08T09:13:36.436452+0000","last_became_active":"2023-10-08T09:13:08.095703+0000","last_became_peered":"2023-10-08T09:13:08.095703+0000","last_unstale":"2023-10-08T09:13:36.436452+0000","last_undegraded":"2023-10-08T09:13:36.436452+0000","last_fullsized":"2023-10-08T09:13:36.436452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:37.511094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440338+0000","last_change":"2023-10-08T09:13:02.372836+0000","last_active":"2023-10-08T09:13:36.440338+0000","last_peered":"2023-10-08T09:13:36.440338+0000","last_clean":"2023-10-08T09:13:36.440338+0000","last_became_active":"2023-10-08T09:13:02.372488+0000","last_became_peered":"2023-10-08T09:13:02.372488+0000","last_unstale":"2023-10-08T09:13:36.440338+0000","last_undegraded":"2023-10-08T09:13:36.440338+0000","last_fullsized":"2023-10-08T09:13:36.440338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:19.269539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443814+0000","last_change":"2023-10-08T09:13:02.827306+0000","last_active":"2023-10-08T09:13:36.443814+0000","last_peered":"2023-10-08T09:13:36.443814+0000","last_clean":"2023-10-08T09:13:36.443814+0000","last_became_active":"2023-10-08T09:13:02.826965+0000","last_became_peered":"2023-10-08T09:13:02.826965+0000","last_unstale":"2023-10-08T09:13:36.443814+0000","last_undegraded":"2023-10-08T09:13:36.443814+0000","last_fullsized":"2023-10-08T09:13:36.443814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:27.513860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446236+0000","last_change":"2023-10-08T09:13:07.862637+0000","last_active":"2023-10-08T09:13:36.446236+0000","last_peered":"2023-10-08T09:13:36.446236+0000","last_clean":"2023-10-08T09:13:36.446236+0000","last_became_active":"2023-10-08T09:13:07.862117+0000","last_became_peered":"2023-10-08T09:13:07.862117+0000","last_unstale":"2023-10-08T09:13:36.446236+0000","last_undegraded":"2023-10-08T09:13:36.446236+0000","last_fullsized":"2023-10-08T09:13:36.446236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:14.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"86'6","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:14.499880+0000","last_change":"2023-10-08T09:13:02.379585+0000","last_active":"2023-10-08T09:14:14.499880+0000","last_peered":"2023-10-08T09:14:14.499880+0000","last_clean":"2023-10-08T09:14:14.499880+0000","last_became_active":"2023-10-08T09:13:02.379324+0000","last_became_peered":"2023-10-08T09:13:02.379324+0000","last_unstale":"2023-10-08T09:14:14.499880+0000","last_undegraded":"2023-10-08T09:14:14.499880+0000","last_fullsized":"2023-10-08T09:14:14.499880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:31.609558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443748+0000","last_change":"2023-10-08T09:13:02.833416+0000","last_active":"2023-10-08T09:13:36.443748+0000","last_peered":"2023-10-08T09:13:36.443748+0000","last_clean":"2023-10-08T09:13:36.443748+0000","last_became_active":"2023-10-08T09:13:02.833040+0000","last_became_peered":"2023-10-08T09:13:02.833040+0000","last_unstale":"2023-10-08T09:13:36.443748+0000","last_undegraded":"2023-10-08T09:13:36.443748+0000","last_fullsized":"2023-10-08T09:13:36.443748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:44.480445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446160+0000","last_change":"2023-10-08T09:13:08.095723+0000","last_active":"2023-10-08T09:13:36.446160+0000","last_peered":"2023-10-08T09:13:36.446160+0000","last_clean":"2023-10-08T09:13:36.446160+0000","last_became_active":"2023-10-08T09:13:08.095271+0000","last_became_peered":"2023-10-08T09:13:08.095271+0000","last_unstale":"2023-10-08T09:13:36.446160+0000","last_undegraded":"2023-10-08T09:13:36.446160+0000","last_fullsized":"2023-10-08T09:13:36.446160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:04.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443625+0000","last_change":"2023-10-08T09:13:02.829663+0000","last_active":"2023-10-08T09:13:36.443625+0000","last_peered":"2023-10-08T09:13:36.443625+0000","last_clean":"2023-10-08T09:13:36.443625+0000","last_became_active":"2023-10-08T09:13:02.829354+0000","last_became_peered":"2023-10-08T09:13:02.829354+0000","last_unstale":"2023-10-08T09:13:36.443625+0000","last_undegraded":"2023-10-08T09:13:36.443625+0000","last_fullsized":"2023-10-08T09:13:36.443625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:55.332346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:13.473976+0000","last_change":"2023-10-08T09:13:02.372114+0000","last_active":"2023-10-08T09:14:13.473976+0000","last_peered":"2023-10-08T09:14:13.473976+0000","last_clean":"2023-10-08T09:14:13.473976+0000","last_became_active":"2023-10-08T09:13:02.371603+0000","last_became_peered":"2023-10-08T09:13:02.371603+0000","last_unstale":"2023-10-08T09:14:13.473976+0000","last_undegraded":"2023-10-08T09:14:13.473976+0000","last_fullsized":"2023-10-08T09:14:13.473976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:39.078307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437022+0000","last_change":"2023-10-08T09:13:07.863492+0000","last_active":"2023-10-08T09:13:36.437022+0000","last_peered":"2023-10-08T09:13:36.437022+0000","last_clean":"2023-10-08T09:13:36.437022+0000","last_became_active":"2023-10-08T09:13:07.862803+0000","last_became_peered":"2023-10-08T09:13:07.862803+0000","last_unstale":"2023-10-08T09:13:36.437022+0000","last_undegraded":"2023-10-08T09:13:36.437022+0000","last_fullsized":"2023-10-08T09:13:36.437022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:39.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442781+0000","last_change":"2023-10-08T09:13:02.836596+0000","last_active":"2023-10-08T09:13:36.442781+0000","last_peered":"2023-10-08T09:13:36.442781+0000","last_clean":"2023-10-08T09:13:36.442781+0000","last_became_active":"2023-10-08T09:13:02.836283+0000","last_became_peered":"2023-10-08T09:13:02.836283+0000","last_unstale":"2023-10-08T09:13:36.442781+0000","last_undegraded":"2023-10-08T09:13:36.442781+0000","last_fullsized":"2023-10-08T09:13:36.442781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:02.393575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440466+0000","last_change":"2023-10-08T09:13:02.379516+0000","last_active":"2023-10-08T09:13:36.440466+0000","last_peered":"2023-10-08T09:13:36.440466+0000","last_clean":"2023-10-08T09:13:36.440466+0000","last_became_active":"2023-10-08T09:13:02.379229+0000","last_became_peered":"2023-10-08T09:13:02.379229+0000","last_unstale":"2023-10-08T09:13:36.440466+0000","last_undegraded":"2023-10-08T09:13:36.440466+0000","last_fullsized":"2023-10-08T09:13:36.440466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:55.294651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":312,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436665+0000","last_change":"2023-10-08T09:12:02.620769+0000","last_active":"2023-10-08T09:13:36.436665+0000","last_peered":"2023-10-08T09:13:36.436665+0000","last_clean":"2023-10-08T09:13:36.436665+0000","last_became_active":"2023-10-08T09:12:02.620388+0000","last_became_peered":"2023-10-08T09:12:02.620388+0000","last_unstale":"2023-10-08T09:13:36.436665+0000","last_undegraded":"2023-10-08T09:13:36.436665+0000","last_fullsized":"2023-10-08T09:13:36.436665+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_clean_scrub_stamp":"2023-10-08T09:09:10.746288+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:55:35.713060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.444776+0000","last_change":"2023-10-08T09:13:08.096318+0000","last_active":"2023-10-08T09:13:36.444776+0000","last_peered":"2023-10-08T09:13:36.444776+0000","last_clean":"2023-10-08T09:13:36.444776+0000","last_became_active":"2023-10-08T09:13:08.095774+0000","last_became_peered":"2023-10-08T09:13:08.095774+0000","last_unstale":"2023-10-08T09:13:36.444776+0000","last_undegraded":"2023-10-08T09:13:36.444776+0000","last_fullsized":"2023-10-08T09:13:36.444776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:53.089468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.445283+0000","last_change":"2023-10-08T09:13:02.835167+0000","last_active":"2023-10-08T09:13:36.445283+0000","last_peered":"2023-10-08T09:13:36.445283+0000","last_clean":"2023-10-08T09:13:36.445283+0000","last_became_active":"2023-10-08T09:13:02.834850+0000","last_became_peered":"2023-10-08T09:13:02.834850+0000","last_unstale":"2023-10-08T09:13:36.445283+0000","last_undegraded":"2023-10-08T09:13:36.445283+0000","last_fullsized":"2023-10-08T09:13:36.445283+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:19.422372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436727+0000","last_change":"2023-10-08T09:13:02.377746+0000","last_active":"2023-10-08T09:13:36.436727+0000","last_peered":"2023-10-08T09:13:36.436727+0000","last_clean":"2023-10-08T09:13:36.436727+0000","last_became_active":"2023-10-08T09:13:02.377422+0000","last_became_peered":"2023-10-08T09:13:02.377422+0000","last_unstale":"2023-10-08T09:13:36.436727+0000","last_undegraded":"2023-10-08T09:13:36.436727+0000","last_fullsized":"2023-10-08T09:13:36.436727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:13.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440674+0000","last_change":"2023-10-08T09:13:07.864678+0000","last_active":"2023-10-08T09:13:36.440674+0000","last_peered":"2023-10-08T09:13:36.440674+0000","last_clean":"2023-10-08T09:13:36.440674+0000","last_became_active":"2023-10-08T09:13:07.864226+0000","last_became_peered":"2023-10-08T09:13:07.864226+0000","last_unstale":"2023-10-08T09:13:36.440674+0000","last_undegraded":"2023-10-08T09:13:36.440674+0000","last_fullsized":"2023-10-08T09:13:36.440674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:43:51.179853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436798+0000","last_change":"2023-10-08T09:13:02.370014+0000","last_active":"2023-10-08T09:13:36.436798+0000","last_peered":"2023-10-08T09:13:36.436798+0000","last_clean":"2023-10-08T09:13:36.436798+0000","last_became_active":"2023-10-08T09:13:02.369704+0000","last_became_peered":"2023-10-08T09:13:02.369704+0000","last_unstale":"2023-10-08T09:13:36.436798+0000","last_undegraded":"2023-10-08T09:13:36.436798+0000","last_fullsized":"2023-10-08T09:13:36.436798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:28.902274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446349+0000","last_change":"2023-10-08T09:13:02.833859+0000","last_active":"2023-10-08T09:13:36.446349+0000","last_peered":"2023-10-08T09:13:36.446349+0000","last_clean":"2023-10-08T09:13:36.446349+0000","last_became_active":"2023-10-08T09:13:02.833546+0000","last_became_peered":"2023-10-08T09:13:02.833546+0000","last_unstale":"2023-10-08T09:13:36.446349+0000","last_undegraded":"2023-10-08T09:13:36.446349+0000","last_fullsized":"2023-10-08T09:13:36.446349+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:35.534588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.441671+0000","last_change":"2023-10-08T09:13:08.095622+0000","last_active":"2023-10-08T09:13:36.441671+0000","last_peered":"2023-10-08T09:13:36.441671+0000","last_clean":"2023-10-08T09:13:36.441671+0000","last_became_active":"2023-10-08T09:13:08.095143+0000","last_became_peered":"2023-10-08T09:13:08.095143+0000","last_unstale":"2023-10-08T09:13:36.441671+0000","last_undegraded":"2023-10-08T09:13:36.441671+0000","last_fullsized":"2023-10-08T09:13:36.441671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:46.738102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.459534+0000","last_change":"2023-10-08T09:13:37.459185+0000","last_active":"2023-10-08T09:13:37.459534+0000","last_peered":"2023-10-08T09:13:37.459534+0000","last_clean":"2023-10-08T09:13:37.459534+0000","last_became_active":"2023-10-08T09:13:37.451105+0000","last_became_peered":"2023-10-08T09:13:37.451105+0000","last_unstale":"2023-10-08T09:13:37.459534+0000","last_undegraded":"2023-10-08T09:13:37.459534+0000","last_fullsized":"2023-10-08T09:13:37.459534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.768269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442434+0000","last_change":"2023-10-08T09:13:02.834053+0000","last_active":"2023-10-08T09:13:36.442434+0000","last_peered":"2023-10-08T09:13:36.442434+0000","last_clean":"2023-10-08T09:13:36.442434+0000","last_became_active":"2023-10-08T09:13:02.833707+0000","last_became_peered":"2023-10-08T09:13:02.833707+0000","last_unstale":"2023-10-08T09:13:36.442434+0000","last_undegraded":"2023-10-08T09:13:36.442434+0000","last_fullsized":"2023-10-08T09:13:36.442434+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:31.372003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442555+0000","last_change":"2023-10-08T09:13:07.855734+0000","last_active":"2023-10-08T09:13:36.442555+0000","last_peered":"2023-10-08T09:13:36.442555+0000","last_clean":"2023-10-08T09:13:36.442555+0000","last_became_active":"2023-10-08T09:13:07.855313+0000","last_became_peered":"2023-10-08T09:13:07.855313+0000","last_unstale":"2023-10-08T09:13:36.442555+0000","last_undegraded":"2023-10-08T09:13:36.442555+0000","last_fullsized":"2023-10-08T09:13:36.442555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:15.207337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440758+0000","last_change":"2023-10-08T09:13:02.371319+0000","last_active":"2023-10-08T09:13:36.440758+0000","last_peered":"2023-10-08T09:13:36.440758+0000","last_clean":"2023-10-08T09:13:36.440758+0000","last_became_active":"2023-10-08T09:13:02.371049+0000","last_became_peered":"2023-10-08T09:13:02.371049+0000","last_unstale":"2023-10-08T09:13:36.440758+0000","last_undegraded":"2023-10-08T09:13:36.440758+0000","last_fullsized":"2023-10-08T09:13:36.440758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:28.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437178+0000","last_change":"2023-10-08T09:13:02.838682+0000","last_active":"2023-10-08T09:13:36.437178+0000","last_peered":"2023-10-08T09:13:36.437178+0000","last_clean":"2023-10-08T09:13:36.437178+0000","last_became_active":"2023-10-08T09:13:02.838389+0000","last_became_peered":"2023-10-08T09:13:02.838389+0000","last_unstale":"2023-10-08T09:13:36.437178+0000","last_undegraded":"2023-10-08T09:13:36.437178+0000","last_fullsized":"2023-10-08T09:13:36.437178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:52.937723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440434+0000","last_change":"2023-10-08T09:13:08.098131+0000","last_active":"2023-10-08T09:13:36.440434+0000","last_peered":"2023-10-08T09:13:36.440434+0000","last_clean":"2023-10-08T09:13:36.440434+0000","last_became_active":"2023-10-08T09:13:08.097352+0000","last_became_peered":"2023-10-08T09:13:08.097352+0000","last_unstale":"2023-10-08T09:13:36.440434+0000","last_undegraded":"2023-10-08T09:13:36.440434+0000","last_fullsized":"2023-10-08T09:13:36.440434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:30.388875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.461067+0000","last_change":"2023-10-08T09:13:37.461067+0000","last_active":"2023-10-08T09:13:37.461067+0000","last_peered":"2023-10-08T09:13:37.461067+0000","last_clean":"2023-10-08T09:13:37.461067+0000","last_became_active":"2023-10-08T09:13:37.451739+0000","last_became_peered":"2023-10-08T09:13:37.451739+0000","last_unstale":"2023-10-08T09:13:37.461067+0000","last_undegraded":"2023-10-08T09:13:37.461067+0000","last_fullsized":"2023-10-08T09:13:37.461067+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:26.603798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436259+0000","last_change":"2023-10-08T09:13:02.830017+0000","last_active":"2023-10-08T09:13:36.436259+0000","last_peered":"2023-10-08T09:13:36.436259+0000","last_clean":"2023-10-08T09:13:36.436259+0000","last_became_active":"2023-10-08T09:13:02.829700+0000","last_became_peered":"2023-10-08T09:13:02.829700+0000","last_unstale":"2023-10-08T09:13:36.436259+0000","last_undegraded":"2023-10-08T09:13:36.436259+0000","last_fullsized":"2023-10-08T09:13:36.436259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:31:31.387049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440739+0000","last_change":"2023-10-08T09:13:07.869097+0000","last_active":"2023-10-08T09:13:36.440739+0000","last_peered":"2023-10-08T09:13:36.440739+0000","last_clean":"2023-10-08T09:13:36.440739+0000","last_became_active":"2023-10-08T09:13:07.868710+0000","last_became_peered":"2023-10-08T09:13:07.868710+0000","last_unstale":"2023-10-08T09:13:36.440739+0000","last_undegraded":"2023-10-08T09:13:36.440739+0000","last_fullsized":"2023-10-08T09:13:36.440739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:37:27.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442264+0000","last_change":"2023-10-08T09:13:02.370694+0000","last_active":"2023-10-08T09:13:36.442264+0000","last_peered":"2023-10-08T09:13:36.442264+0000","last_clean":"2023-10-08T09:13:36.442264+0000","last_became_active":"2023-10-08T09:13:02.370161+0000","last_became_peered":"2023-10-08T09:13:02.370161+0000","last_unstale":"2023-10-08T09:13:36.442264+0000","last_undegraded":"2023-10-08T09:13:36.442264+0000","last_fullsized":"2023-10-08T09:13:36.442264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:04.809703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440513+0000","last_change":"2023-10-08T09:13:02.831026+0000","last_active":"2023-10-08T09:13:36.440513+0000","last_peered":"2023-10-08T09:13:36.440513+0000","last_clean":"2023-10-08T09:13:36.440513+0000","last_became_active":"2023-10-08T09:13:02.830715+0000","last_became_peered":"2023-10-08T09:13:02.830715+0000","last_unstale":"2023-10-08T09:13:36.440513+0000","last_undegraded":"2023-10-08T09:13:36.440513+0000","last_fullsized":"2023-10-08T09:13:36.440513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:47.709820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.439763+0000","last_change":"2023-10-08T09:13:07.860464+0000","last_active":"2023-10-08T09:13:36.439763+0000","last_peered":"2023-10-08T09:13:36.439763+0000","last_clean":"2023-10-08T09:13:36.439763+0000","last_became_active":"2023-10-08T09:13:07.859962+0000","last_became_peered":"2023-10-08T09:13:07.859962+0000","last_unstale":"2023-10-08T09:13:36.439763+0000","last_undegraded":"2023-10-08T09:13:36.439763+0000","last_fullsized":"2023-10-08T09:13:36.439763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:04.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.708490+0000","last_change":"2023-10-08T09:13:02.371308+0000","last_active":"2023-10-08T09:13:36.708490+0000","last_peered":"2023-10-08T09:13:36.708490+0000","last_clean":"2023-10-08T09:13:36.708490+0000","last_became_active":"2023-10-08T09:13:02.370968+0000","last_became_peered":"2023-10-08T09:13:02.370968+0000","last_unstale":"2023-10-08T09:13:36.708490+0000","last_undegraded":"2023-10-08T09:13:36.708490+0000","last_fullsized":"2023-10-08T09:13:36.708490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:58.491112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442038+0000","last_change":"2023-10-08T09:13:02.826511+0000","last_active":"2023-10-08T09:13:36.442038+0000","last_peered":"2023-10-08T09:13:36.442038+0000","last_clean":"2023-10-08T09:13:36.442038+0000","last_became_active":"2023-10-08T09:13:02.826205+0000","last_became_peered":"2023-10-08T09:13:02.826205+0000","last_unstale":"2023-10-08T09:13:36.442038+0000","last_undegraded":"2023-10-08T09:13:36.442038+0000","last_fullsized":"2023-10-08T09:13:36.442038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:12.270176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440248+0000","last_change":"2023-10-08T09:13:08.096360+0000","last_active":"2023-10-08T09:13:36.440248+0000","last_peered":"2023-10-08T09:13:36.440248+0000","last_clean":"2023-10-08T09:13:36.440248+0000","last_became_active":"2023-10-08T09:13:08.095915+0000","last_became_peered":"2023-10-08T09:13:08.095915+0000","last_unstale":"2023-10-08T09:13:36.440248+0000","last_undegraded":"2023-10-08T09:13:36.440248+0000","last_fullsized":"2023-10-08T09:13:36.440248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:59.412471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436477+0000","last_change":"2023-10-08T09:13:02.375287+0000","last_active":"2023-10-08T09:13:36.436477+0000","last_peered":"2023-10-08T09:13:36.436477+0000","last_clean":"2023-10-08T09:13:36.436477+0000","last_became_active":"2023-10-08T09:13:02.374668+0000","last_became_peered":"2023-10-08T09:13:02.374668+0000","last_unstale":"2023-10-08T09:13:36.436477+0000","last_undegraded":"2023-10-08T09:13:36.436477+0000","last_fullsized":"2023-10-08T09:13:36.436477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:22.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436931+0000","last_change":"2023-10-08T09:13:02.828240+0000","last_active":"2023-10-08T09:13:36.436931+0000","last_peered":"2023-10-08T09:13:36.436931+0000","last_clean":"2023-10-08T09:13:36.436931+0000","last_became_active":"2023-10-08T09:13:02.827951+0000","last_became_peered":"2023-10-08T09:13:02.827951+0000","last_unstale":"2023-10-08T09:13:36.436931+0000","last_undegraded":"2023-10-08T09:13:36.436931+0000","last_fullsized":"2023-10-08T09:13:36.436931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:37.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443935+0000","last_change":"2023-10-08T09:13:07.865846+0000","last_active":"2023-10-08T09:13:36.443935+0000","last_peered":"2023-10-08T09:13:36.443935+0000","last_clean":"2023-10-08T09:13:36.443935+0000","last_became_active":"2023-10-08T09:13:07.865529+0000","last_became_peered":"2023-10-08T09:13:07.865529+0000","last_unstale":"2023-10-08T09:13:36.443935+0000","last_undegraded":"2023-10-08T09:13:36.443935+0000","last_fullsized":"2023-10-08T09:13:36.443935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:34:03.408036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:14:14.500688+0000","last_change":"2023-10-08T09:13:02.380128+0000","last_active":"2023-10-08T09:14:14.500688+0000","last_peered":"2023-10-08T09:14:14.500688+0000","last_clean":"2023-10-08T09:14:14.500688+0000","last_became_active":"2023-10-08T09:13:02.379800+0000","last_became_peered":"2023-10-08T09:13:02.379800+0000","last_unstale":"2023-10-08T09:14:14.500688+0000","last_undegraded":"2023-10-08T09:14:14.500688+0000","last_fullsized":"2023-10-08T09:14:14.500688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:01.546289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437151+0000","last_change":"2023-10-08T09:13:02.829474+0000","last_active":"2023-10-08T09:13:36.437151+0000","last_peered":"2023-10-08T09:13:36.437151+0000","last_clean":"2023-10-08T09:13:36.437151+0000","last_became_active":"2023-10-08T09:13:02.829173+0000","last_became_peered":"2023-10-08T09:13:02.829173+0000","last_unstale":"2023-10-08T09:13:36.437151+0000","last_undegraded":"2023-10-08T09:13:36.437151+0000","last_fullsized":"2023-10-08T09:13:36.437151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:02.469158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T09:13:37.453778+0000","last_change":"2023-10-08T09:13:37.453778+0000","last_active":"2023-10-08T09:13:37.453778+0000","last_peered":"2023-10-08T09:13:37.453778+0000","last_clean":"2023-10-08T09:13:37.453778+0000","last_became_active":"2023-10-08T09:13:37.453354+0000","last_became_peered":"2023-10-08T09:13:37.453354+0000","last_unstale":"2023-10-08T09:13:37.453778+0000","last_undegraded":"2023-10-08T09:13:37.453778+0000","last_fullsized":"2023-10-08T09:13:37.453778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:50.035588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443501+0000","last_change":"2023-10-08T09:13:02.376641+0000","last_active":"2023-10-08T09:13:36.443501+0000","last_peered":"2023-10-08T09:13:36.443501+0000","last_clean":"2023-10-08T09:13:36.443501+0000","last_became_active":"2023-10-08T09:13:02.376348+0000","last_became_peered":"2023-10-08T09:13:02.376348+0000","last_unstale":"2023-10-08T09:13:36.443501+0000","last_undegraded":"2023-10-08T09:13:36.443501+0000","last_fullsized":"2023-10-08T09:13:36.443501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:42.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437239+0000","last_change":"2023-10-08T09:13:02.836201+0000","last_active":"2023-10-08T09:13:36.437239+0000","last_peered":"2023-10-08T09:13:36.437239+0000","last_clean":"2023-10-08T09:13:36.437239+0000","last_became_active":"2023-10-08T09:13:02.835885+0000","last_became_peered":"2023-10-08T09:13:02.835885+0000","last_unstale":"2023-10-08T09:13:36.437239+0000","last_undegraded":"2023-10-08T09:13:36.437239+0000","last_fullsized":"2023-10-08T09:13:36.437239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:34.310854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443140+0000","last_change":"2023-10-08T09:13:07.863854+0000","last_active":"2023-10-08T09:13:36.443140+0000","last_peered":"2023-10-08T09:13:36.443140+0000","last_clean":"2023-10-08T09:13:36.443140+0000","last_became_active":"2023-10-08T09:13:07.863460+0000","last_became_peered":"2023-10-08T09:13:07.863460+0000","last_unstale":"2023-10-08T09:13:36.443140+0000","last_undegraded":"2023-10-08T09:13:36.443140+0000","last_fullsized":"2023-10-08T09:13:36.443140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:03.093858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446178+0000","last_change":"2023-10-08T09:13:02.372336+0000","last_active":"2023-10-08T09:13:36.446178+0000","last_peered":"2023-10-08T09:13:36.446178+0000","last_clean":"2023-10-08T09:13:36.446178+0000","last_became_active":"2023-10-08T09:13:02.372022+0000","last_became_peered":"2023-10-08T09:13:02.372022+0000","last_unstale":"2023-10-08T09:13:36.446178+0000","last_undegraded":"2023-10-08T09:13:36.446178+0000","last_fullsized":"2023-10-08T09:13:36.446178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:10.610332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.437193+0000","last_change":"2023-10-08T09:13:02.832430+0000","last_active":"2023-10-08T09:13:36.437193+0000","last_peered":"2023-10-08T09:13:36.437193+0000","last_clean":"2023-10-08T09:13:36.437193+0000","last_became_active":"2023-10-08T09:13:02.832092+0000","last_became_peered":"2023-10-08T09:13:02.832092+0000","last_unstale":"2023-10-08T09:13:36.437193+0000","last_undegraded":"2023-10-08T09:13:36.437193+0000","last_fullsized":"2023-10-08T09:13:36.437193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:00.798900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440609+0000","last_change":"2023-10-08T09:13:02.833114+0000","last_active":"2023-10-08T09:13:36.440609+0000","last_peered":"2023-10-08T09:13:36.440609+0000","last_clean":"2023-10-08T09:13:36.440609+0000","last_became_active":"2023-10-08T09:13:02.830959+0000","last_became_peered":"2023-10-08T09:13:02.830959+0000","last_unstale":"2023-10-08T09:13:36.440609+0000","last_undegraded":"2023-10-08T09:13:36.440609+0000","last_fullsized":"2023-10-08T09:13:36.440609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:13.853585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442865+0000","last_change":"2023-10-08T09:13:02.836046+0000","last_active":"2023-10-08T09:13:36.442865+0000","last_peered":"2023-10-08T09:13:36.442865+0000","last_clean":"2023-10-08T09:13:36.442865+0000","last_became_active":"2023-10-08T09:13:02.835722+0000","last_became_peered":"2023-10-08T09:13:02.835722+0000","last_unstale":"2023-10-08T09:13:36.442865+0000","last_undegraded":"2023-10-08T09:13:36.442865+0000","last_fullsized":"2023-10-08T09:13:36.442865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:00.316259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442665+0000","last_change":"2023-10-08T09:13:02.830021+0000","last_active":"2023-10-08T09:13:36.442665+0000","last_peered":"2023-10-08T09:13:36.442665+0000","last_clean":"2023-10-08T09:13:36.442665+0000","last_became_active":"2023-10-08T09:13:02.829397+0000","last_became_peered":"2023-10-08T09:13:02.829397+0000","last_unstale":"2023-10-08T09:13:36.442665+0000","last_undegraded":"2023-10-08T09:13:36.442665+0000","last_fullsized":"2023-10-08T09:13:36.442665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:29.153457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.436590+0000","last_change":"2023-10-08T09:13:02.832002+0000","last_active":"2023-10-08T09:13:36.436590+0000","last_peered":"2023-10-08T09:13:36.436590+0000","last_clean":"2023-10-08T09:13:36.436590+0000","last_became_active":"2023-10-08T09:13:02.831666+0000","last_became_peered":"2023-10-08T09:13:02.831666+0000","last_unstale":"2023-10-08T09:13:36.436590+0000","last_undegraded":"2023-10-08T09:13:36.436590+0000","last_fullsized":"2023-10-08T09:13:36.436590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:16.414127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446412+0000","last_change":"2023-10-08T09:13:02.835881+0000","last_active":"2023-10-08T09:13:36.446412+0000","last_peered":"2023-10-08T09:13:36.446412+0000","last_clean":"2023-10-08T09:13:36.446412+0000","last_became_active":"2023-10-08T09:13:02.835587+0000","last_became_peered":"2023-10-08T09:13:02.835587+0000","last_unstale":"2023-10-08T09:13:36.446412+0000","last_undegraded":"2023-10-08T09:13:36.446412+0000","last_fullsized":"2023-10-08T09:13:36.446412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:52.323234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440978+0000","last_change":"2023-10-08T09:13:02.826371+0000","last_active":"2023-10-08T09:13:36.440978+0000","last_peered":"2023-10-08T09:13:36.440978+0000","last_clean":"2023-10-08T09:13:36.440978+0000","last_became_active":"2023-10-08T09:13:02.826093+0000","last_became_peered":"2023-10-08T09:13:02.826093+0000","last_unstale":"2023-10-08T09:13:36.440978+0000","last_undegraded":"2023-10-08T09:13:36.440978+0000","last_fullsized":"2023-10-08T09:13:36.440978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:08.113739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443860+0000","last_change":"2023-10-08T09:13:02.829936+0000","last_active":"2023-10-08T09:13:36.443860+0000","last_peered":"2023-10-08T09:13:36.443860+0000","last_clean":"2023-10-08T09:13:36.443860+0000","last_became_active":"2023-10-08T09:13:02.829498+0000","last_became_peered":"2023-10-08T09:13:02.829498+0000","last_unstale":"2023-10-08T09:13:36.443860+0000","last_undegraded":"2023-10-08T09:13:36.443860+0000","last_fullsized":"2023-10-08T09:13:36.443860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:42.200791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442594+0000","last_change":"2023-10-08T09:13:02.840415+0000","last_active":"2023-10-08T09:13:36.442594+0000","last_peered":"2023-10-08T09:13:36.442594+0000","last_clean":"2023-10-08T09:13:36.442594+0000","last_became_active":"2023-10-08T09:13:02.840075+0000","last_became_peered":"2023-10-08T09:13:02.840075+0000","last_unstale":"2023-10-08T09:13:36.442594+0000","last_undegraded":"2023-10-08T09:13:36.442594+0000","last_fullsized":"2023-10-08T09:13:36.442594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:48.959372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443355+0000","last_change":"2023-10-08T09:13:02.828337+0000","last_active":"2023-10-08T09:13:36.443355+0000","last_peered":"2023-10-08T09:13:36.443355+0000","last_clean":"2023-10-08T09:13:36.443355+0000","last_became_active":"2023-10-08T09:13:02.827998+0000","last_became_peered":"2023-10-08T09:13:02.827998+0000","last_unstale":"2023-10-08T09:13:36.443355+0000","last_undegraded":"2023-10-08T09:13:36.443355+0000","last_fullsized":"2023-10-08T09:13:36.443355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:27.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.442365+0000","last_change":"2023-10-08T09:13:02.833052+0000","last_active":"2023-10-08T09:13:36.442365+0000","last_peered":"2023-10-08T09:13:36.442365+0000","last_clean":"2023-10-08T09:13:36.442365+0000","last_became_active":"2023-10-08T09:13:02.832635+0000","last_became_peered":"2023-10-08T09:13:02.832635+0000","last_unstale":"2023-10-08T09:13:36.442365+0000","last_undegraded":"2023-10-08T09:13:36.442365+0000","last_fullsized":"2023-10-08T09:13:36.442365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:50.995768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.443151+0000","last_change":"2023-10-08T09:13:02.840921+0000","last_active":"2023-10-08T09:13:36.443151+0000","last_peered":"2023-10-08T09:13:36.443151+0000","last_clean":"2023-10-08T09:13:36.443151+0000","last_became_active":"2023-10-08T09:13:02.840566+0000","last_became_peered":"2023-10-08T09:13:02.840566+0000","last_unstale":"2023-10-08T09:13:36.443151+0000","last_undegraded":"2023-10-08T09:13:36.443151+0000","last_fullsized":"2023-10-08T09:13:36.443151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:20.219581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.446239+0000","last_change":"2023-10-08T09:13:02.834491+0000","last_active":"2023-10-08T09:13:36.446239+0000","last_peered":"2023-10-08T09:13:36.446239+0000","last_clean":"2023-10-08T09:13:36.446239+0000","last_became_active":"2023-10-08T09:13:02.834058+0000","last_became_peered":"2023-10-08T09:13:02.834058+0000","last_unstale":"2023-10-08T09:13:36.446239+0000","last_undegraded":"2023-10-08T09:13:36.446239+0000","last_fullsized":"2023-10-08T09:13:36.446239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:47.699423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T09:13:36.440869+0000","last_change":"2023-10-08T09:13:02.836433+0000","last_active":"2023-10-08T09:13:36.440869+0000","last_peered":"2023-10-08T09:13:36.440869+0000","last_clean":"2023-10-08T09:13:36.440869+0000","last_became_active":"2023-10-08T09:13:02.835147+0000","last_became_peered":"2023-10-08T09:13:02.835147+0000","last_unstale":"2023-10-08T09:13:36.440869+0000","last_undegraded":"2023-10-08T09:13:36.440869+0000","last_fullsized":"2023-10-08T09:13:36.440869+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:51.233662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247538,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65785,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":469,"num_read_kb":303,"num_write":235,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":197355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514523,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":488411,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45300000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57099999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.63600000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.69599999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.79600000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.82899999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.73299999999999998}]}]},{"osd":10,"up_from":69,"seq":296352743454,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":1137516,"data_compressed":53691,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.54000000000000004}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.502}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.61399999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73499999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83299999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.505,"5min":0.434,"15min":0.434},"max":{"1min":0.96199999999999997,"5min":1.01,"15min":1.01},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80700000000000005,"5min":1.115,"15min":1.115},"last":0.82799999999999996}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.441},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74299999999999999,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89300000000000002,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.125,"5min":1.534,"15min":1.534},"last":0.84699999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.879,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0840000000000001,"5min":1.649,"15min":1.649},"last":0.753}]},{"osd":11,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80800000000000005}]}]},{"osd":9,"up_from":64,"seq":274877906979,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":487951,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44600000000000001,"15min":0.44700000000000001},"min":{"1min":0.30599999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64100000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71099999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.27300000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.78800000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80100000000000005,"5min":1.012,"15min":1.012},"last":0.55600000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67400000000000004,"15min":0.67900000000000005},"min":{"1min":0.47799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.74199999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76500000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.61399999999999999,"15min":0.628},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91400000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79500000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84899999999999998,"5min":1.165,"15min":1.165},"last":0.57099999999999995}]},{"osd":5,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.997,"5min":1.216,"15min":1.216},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.98199999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0449999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.81599999999999995,"15min":0.83199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.056,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63800000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92800000000000005,"5min":1.337,"15min":1.337},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.69399999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.81200000000000006,"15min":0.82999999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96299999999999997,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79500000000000004,"5min":1.024,"15min":1.024},"last":0.64800000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75900000000000001}]}]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":483902,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54500000000000004,"15min":0.54100000000000004},"min":{"1min":0.32800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57799999999999996,"15min":0.57199999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58199999999999996}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.45000000000000001,"15min":0.436},"min":{"1min":0.36599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.443},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.33300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73499999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59999999999999998,"15min":0.59599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68500000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.60499999999999998},"min":{"1min":0.52700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0229999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56699999999999995,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77800000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.66800000000000004},"min":{"1min":0.51100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92800000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.53400000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.96499999999999997,"5min":0.999,"15min":0.999},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.93100000000000005,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.67500000000000004,"15min":0.66000000000000003},"min":{"1min":0.53900000000000003,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.84299999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.74399999999999999,"15min":0.73599999999999999},"min":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.85899999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.754,"15min":0.72899999999999998},"min":{"1min":0.76200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.879},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.69499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91400000000000003}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.89200000000000002}]}]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":1080729,"data_compressed":53691,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.41199999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54700000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.499},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.745},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83599999999999997,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.518,"15min":1.518},"last":0.76400000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81599999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.60899999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.505,"15min":0.47699999999999998},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56000000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.587,"15min":1.587},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":1.496,"15min":1.496},"last":0.79400000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.71399999999999997,"15min":0.68100000000000005},"min":{"1min":0.54300000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.71099999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66900000000000004,"15min":0.65900000000000003},"min":{"1min":0.496,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.80900000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.75},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.90300000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68999999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91700000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.63500000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88500000000000001}]}]},{"osd":6,"up_from":44,"seq":188978561072,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":483845,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39000000000000001,"15min":0.371},"min":{"1min":0.32100000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.40699999999999997,"15min":0.38300000000000001},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47899999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.54900000000000004},"min":{"1min":0.42799999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79200000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80900000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68200000000000005}]},{"osd":2,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60999999999999999,"15min":0.58499999999999996},"min":{"1min":0.46200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59999999999999998,"15min":0.57199999999999995},"min":{"1min":0.45200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1.145,"15min":1.145},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46800000000000003,"15min":0.44},"min":{"1min":0.35399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.48799999999999999,"15min":0.46200000000000002},"min":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80000000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.61499999999999999,"15min":0.58799999999999997},"min":{"1min":0.42499999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92400000000000004,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.64900000000000002,"15min":0.60699999999999998},"min":{"1min":0.44500000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89200000000000002,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66400000000000003,"15min":0.64200000000000002},"min":{"1min":0.46300000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.0189999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69799999999999995,"15min":0.68000000000000005},"min":{"1min":0.51200000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.622}]},{"osd":7,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.57599999999999996},{"interface":"front","average":{"1min":1.1879999999999999,"5min":0.81299999999999994,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.1229999999999993,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.63200000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.55700000000000005,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.755},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.747,"15min":0.751},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92200000000000004,"5min":1.175,"15min":1.175},"last":0.60199999999999998}]},{"osd":9,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76900000000000002,"15min":0.77400000000000002},"min":{"1min":0.40699999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.442,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.075,"5min":1.274,"15min":1.274},"last":0.54000000000000004}]},{"osd":10,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.61299999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.25,"5min":0.96999999999999997,"15min":0.92400000000000004},"min":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":9.1769999999999996,"5min":9.1769999999999996,"15min":9.1769999999999996},"last":0.67200000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":5.9610000000000003,"5min":5.9610000000000003,"15min":5.9610000000000003},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":547281,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91500000000000004,"5min":0.999,"15min":0.999},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 09:13:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56499999999999995,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.49099999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53100000000000003,"15min":0.499},"min":{"1min":0.39500000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73199999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.439,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":0.751,"15min":0.751},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.42099999999999999},"min":{"1min":0.41799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.87}]},{"osd":6,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84399999999999997,"5min":1.034,"15min":1.034},"last":0.748},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.497,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.59299999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Sun Oct 8 09:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92900000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.79200000000000004}]},{"osd":9,"last update":"Sun Oct 8 09:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.89200000000000002}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.73599999999999999,"15min":0.71399999999999997},"min":{"1min":0.72899999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.91700000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.98399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":485301,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54200000000000004,"15min":0.51000000000000001},"min":{"1min":0.45700000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.77400000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.57699999999999996,"15min":0.52000000000000002},"min":{"1min":0.443,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.65100000000000002}]},{"osd":2,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.59799999999999998,"15min":0.52800000000000002},"min":{"1min":0.53800000000000003,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.468,"5min":3.468,"15min":3.468},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.62,"15min":0.55400000000000005},"min":{"1min":0.52900000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.4409999999999998,"5min":3.4409999999999998,"15min":3.4409999999999998},"last":0.47899999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.435},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.45400000000000001}]},{"osd":4,"last update":"Sun Oct 8 09:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60999999999999999,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79800000000000004,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.53800000000000003,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91800000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.47999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79100000000000004,"5min":1.133,"15min":1.133},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.40799999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.878,"5min":1.095,"15min":1.095},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.51500000000000001},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.30299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.75700000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0209999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.69299999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.626,"15min":0.63400000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80700000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73199999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.79400000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.4569999999999999,"5min":3.4569999999999999,"15min":3.4569999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.58499999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90700000000000003,"5min":1.008,"15min":1.008},"last":0.746}]},{"osd":11,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":3.427,"5min":3.427,"15min":3.427},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.78500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":547774,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.49099999999999999},"min":{"1min":0.37,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.80800000000000005,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.50900000000000001,"15min":0.48699999999999999},"min":{"1min":0.33500000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.81799999999999995,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.54100000000000004}]},{"osd":1,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.41099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.318,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67900000000000005}]},{"osd":3,"last update":"Sun Oct 8 09:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.52500000000000002},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88600000000000001,"5min":1.758,"15min":1.758},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59399999999999997,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.93100000000000005,"5min":1.772,"15min":1.772},"last":0.60099999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Sun Oct 8 09:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.438},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.51600000000000001,"15min":0.505},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.47999999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96699999999999997,"5min":1.839,"15min":1.839},"last":0.63},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.66600000000000004,"15min":0.65400000000000003},"min":{"1min":0.50900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.014,"5min":1.492,"15min":1.492},"last":0.56899999999999995}]},{"osd":7,"last update":"Sun Oct 8 09:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81000000000000005}]},{"osd":8,"last update":"Sun Oct 8 09:13:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65000000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.48299999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.755,"15min":0.747},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1040000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.72499999999999998}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.877,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.83099999999999996}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.755},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.73899999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":482832,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.39200000000000002,"15min":0.376},"min":{"1min":0.27400000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.60699999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.379},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.30399999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63900000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.34599999999999997}]},{"osd":1,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.748,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.436},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77900000000000003,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.78800000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.501}]},{"osd":4,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82199999999999995,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.49399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72399999999999998,"5min":1.494,"15min":1.494},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.58499999999999996,"15min":0.55100000000000005},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.89900000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85999999999999999,"5min":0.87,"15min":0.87},"last":0.46500000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:13:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.374,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95499999999999996,"5min":1.01,"15min":1.01},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.81499999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86099999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60599999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.745,"15min":0.75800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.63600000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92500000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.81799999999999995,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":1.125,"15min":1.125},"last":0.64500000000000002}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67800000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":487529,"data_compressed":17626,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.499,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":1.466,"15min":1.466},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.36799999999999999},"min":{"1min":0.30199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.438,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.35199999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.48099999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.52700000000000002},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.89300000000000002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.52500000000000002},"min":{"1min":0.36899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77100000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.441}]},{"osd":5,"last update":"Sun Oct 8 09:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.56000000000000005},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":1.429,"15min":1.429},"last":0.56899999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93500000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.495},{"interface":"front","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79300000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.38800000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78900000000000003,"5min":0.873,"15min":0.873},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.40999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.82299999999999995,"5min":1.353,"15min":1.353},"last":0.624}]},{"osd":8,"last update":"Sun Oct 8 09:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.48799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.872,"5min":1.373,"15min":1.373},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.71099999999999997},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.079,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.81599999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77100000000000002,"15min":0.77700000000000002},"min":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92100000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78000000000000003,"15min":0.77900000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.66000000000000003}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.48699999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.50900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.69299999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.72399999999999998}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1080017,"data_compressed":53691,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.51500000000000001,"15min":0.502},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.32700000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.627,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54300000000000004,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.747,"5min":0.998,"15min":0.998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.27400000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64800000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.433}]},{"osd":4,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88400000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63900000000000001}]},{"osd":6,"last update":"Sun Oct 8 09:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.629,"15min":0.61099999999999999},"min":{"1min":0.42199999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89800000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.442,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.51900000000000002}]},{"osd":7,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.52400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.016,"15min":1.016},"last":0.71299999999999997}]},{"osd":8,"last update":"Sun Oct 8 09:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.375,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.67900000000000005}]},{"osd":9,"last update":"Sun Oct 8 09:13:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48299999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.99099999999999999,"5min":1.071,"15min":1.071},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69799999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.006},"last":0.79200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":60319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:14:20.250 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-08T09:14:20.250 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T09:14:20.250 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-08T09:14:20.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph health --format=json 2023-10-08T09:14:20.817 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:20.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:20 smithi184 ceph-mon[103675]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:14:21.036 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:20 smithi049 ceph-mon[99485]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:14:21.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:20 smithi132 ceph-mon[103389]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:14:21.649 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:21 smithi049 ceph-mon[99485]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-08T09:14:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:21 smithi184 ceph-mon[103675]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-08T09:14:22.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:21 smithi132 ceph-mon[103389]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-08T09:14:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 -- 172.21.15.49:0/2584145647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 msgr2=0x7f3d2810d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/2584145647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2810d670 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000b0a0 tx=0x7f3d2000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 -- 172.21.15.49:0/2584145647 shutdown_connections 2023-10-08T09:14:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/2584145647 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d281067a0 0x7f3d28106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/2584145647 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2810d670 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/2584145647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d28068190 0x7f3d280685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 -- 172.21.15.49:0/2584145647 >> 172.21.15.49:0/2584145647 conn(0x7f3d28075790 msgr2=0x7f3d28075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 -- 172.21.15.49:0/2584145647 shutdown_connections 2023-10-08T09:14:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.125+0000 7f3d2e03a700 1 -- 172.21.15.49:0/2584145647 wait complete. 2023-10-08T09:14:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 Processor -- start 2023-10-08T09:14:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 -- start start 2023-10-08T09:14:22.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d28068190 0x7f3d2819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d281067a0 0x7f3d28196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d28197190 con 0x7f3d28068b30 2023-10-08T09:14:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d28197300 con 0x7f3d281067a0 2023-10-08T09:14:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d2e03a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d281974c0 con 0x7f3d28068190 2023-10-08T09:14:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48916/0 (socket says 172.21.15.49:48916) 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 -- 172.21.15.49:0/4104035276 learned_addr learned my addr 172.21.15.49:0/4104035276 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d27fff700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d281067a0 0x7f3d28196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d277fe700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d28068190 0x7f3d2819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 -- 172.21.15.49:0/4104035276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d28068190 msgr2=0x7f3d2819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d28068190 0x7f3d2819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 -- 172.21.15.49:0/4104035276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d281067a0 msgr2=0x7f3d28196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d281067a0 0x7f3d28196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.126+0000 7f3d26ffd700 1 -- 172.21.15.49:0/4104035276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d20009d00 con 0x7f3d28068b30 2023-10-08T09:14:22.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.127+0000 7f3d26ffd700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2819cf30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d950 tx=0x7f3d1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.127+0000 7f3d24ff9700 1 -- 172.21.15.49:0/4104035276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00f890 con 0x7f3d28068b30 2023-10-08T09:14:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.127+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d281977a0 con 0x7f3d28068b30 2023-10-08T09:14:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.127+0000 7f3d24ff9700 1 -- 172.21.15.49:0/4104035276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d1c010460 con 0x7f3d28068b30 2023-10-08T09:14:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.127+0000 7f3d24ff9700 1 -- 172.21.15.49:0/4104035276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00e610 con 0x7f3d28068b30 2023-10-08T09:14:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.127+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d281a39e0 con 0x7f3d28068b30 2023-10-08T09:14:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.128+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d28108d50 con 0x7f3d28068b30 2023-10-08T09:14:22.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.129+0000 7f3d24ff9700 1 -- 172.21.15.49:0/4104035276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3d1c0108d0 con 0x7f3d28068b30 2023-10-08T09:14:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.129+0000 7f3d24ff9700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d100a1510 0x7f3d100a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.130+0000 7f3d24ff9700 1 -- 172.21.15.49:0/4104035276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d1c0cd2c0 con 0x7f3d28068b30 2023-10-08T09:14:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.130+0000 7f3d277fe700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d100a1510 0x7f3d100a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.130+0000 7f3d277fe700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d100a1510 0x7f3d100a39d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005950 tx=0x7f3d1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:22.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.134+0000 7f3d24ff9700 1 -- 172.21.15.49:0/4104035276 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3d1c0964b0 con 0x7f3d28068b30 2023-10-08T09:14:22.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.383+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3d28197ff0 con 0x7f3d28068b30 2023-10-08T09:14:22.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.383+0000 7f3d24ff9700 1 -- 172.21.15.49:0/4104035276 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3d1c099ad0 con 0x7f3d28068b30 2023-10-08T09:14:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:22.385 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d100a1510 msgr2=0x7f3d100a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d100a1510 0x7f3d100a39d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005950 tx=0x7f3d1800b410 comp rx=0 tx=0).stop 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 msgr2=0x7f3d2819cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2819cf30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00d950 tx=0x7f3d1c00dc60 comp rx=0 tx=0).stop 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 shutdown_connections 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3d100a1510 0x7f3d100a39d0 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005950 tx=0x7f3d1800b410 comp rx=0 tx=0).stop 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d28068190 0x7f3d2819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d281067a0 0x7f3d28196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 --2- 172.21.15.49:0/4104035276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3d28068b30 0x7f3d2819cf30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 >> 172.21.15.49:0/4104035276 conn(0x7f3d28075790 msgr2=0x7f3d280fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:22.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.385+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 shutdown_connections 2023-10-08T09:14:22.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.386+0000 7f3d2e03a700 1 -- 172.21.15.49:0/4104035276 wait complete. 2023-10-08T09:14:22.745 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:22 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4104035276' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T09:14:22.773 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-08T09:14:22.773 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:22.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 -- 172.21.15.49:0/4078706805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3040100790 msgr2=0x7f3040100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 --2- 172.21.15.49:0/4078706805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3040100790 0x7f3040100bf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f30340099c0 tx=0x7f3034009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 -- 172.21.15.49:0/4078706805 shutdown_connections 2023-10-08T09:14:22.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 --2- 172.21.15.49:0/4078706805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3040101130 0x7f30401079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 --2- 172.21.15.49:0/4078706805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30400ff5a0 0x7f30400ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 --2- 172.21.15.49:0/4078706805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3040100790 0x7f3040100bf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 -- 172.21.15.49:0/4078706805 >> 172.21.15.49:0/4078706805 conn(0x7f30400fab10 msgr2=0x7f30400fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:22.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 -- 172.21.15.49:0/4078706805 shutdown_connections 2023-10-08T09:14:22.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.906+0000 7f30469c2700 1 -- 172.21.15.49:0/4078706805 wait complete. 2023-10-08T09:14:22.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.907+0000 7f30469c2700 1 Processor -- start 2023-10-08T09:14:22.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.907+0000 7f30469c2700 1 -- start start 2023-10-08T09:14:22.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.907+0000 7f30469c2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30400ff5a0 0x7f30401954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f30469c2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3040100790 0x7f3040195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f30469c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040101130 0x7f304019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f30469c2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304019cff0 con 0x7f30400ff5a0 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f30469c2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304019d130 con 0x7f3040101130 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f30469c2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304019d2c0 con 0x7f3040100790 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30400ff5a0 0x7f30401954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30400ff5a0 0x7f30401954c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48926/0 (socket says 172.21.15.49:48926) 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 -- 172.21.15.49:0/1044228930 learned_addr learned my addr 172.21.15.49:0/1044228930 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f3044f5f700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040101130 0x7f304019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303f7fe700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3040100790 0x7f3040195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 -- 172.21.15.49:0/1044228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3040100790 msgr2=0x7f3040195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3040100790 0x7f3040195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 -- 172.21.15.49:0/1044228930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040101130 msgr2=0x7f304019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040101130 0x7f304019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 -- 172.21.15.49:0/1044228930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340095f0 con 0x7f30400ff5a0 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f3044f5f700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040101130 0x7f304019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303ffff700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30400ff5a0 0x7f30401954c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f303000c970 tx=0x7f303000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303d7fa700 1 -- 172.21.15.49:0/1044228930 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030004220 con 0x7f30400ff5a0 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304019d5a0 con 0x7f30400ff5a0 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303d7fa700 1 -- 172.21.15.49:0/1044228930 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3030007d60 con 0x7f30400ff5a0 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f303d7fa700 1 -- 172.21.15.49:0/1044228930 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030003d00 con 0x7f30400ff5a0 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.908+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304019db00 con 0x7f30400ff5a0 2023-10-08T09:14:22.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.909+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30401069d0 con 0x7f30400ff5a0 2023-10-08T09:14:22.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.910+0000 7f303d7fa700 1 -- 172.21.15.49:0/1044228930 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3030004380 con 0x7f30400ff5a0 2023-10-08T09:14:22.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.911+0000 7f303d7fa700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f30280a1590 0x7f30280a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:22.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.911+0000 7f303f7fe700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f30280a1590 0x7f30280a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:22.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.911+0000 7f303d7fa700 1 -- 172.21.15.49:0/1044228930 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f30300cd130 con 0x7f30400ff5a0 2023-10-08T09:14:22.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.912+0000 7f303f7fe700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f30280a1590 0x7f30280a3a50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3034000c00 tx=0x7f3034005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:22.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:22.915+0000 7f303d7fa700 1 -- 172.21.15.49:0/1044228930 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3030096340 con 0x7f30400ff5a0 2023-10-08T09:14:22.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:22 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4104035276' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T09:14:23.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:22 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4104035276' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-08T09:14:23.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.116+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3040067450 con 0x7f30400ff5a0 2023-10-08T09:14:23.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.117+0000 7f303d7fa700 1 -- 172.21.15.49:0/1044228930 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24441 (secure 0 0 0) 0x7f303001d020 con 0x7f30400ff5a0 2023-10-08T09:14:23.118 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:23.118 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:37.433726+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:14:23.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f30280a1590 msgr2=0x7f30280a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f30280a1590 0x7f30280a3a50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3034000c00 tx=0x7f3034005fd0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30400ff5a0 msgr2=0x7f30401954c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30400ff5a0 0x7f30401954c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f303000c970 tx=0x7f303000cd30 comp rx=0 tx=0).stop 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 shutdown_connections 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f30280a1590 0x7f30280a3a50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3040101130 0x7f304019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f30400ff5a0 0x7f30401954c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 --2- 172.21.15.49:0/1044228930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3040100790 0x7f3040195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 >> 172.21.15.49:0/1044228930 conn(0x7f30400fab10 msgr2=0x7f3040103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 shutdown_connections 2023-10-08T09:14:23.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.120+0000 7f30469c2700 1 -- 172.21.15.49:0/1044228930 wait complete. 2023-10-08T09:14:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:14:23.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:14:23.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 -- 172.21.15.49:0/1649980673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 msgr2=0x7fc940108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 --2- 172.21.15.49:0/1649980673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc940108d90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc93800b0d0 tx=0x7fc93800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 -- 172.21.15.49:0/1649980673 shutdown_connections 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 --2- 172.21.15.49:0/1649980673 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc940108d90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 --2- 172.21.15.49:0/1649980673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc940101db0 0x7fc9401021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 --2- 172.21.15.49:0/1649980673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9400fe820 0x7fc9400fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 -- 172.21.15.49:0/1649980673 >> 172.21.15.49:0/1649980673 conn(0x7fc9400fa3c0 msgr2=0x7fc9400fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 -- 172.21.15.49:0/1649980673 shutdown_connections 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 -- 172.21.15.49:0/1649980673 wait complete. 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.262+0000 7fc946bed700 1 Processor -- start 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 -- start start 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400fe820 0x7fc940101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc9400ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc940101db0 0x7fc940100030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401007d0 con 0x7fc9400ff1c0 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc940100940 con 0x7fc9400fe820 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401a0880 con 0x7fc940101db0 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc9400ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc9400ffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48942/0 (socket says 172.21.15.49:48942) 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 -- 172.21.15.49:0/3658588287 learned_addr learned my addr 172.21.15.49:0/3658588287 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc94518a700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc940101db0 0x7fc940100030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc944989700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400fe820 0x7fc940101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 -- 172.21.15.49:0/3658588287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc940101db0 msgr2=0x7fc940100030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc940101db0 0x7fc940100030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 -- 172.21.15.49:0/3658588287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400fe820 msgr2=0x7fc940101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400fe820 0x7fc940101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 -- 172.21.15.49:0/3658588287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938009d00 con 0x7fc9400ff1c0 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93ffff700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc9400ffaf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00d950 tx=0x7fc92c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93dffb700 1 -- 172.21.15.49:0/3658588287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c009ae0 con 0x7fc9400ff1c0 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93dffb700 1 -- 172.21.15.49:0/3658588287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92c009c40 con 0x7fc9400ff1c0 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9401a0b10 con 0x7fc9400ff1c0 2023-10-08T09:14:23.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.263+0000 7fc93dffb700 1 -- 172.21.15.49:0/3658588287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00f620 con 0x7fc9400ff1c0 2023-10-08T09:14:23.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.264+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9401a1020 con 0x7fc9400ff1c0 2023-10-08T09:14:23.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.265+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940107d00 con 0x7fc9400ff1c0 2023-10-08T09:14:23.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.265+0000 7fc93dffb700 1 -- 172.21.15.49:0/3658588287 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc92c010460 con 0x7fc9400ff1c0 2023-10-08T09:14:23.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.266+0000 7fc93dffb700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc9300a1510 0x7fc9300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.266+0000 7fc93dffb700 1 -- 172.21.15.49:0/3658588287 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc92c0cd0d0 con 0x7fc9400ff1c0 2023-10-08T09:14:23.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.266+0000 7fc944989700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc9300a1510 0x7fc9300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.267+0000 7fc944989700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc9300a1510 0x7fc9300a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc9400689f0 tx=0x7fc934009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:23.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.271+0000 7fc93dffb700 1 -- 172.21.15.49:0/3658588287 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc92c0962c0 con 0x7fc9400ff1c0 2023-10-08T09:14:23.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.471+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc94010bfe0 con 0x7fc9400ff1c0 2023-10-08T09:14:23.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.471+0000 7fc93dffb700 1 -- 172.21.15.49:0/3658588287 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc92c0998e0 con 0x7fc9400ff1c0 2023-10-08T09:14:23.472 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc9300a1510 msgr2=0x7fc9300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc9300a1510 0x7fc9300a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc9400689f0 tx=0x7fc934009260 comp rx=0 tx=0).stop 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 msgr2=0x7fc9400ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc9400ffaf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00d950 tx=0x7fc92c00dd10 comp rx=0 tx=0).stop 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 shutdown_connections 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc9300a1510 0x7fc9300a39d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc940101db0 0x7fc940100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9400fe820 0x7fc940101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 --2- 172.21.15.49:0/3658588287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc9400ff1c0 0x7fc9400ffaf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 >> 172.21.15.49:0/3658588287 conn(0x7fc9400fa3c0 msgr2=0x7fc940104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:23.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 shutdown_connections 2023-10-08T09:14:23.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.473+0000 7fc946bed700 1 -- 172.21.15.49:0/3658588287 wait complete. 2023-10-08T09:14:23.486 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:14:23.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 -- 172.21.15.49:0/2887214005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 msgr2=0x7fdd600ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/2887214005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd600ffac0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd540099c0 tx=0x7fdd54009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 -- 172.21.15.49:0/2887214005 shutdown_connections 2023-10-08T09:14:23.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/2887214005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd60101230 0x7fdd60107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/2887214005 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd600ffac0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/2887214005 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd60100890 0x7fdd60100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 -- 172.21.15.49:0/2887214005 >> 172.21.15.49:0/2887214005 conn(0x7fdd600fac10 msgr2=0x7fdd600fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 -- 172.21.15.49:0/2887214005 shutdown_connections 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 -- 172.21.15.49:0/2887214005 wait complete. 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 Processor -- start 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 -- start start 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.615+0000 7fdd66d0a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd60068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd66d0a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd60100890 0x7fdd6019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd66d0a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd60101230 0x7fdd6019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd66d0a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019e260 con 0x7fdd600ff6a0 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd66d0a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019e3d0 con 0x7fdd60101230 2023-10-08T09:14:23.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd66d0a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6019e590 con 0x7fdd60100890 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd60068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd60068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48954/0 (socket says 172.21.15.49:48954) 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 -- 172.21.15.49:0/391655344 learned_addr learned my addr 172.21.15.49:0/391655344 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 -- 172.21.15.49:0/391655344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd60100890 msgr2=0x7fdd6019b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd5ffff700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd60100890 0x7fdd6019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd652a7700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd60101230 0x7fdd6019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd60100890 0x7fdd6019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 -- 172.21.15.49:0/391655344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd60101230 msgr2=0x7fdd6019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd60101230 0x7fdd6019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 -- 172.21.15.49:0/391655344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd540095f0 con 0x7fdd600ff6a0 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd652a7700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd60101230 0x7fdd6019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:23.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd64aa6700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd60068350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009f50 tx=0x7fdd54003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:23.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd5dffb700 1 -- 172.21.15.49:0/391655344 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5401d070 con 0x7fdd600ff6a0 2023-10-08T09:14:23.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd5dffb700 1 -- 172.21.15.49:0/391655344 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd54022470 con 0x7fdd600ff6a0 2023-10-08T09:14:23.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6019e810 con 0x7fdd600ff6a0 2023-10-08T09:14:23.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.616+0000 7fdd5dffb700 1 -- 172.21.15.49:0/391655344 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400f6c0 con 0x7fdd600ff6a0 2023-10-08T09:14:23.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.617+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6019ec30 con 0x7fdd600ff6a0 2023-10-08T09:14:23.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.618+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60106b90 con 0x7fdd600ff6a0 2023-10-08T09:14:23.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.619+0000 7fdd5dffb700 1 -- 172.21.15.49:0/391655344 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fdd540225e0 con 0x7fdd600ff6a0 2023-10-08T09:14:23.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.619+0000 7fdd5dffb700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd500a1510 0x7fdd500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.619+0000 7fdd5dffb700 1 -- 172.21.15.49:0/391655344 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdd540ce790 con 0x7fdd600ff6a0 2023-10-08T09:14:23.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.619+0000 7fdd5ffff700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd500a1510 0x7fdd500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.620+0000 7fdd5ffff700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd500a1510 0x7fdd500a39d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd60068910 tx=0x7fdd4c005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:23.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.624+0000 7fdd5dffb700 1 -- 172.21.15.49:0/391655344 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd54097a10 con 0x7fdd600ff6a0 2023-10-08T09:14:23.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.826+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd6010b080 con 0x7fdd600ff6a0 2023-10-08T09:14:23.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.826+0000 7fdd5dffb700 1 -- 172.21.15.49:0/391655344 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd5409b030 con 0x7fdd600ff6a0 2023-10-08T09:14:23.827 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:23.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd500a1510 msgr2=0x7fdd500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd500a1510 0x7fdd500a39d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd60068910 tx=0x7fdd4c005cf0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 msgr2=0x7fdd60068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd60068350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009f50 tx=0x7fdd54003970 comp rx=0 tx=0).stop 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 shutdown_connections 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd500a1510 0x7fdd500a39d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd60101230 0x7fdd6019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd600ff6a0 0x7fdd60068350 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 --2- 172.21.15.49:0/391655344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd60100890 0x7fdd6019b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 >> 172.21.15.49:0/391655344 conn(0x7fdd600fac10 msgr2=0x7fdd60103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 shutdown_connections 2023-10-08T09:14:23.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.829+0000 7fdd66d0a700 1 -- 172.21.15.49:0/391655344 wait complete. 2023-10-08T09:14:23.840 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:23 smithi049 ceph-mon[99485]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T09:14:23.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:23 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1044228930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:23.841 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:23 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3658588287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:23.842 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:14:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 -- 172.21.15.49:0/2741059653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c1012c0 msgr2=0x7ffb4c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2741059653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c1012c0 0x7ffb4c1036b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009990 tx=0x7ffb3c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 -- 172.21.15.49:0/2741059653 shutdown_connections 2023-10-08T09:14:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2741059653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb4c106520 0x7ffb4c108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2741059653 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c1012c0 0x7ffb4c1036b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2741059653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c103bf0 0x7ffb4c105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 -- 172.21.15.49:0/2741059653 >> 172.21.15.49:0/2741059653 conn(0x7ffb4c0fac10 msgr2=0x7ffb4c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 -- 172.21.15.49:0/2741059653 shutdown_connections 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.970+0000 7ffb51c70700 1 -- 172.21.15.49:0/2741059653 wait complete. 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 Processor -- start 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 -- start start 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c1012c0 0x7ffb4c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb4c103bf0 0x7ffb4c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c106520 0x7ffb4c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c1a0370 con 0x7ffb4c106520 2023-10-08T09:14:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c1a04b0 con 0x7ffb4c103bf0 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb51c70700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c1a0640 con 0x7ffb4c1012c0 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c106520 0x7ffb4c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c106520 0x7ffb4c19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48972/0 (socket says 172.21.15.49:48972) 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb4bfff700 1 -- 172.21.15.49:0/2977471875 learned_addr learned my addr 172.21.15.49:0/2977471875 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb4b7fe700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c1012c0 0x7ffb4c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.971+0000 7ffb4affd700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb4c103bf0 0x7ffb4c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb4bfff700 1 -- 172.21.15.49:0/2977471875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c1012c0 msgr2=0x7ffb4c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb4bfff700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c1012c0 0x7ffb4c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb4bfff700 1 -- 172.21.15.49:0/2977471875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb4c103bf0 msgr2=0x7ffb4c198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb4bfff700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb4c103bf0 0x7ffb4c198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb4bfff700 1 -- 172.21.15.49:0/2977471875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0095f0 con 0x7ffb4c106520 2023-10-08T09:14:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb4affd700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb4c103bf0 0x7ffb4c198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb4bfff700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c106520 0x7ffb4c19fda0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb44007d70 tx=0x7ffb4400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb48ff9700 1 -- 172.21.15.49:0/2977471875 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400ad40 con 0x7ffb4c106520 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb48ff9700 1 -- 172.21.15.49:0/2977471875 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb440128e0 con 0x7ffb4c106520 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c1a0920 con 0x7ffb4c106520 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb48ff9700 1 -- 172.21.15.49:0/2977471875 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb440116b0 con 0x7ffb4c106520 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.972+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c1a0de0 con 0x7ffb4c106520 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.973+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c1080a0 con 0x7ffb4c106520 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.974+0000 7ffb48ff9700 1 -- 172.21.15.49:0/2977471875 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ffb44012420 con 0x7ffb4c106520 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.975+0000 7ffb48ff9700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb340a1510 0x7ffb340a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.975+0000 7ffb4b7fe700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb340a1510 0x7ffb340a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.975+0000 7ffb48ff9700 1 -- 172.21.15.49:0/2977471875 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffb440cd300 con 0x7ffb4c106520 2023-10-08T09:14:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.975+0000 7ffb4b7fe700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb340a1510 0x7ffb340a39d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b5c0 tx=0x7ffb3c003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:23.979+0000 7ffb48ff9700 1 -- 172.21.15.49:0/2977471875 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffb44096600 con 0x7ffb4c106520 2023-10-08T09:14:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:23 smithi184 ceph-mon[103675]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T09:14:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:23 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1044228930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:23 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3658588287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:24.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:23 smithi132 ceph-mon[103389]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T09:14:24.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:23 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1044228930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:24.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:23 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3658588287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:24.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.184+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb4c10c090 con 0x7ffb4c106520 2023-10-08T09:14:24.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.184+0000 7ffb48ff9700 1 -- 172.21.15.49:0/2977471875 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb44099c20 con 0x7ffb4c106520 2023-10-08T09:14:24.185 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:14:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb340a1510 msgr2=0x7ffb340a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb340a1510 0x7ffb340a39d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b5c0 tx=0x7ffb3c003790 comp rx=0 tx=0).stop 2023-10-08T09:14:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c106520 msgr2=0x7ffb4c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c106520 0x7ffb4c19fda0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb44007d70 tx=0x7ffb4400a340 comp rx=0 tx=0).stop 2023-10-08T09:14:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 shutdown_connections 2023-10-08T09:14:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ffb340a1510 0x7ffb340a39d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb4c1012c0 0x7ffb4c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffb4c106520 0x7ffb4c19fda0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 --2- 172.21.15.49:0/2977471875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb4c103bf0 0x7ffb4c198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.186+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 >> 172.21.15.49:0/2977471875 conn(0x7ffb4c0fac10 msgr2=0x7ffb4c1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.187+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 shutdown_connections 2023-10-08T09:14:24.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.187+0000 7ffb51c70700 1 -- 172.21.15.49:0/2977471875 wait complete. 2023-10-08T09:14:24.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.327+0000 7f49b9cd2700 1 -- 172.21.15.49:0/2418628651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 msgr2=0x7f49b4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.327+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/2418628651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4100be0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f49a80099c0 tx=0x7f49a8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 -- 172.21.15.49:0/2418628651 shutdown_connections 2023-10-08T09:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/2418628651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b4102350 0x7f49b4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/2418628651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4100be0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/2418628651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49b41019b0 0x7f49b4101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 -- 172.21.15.49:0/2418628651 >> 172.21.15.49:0/2418628651 conn(0x7f49b40fbd30 msgr2=0x7f49b40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 -- 172.21.15.49:0/2418628651 shutdown_connections 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 -- 172.21.15.49:0/2418628651 wait complete. 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 Processor -- start 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 -- start start 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.328+0000 7f49b9cd2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b9cd2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49b41019b0 0x7f49b4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b9cd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b4102350 0x7f49b4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b9cd2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40738c0 con 0x7f49b41007c0 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b9cd2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4073a30 con 0x7f49b4102350 2023-10-08T09:14:24.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b9cd2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b41a0740 con 0x7f49b41019b0 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48982/0 (socket says 172.21.15.49:48982) 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 -- 172.21.15.49:0/89613869 learned_addr learned my addr 172.21.15.49:0/89613869 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b3fff700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b4102350 0x7f49b4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b2ffd700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49b41019b0 0x7f49b4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 -- 172.21.15.49:0/89613869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49b41019b0 msgr2=0x7f49b4072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49b41019b0 0x7f49b4072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 -- 172.21.15.49:0/89613869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b4102350 msgr2=0x7f49b4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b4102350 0x7f49b4073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 -- 172.21.15.49:0/89613869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80095f0 con 0x7f49b41007c0 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b3fff700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b4102350 0x7f49b4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:24.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b37fe700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4074550 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009f50 tx=0x7f49a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:24.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b0ff9700 1 -- 172.21.15.49:0/89613869 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a801d070 con 0x7f49b41007c0 2023-10-08T09:14:24.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b41a0970 con 0x7f49b41007c0 2023-10-08T09:14:24.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b0ff9700 1 -- 172.21.15.49:0/89613869 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49a8004200 con 0x7f49b41007c0 2023-10-08T09:14:24.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b0ff9700 1 -- 172.21.15.49:0/89613869 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8022e40 con 0x7f49b41007c0 2023-10-08T09:14:24.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.329+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b41a0dc0 con 0x7f49b41007c0 2023-10-08T09:14:24.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.330+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b4107bf0 con 0x7f49b41007c0 2023-10-08T09:14:24.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.331+0000 7f49b0ff9700 1 -- 172.21.15.49:0/89613869 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f49a8022570 con 0x7f49b41007c0 2023-10-08T09:14:24.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.332+0000 7f49b0ff9700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f499c0a1510 0x7f499c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.332+0000 7f49b2ffd700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f499c0a1510 0x7f499c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.332+0000 7f49b0ff9700 1 -- 172.21.15.49:0/89613869 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f49a80cecd0 con 0x7f49b41007c0 2023-10-08T09:14:24.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.333+0000 7f49b2ffd700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f499c0a1510 0x7f499c0a39d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f49a4005ea0 tx=0x7f49a4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:24.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.336+0000 7f49b0ff9700 1 -- 172.21.15.49:0/89613869 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f49a8097f50 con 0x7f49b41007c0 2023-10-08T09:14:24.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.537+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f49b410c0f0 con 0x7f49b41007c0 2023-10-08T09:14:24.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.537+0000 7f49b0ff9700 1 -- 172.21.15.49:0/89613869 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f49a809b570 con 0x7f49b41007c0 2023-10-08T09:14:24.538 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f499c0a1510 msgr2=0x7f499c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f499c0a1510 0x7f499c0a39d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f49a4005ea0 tx=0x7f49a4005e30 comp rx=0 tx=0).stop 2023-10-08T09:14:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 msgr2=0x7f49b4074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4074550 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009f50 tx=0x7f49a80057f0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 shutdown_connections 2023-10-08T09:14:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f499c0a1510 0x7f499c0a39d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b4102350 0x7f49b4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49b41007c0 0x7f49b4074550 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 --2- 172.21.15.49:0/89613869 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49b41019b0 0x7f49b4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 >> 172.21.15.49:0/89613869 conn(0x7f49b40fbd30 msgr2=0x7f49b4104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 shutdown_connections 2023-10-08T09:14:24.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.539+0000 7f49b9cd2700 1 -- 172.21.15.49:0/89613869 wait complete. 2023-10-08T09:14:24.552 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:14:24.679 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1793436965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681006b0 msgr2=0x7f5c68100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1793436965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681006b0 0x7f5c68100ad0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0099c0 tx=0x7f5c5c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1793436965 shutdown_connections 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1793436965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c68102240 0x7f5c68108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1793436965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681006b0 0x7f5c68100ad0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1793436965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c681018a0 0x7f5c68101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1793436965 >> 172.21.15.49:0/1793436965 conn(0x7f5c680fbc20 msgr2=0x7f5c680fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1793436965 shutdown_connections 2023-10-08T09:14:24.680 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.679+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1793436965 wait complete. 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 Processor -- start 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 -- start start 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c681006b0 0x7f5c681966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681018a0 0x7f5c68196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c68102240 0x7f5c6819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6819e1d0 con 0x7f5c681018a0 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6819e310 con 0x7f5c681006b0 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6cca6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6819e4a0 con 0x7f5c68102240 2023-10-08T09:14:24.683 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681018a0 0x7f5c68196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681018a0 0x7f5c68196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48998/0 (socket says 172.21.15.49:48998) 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 -- 172.21.15.49:0/1091977316 learned_addr learned my addr 172.21.15.49:0/1091977316 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c66d9d700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c68102240 0x7f5c6819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6659c700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c681006b0 0x7f5c681966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 -- 172.21.15.49:0/1091977316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c68102240 msgr2=0x7f5c6819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c68102240 0x7f5c6819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 -- 172.21.15.49:0/1091977316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c681006b0 msgr2=0x7f5c681966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c681006b0 0x7f5c681966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.684 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c65d9b700 1 -- 172.21.15.49:0/1091977316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0095f0 con 0x7f5c681018a0 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c66d9d700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c68102240 0x7f5c6819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.680+0000 7f5c6659c700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c681006b0 0x7f5c681966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.681+0000 7f5c65d9b700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681018a0 0x7f5c68196be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800b530 tx=0x7f5c5800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.681+0000 7f5c577fe700 1 -- 172.21.15.49:0/1091977316 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58011830 con 0x7f5c681018a0 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.681+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6819e780 con 0x7f5c681018a0 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.681+0000 7f5c577fe700 1 -- 172.21.15.49:0/1091977316 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c58011e50 con 0x7f5c681018a0 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.681+0000 7f5c577fe700 1 -- 172.21.15.49:0/1091977316 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800f2e0 con 0x7f5c681018a0 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.681+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6819ec90 con 0x7f5c681018a0 2023-10-08T09:14:24.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.682+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c68107b80 con 0x7f5c681018a0 2023-10-08T09:14:24.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.683+0000 7f5c577fe700 1 -- 172.21.15.49:0/1091977316 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5c58010580 con 0x7f5c681018a0 2023-10-08T09:14:24.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.683+0000 7f5c577fe700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c500a1510 0x7f5c500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:24.686 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.684+0000 7f5c6659c700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c500a1510 0x7f5c500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.684+0000 7f5c577fe700 1 -- 172.21.15.49:0/1091977316 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c580cd190 con 0x7f5c681018a0 2023-10-08T09:14:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.684+0000 7f5c6659c700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c500a1510 0x7f5c500a39d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c006010 tx=0x7f5c5c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:24.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.687+0000 7f5c577fe700 1 -- 172.21.15.49:0/1091977316 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5c58096430 con 0x7f5c681018a0 2023-10-08T09:14:24.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:24 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/391655344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:24.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:24 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2977471875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:24.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:24 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/89613869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:24.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.934+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c680678a0 con 0x7f5c681018a0 2023-10-08T09:14:24.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.935+0000 7f5c577fe700 1 -- 172.21.15.49:0/1091977316 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f5c58015090 con 0x7f5c681018a0 2023-10-08T09:14:24.935 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:24.936 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:14:24.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c500a1510 msgr2=0x7f5c500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c500a1510 0x7f5c500a39d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c006010 tx=0x7f5c5c009f10 comp rx=0 tx=0).stop 2023-10-08T09:14:24.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681018a0 msgr2=0x7f5c68196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:24.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681018a0 0x7f5c68196be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800b530 tx=0x7f5c5800b8f0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 shutdown_connections 2023-10-08T09:14:24.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c500a1510 0x7f5c500a39d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c68102240 0x7f5c6819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c681006b0 0x7f5c681966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 --2- 172.21.15.49:0/1091977316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c681018a0 0x7f5c68196be0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:24.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 >> 172.21.15.49:0/1091977316 conn(0x7f5c680fbc20 msgr2=0x7f5c68104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:24.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 shutdown_connections 2023-10-08T09:14:24.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:24.937+0000 7f5c6cca6700 1 -- 172.21.15.49:0/1091977316 wait complete. 2023-10-08T09:14:24.941 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:14:24.950 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:24.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:24 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/391655344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:24.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:24 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2977471875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:24.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:24 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/89613869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:25.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:24 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/391655344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:25.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:24 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2977471875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:25.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:24 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/89613869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:25.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.076+0000 7f02f027e700 1 -- 172.21.15.49:0/2598606193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 msgr2=0x7f02e81079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.076+0000 7f02f027e700 1 --2- 172.21.15.49:0/2598606193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e81079e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b0d0 tx=0x7f02e000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.076+0000 7f02f027e700 1 -- 172.21.15.49:0/2598606193 shutdown_connections 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.076+0000 7f02f027e700 1 --2- 172.21.15.49:0/2598606193 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02e80ff5a0 0x7f02e80ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.076+0000 7f02f027e700 1 --2- 172.21.15.49:0/2598606193 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e81079e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.076+0000 7f02f027e700 1 --2- 172.21.15.49:0/2598606193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02e8100790 0x7f02e8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 -- 172.21.15.49:0/2598606193 >> 172.21.15.49:0/2598606193 conn(0x7f02e80fab10 msgr2=0x7f02e80fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 -- 172.21.15.49:0/2598606193 shutdown_connections 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 -- 172.21.15.49:0/2598606193 wait complete. 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 Processor -- start 2023-10-08T09:14:25.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 -- start start 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02e80ff5a0 0x7f02e8196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02e8100790 0x7f02e8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e819e240 con 0x7f02e8101130 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e819e380 con 0x7f02e8100790 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02f027e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e819e510 con 0x7f02e80ff5a0 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02ee81b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02ee81b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e819dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49014/0 (socket says 172.21.15.49:49014) 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.077+0000 7f02ee81b700 1 -- 172.21.15.49:0/3120202982 learned_addr learned my addr 172.21.15.49:0/3120202982 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ed819700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02e8100790 0x7f02e8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ee01a700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02e80ff5a0 0x7f02e8196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ee81b700 1 -- 172.21.15.49:0/3120202982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02e80ff5a0 msgr2=0x7f02e8196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ee81b700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02e80ff5a0 0x7f02e8196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ee81b700 1 -- 172.21.15.49:0/3120202982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02e8100790 msgr2=0x7f02e8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ee81b700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02e8100790 0x7f02e8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ee81b700 1 -- 172.21.15.49:0/3120202982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e0009d00 con 0x7f02e8101130 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ed819700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02e8100790 0x7f02e8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02ee81b700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e819dc70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02e0007bd0 tx=0x7f02e00094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02db7fe700 1 -- 172.21.15.49:0/3120202982 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e000e040 con 0x7f02e8101130 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02db7fe700 1 -- 172.21.15.49:0/3120202982 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02e001dd50 con 0x7f02e8101130 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e819e790 con 0x7f02e8101130 2023-10-08T09:14:25.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02db7fe700 1 -- 172.21.15.49:0/3120202982 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e001b450 con 0x7f02e8101130 2023-10-08T09:14:25.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.078+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e819ec10 con 0x7f02e8101130 2023-10-08T09:14:25.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.079+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e8067450 con 0x7f02e8101130 2023-10-08T09:14:25.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.080+0000 7f02db7fe700 1 -- 172.21.15.49:0/3120202982 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f02e0012470 con 0x7f02e8101130 2023-10-08T09:14:25.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.081+0000 7f02db7fe700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02d40a1510 0x7f02d40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.081+0000 7f02db7fe700 1 -- 172.21.15.49:0/3120202982 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f02e00cf8e0 con 0x7f02e8101130 2023-10-08T09:14:25.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.081+0000 7f02ee01a700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02d40a1510 0x7f02d40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.081+0000 7f02ee01a700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02d40a1510 0x7f02d40a39d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02e81005f0 tx=0x7f02e400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:25.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.084+0000 7f02db7fe700 1 -- 172.21.15.49:0/3120202982 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f02e0098b60 con 0x7f02e8101130 2023-10-08T09:14:25.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.289+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02e810b050 con 0x7f02e8101130 2023-10-08T09:14:25.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.291+0000 7f02db7fe700 1 -- 172.21.15.49:0/3120202982 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24441 (secure 0 0 0) 0x7f02e009c180 con 0x7f02e8101130 2023-10-08T09:14:25.291 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:25.292 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:37.433726+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:14:25.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02d40a1510 msgr2=0x7f02d40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02d40a1510 0x7f02d40a39d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f02e81005f0 tx=0x7f02e400b5b0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 msgr2=0x7f02e819dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e819dc70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02e0007bd0 tx=0x7f02e00094f0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 shutdown_connections 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f02d40a1510 0x7f02d40a39d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02e80ff5a0 0x7f02e8196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02e8101130 0x7f02e819dc70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 --2- 172.21.15.49:0/3120202982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02e8100790 0x7f02e8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 >> 172.21.15.49:0/3120202982 conn(0x7f02e80fab10 msgr2=0x7f02e8067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 shutdown_connections 2023-10-08T09:14:25.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.293+0000 7f02f027e700 1 -- 172.21.15.49:0/3120202982 wait complete. 2023-10-08T09:14:25.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:14:25.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 -- 172.21.15.49:0/2484294947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 msgr2=0x7f948c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 --2- 172.21.15.49:0/2484294947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c107af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f948400b0d0 tx=0x7f948400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 -- 172.21.15.49:0/2484294947 shutdown_connections 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 --2- 172.21.15.49:0/2484294947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c107af0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 --2- 172.21.15.49:0/2484294947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f948c0ff6b0 0x7f948c0ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 --2- 172.21.15.49:0/2484294947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f948c1008a0 0x7f948c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 -- 172.21.15.49:0/2484294947 >> 172.21.15.49:0/2484294947 conn(0x7f948c0fac40 msgr2=0x7f948c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 -- 172.21.15.49:0/2484294947 shutdown_connections 2023-10-08T09:14:25.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.435+0000 7f9491599700 1 -- 172.21.15.49:0/2484294947 wait complete. 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 Processor -- start 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 -- start start 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f948c0ff6b0 0x7f948c069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f948c1008a0 0x7f948c19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c19e350 con 0x7f948c101240 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c19e4c0 con 0x7f948c0ff6b0 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f9491599700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c19e680 con 0x7f948c1008a0 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49028/0 (socket says 172.21.15.49:49028) 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948b7fe700 1 -- 172.21.15.49:0/691051795 learned_addr learned my addr 172.21.15.49:0/691051795 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948affd700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f948c0ff6b0 0x7f948c069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948a7fc700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f948c1008a0 0x7f948c19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948b7fe700 1 -- 172.21.15.49:0/691051795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f948c1008a0 msgr2=0x7f948c19b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948b7fe700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f948c1008a0 0x7f948c19b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948b7fe700 1 -- 172.21.15.49:0/691051795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f948c0ff6b0 msgr2=0x7f948c069830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.436+0000 7f948b7fe700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f948c0ff6b0 0x7f948c069830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f948b7fe700 1 -- 172.21.15.49:0/691051795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484009d00 con 0x7f948c101240 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f948affd700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f948c0ff6b0 0x7f948c069830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f948b7fe700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c19db60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f948400d010 tx=0x7f9484009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f9483fff700 1 -- 172.21.15.49:0/691051795 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948400e040 con 0x7f948c101240 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f9483fff700 1 -- 172.21.15.49:0/691051795 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9484003e10 con 0x7f948c101240 2023-10-08T09:14:25.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c19e8a0 con 0x7f948c101240 2023-10-08T09:14:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f9483fff700 1 -- 172.21.15.49:0/691051795 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484012a10 con 0x7f948c101240 2023-10-08T09:14:25.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.437+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c19ecc0 con 0x7f948c101240 2023-10-08T09:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.438+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c106a00 con 0x7f948c101240 2023-10-08T09:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.439+0000 7f9483fff700 1 -- 172.21.15.49:0/691051795 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9484005630 con 0x7f948c101240 2023-10-08T09:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.439+0000 7f9483fff700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f94780a1510 0x7f94780a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.440+0000 7f9483fff700 1 -- 172.21.15.49:0/691051795 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f94840ce7f0 con 0x7f948c101240 2023-10-08T09:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.440+0000 7f948affd700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f94780a1510 0x7f94780a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.440+0000 7f948affd700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f94780a1510 0x7f94780a39d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f948c100700 tx=0x7f947c00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:25.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.444+0000 7f9483fff700 1 -- 172.21.15.49:0/691051795 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9484097a70 con 0x7f948c101240 2023-10-08T09:14:25.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.646+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f948c10afe0 con 0x7f948c101240 2023-10-08T09:14:25.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.647+0000 7f9483fff700 1 -- 172.21.15.49:0/691051795 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f948409b090 con 0x7f948c101240 2023-10-08T09:14:25.647 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:14:25.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f94780a1510 msgr2=0x7f94780a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f94780a1510 0x7f94780a39d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f948c100700 tx=0x7f947c00a680 comp rx=0 tx=0).stop 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 msgr2=0x7f948c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c19db60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f948400d010 tx=0x7f9484009430 comp rx=0 tx=0).stop 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 shutdown_connections 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f94780a1510 0x7f94780a39d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f948c101240 0x7f948c19db60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f948c0ff6b0 0x7f948c069830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 --2- 172.21.15.49:0/691051795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f948c1008a0 0x7f948c19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 >> 172.21.15.49:0/691051795 conn(0x7f948c0fac40 msgr2=0x7f948c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 shutdown_connections 2023-10-08T09:14:25.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.649+0000 7f9491599700 1 -- 172.21.15.49:0/691051795 wait complete. 2023-10-08T09:14:25.662 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:14:25.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 -- 172.21.15.49:0/574163709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd400ff6a0 msgr2=0x7fbd400ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 --2- 172.21.15.49:0/574163709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd400ff6a0 0x7fbd400ffac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0099c0 tx=0x7fbd3c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 -- 172.21.15.49:0/574163709 shutdown_connections 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 --2- 172.21.15.49:0/574163709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd40101230 0x7fbd40107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 --2- 172.21.15.49:0/574163709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd400ff6a0 0x7fbd400ffac0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 --2- 172.21.15.49:0/574163709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd40100890 0x7fbd40100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 -- 172.21.15.49:0/574163709 >> 172.21.15.49:0/574163709 conn(0x7fbd400fac10 msgr2=0x7fbd400fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 -- 172.21.15.49:0/574163709 shutdown_connections 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 -- 172.21.15.49:0/574163709 wait complete. 2023-10-08T09:14:25.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 Processor -- start 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 -- start start 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd400ff6a0 0x7fbd401955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.791+0000 7fbd48221700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd40100890 0x7fbd40195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd48221700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd40101230 0x7fbd4019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd48221700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4019d120 con 0x7fbd40100890 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd48221700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4019d260 con 0x7fbd400ff6a0 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd48221700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4019d3f0 con 0x7fbd40101230 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd40100890 0x7fbd40195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd40100890 0x7fbd40195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49040/0 (socket says 172.21.15.49:49040) 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 -- 172.21.15.49:0/3813044224 learned_addr learned my addr 172.21.15.49:0/3813044224 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd467be700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd40101230 0x7fbd4019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd45fbd700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd400ff6a0 0x7fbd401955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 -- 172.21.15.49:0/3813044224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd40101230 msgr2=0x7fbd4019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd40101230 0x7fbd4019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 -- 172.21.15.49:0/3813044224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd400ff6a0 msgr2=0x7fbd401955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd400ff6a0 0x7fbd401955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 -- 172.21.15.49:0/3813044224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0095f0 con 0x7fbd40100890 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd457bc700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd40100890 0x7fbd40195b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000d950 tx=0x7fbd3000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:25.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd36ffd700 1 -- 172.21.15.49:0/3813044224 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd30009940 con 0x7fbd40100890 2023-10-08T09:14:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4019d6d0 con 0x7fbd40100890 2023-10-08T09:14:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd36ffd700 1 -- 172.21.15.49:0/3813044224 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd30010460 con 0x7fbd40100890 2023-10-08T09:14:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd36ffd700 1 -- 172.21.15.49:0/3813044224 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000f620 con 0x7fbd40100890 2023-10-08T09:14:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.792+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4019dbe0 con 0x7fbd40100890 2023-10-08T09:14:25.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.793+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd40067530 con 0x7fbd40100890 2023-10-08T09:14:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.795+0000 7fbd36ffd700 1 -- 172.21.15.49:0/3813044224 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbd300108d0 con 0x7fbd40100890 2023-10-08T09:14:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.795+0000 7fbd36ffd700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd2c0a1510 0x7fbd2c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.795+0000 7fbd45fbd700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd2c0a1510 0x7fbd2c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.795+0000 7fbd36ffd700 1 -- 172.21.15.49:0/3813044224 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbd300cd030 con 0x7fbd40100890 2023-10-08T09:14:25.798 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.796+0000 7fbd45fbd700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd2c0a1510 0x7fbd2c0a39d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b580 tx=0x7fbd3c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:25.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:25.799+0000 7fbd36ffd700 1 -- 172.21.15.49:0/3813044224 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbd30096220 con 0x7fbd40100890 2023-10-08T09:14:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:25 smithi184 ceph-mon[103675]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:25 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1091977316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:25.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:25 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3120202982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:25.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:25 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/691051795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:26.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.001+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd4010b040 con 0x7fbd40100890 2023-10-08T09:14:26.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.001+0000 7fbd36ffd700 1 -- 172.21.15.49:0/3813044224 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd30099840 con 0x7fbd40100890 2023-10-08T09:14:26.001 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd2c0a1510 msgr2=0x7fbd2c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd2c0a1510 0x7fbd2c0a39d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b580 tx=0x7fbd3c005fb0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd40100890 msgr2=0x7fbd40195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd40100890 0x7fbd40195b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000d950 tx=0x7fbd3000dc60 comp rx=0 tx=0).stop 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 shutdown_connections 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd2c0a1510 0x7fbd2c0a39d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd40101230 0x7fbd4019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd400ff6a0 0x7fbd401955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 --2- 172.21.15.49:0/3813044224 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd40100890 0x7fbd40195b30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 >> 172.21.15.49:0/3813044224 conn(0x7fbd400fac10 msgr2=0x7fbd40103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:26.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 shutdown_connections 2023-10-08T09:14:26.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.003+0000 7fbd48221700 1 -- 172.21.15.49:0/3813044224 wait complete. 2023-10-08T09:14:26.015 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:25 smithi049 ceph-mon[99485]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:26.015 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:25 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1091977316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:26.015 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:25 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3120202982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:26.015 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:25 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/691051795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:26.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:14:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:25 smithi132 ceph-mon[103389]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:25 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1091977316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:25 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3120202982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:25 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/691051795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 -- 172.21.15.49:0/1079436215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 msgr2=0x7fa298100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 --2- 172.21.15.49:0/1079436215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa298100420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa280009990 tx=0x7fa280009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 -- 172.21.15.49:0/1079436215 shutdown_connections 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 --2- 172.21.15.49:0/1079436215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2980fee50 0x7fa2980ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 --2- 172.21.15.49:0/1079436215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa298100960 0x7fa298107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 --2- 172.21.15.49:0/1079436215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa298100420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 -- 172.21.15.49:0/1079436215 >> 172.21.15.49:0/1079436215 conn(0x7fa2980fa3c0 msgr2=0x7fa2980fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:26.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 -- 172.21.15.49:0/1079436215 shutdown_connections 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.144+0000 7fa29d412700 1 -- 172.21.15.49:0/1079436215 wait complete. 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 Processor -- start 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 -- start start 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2980fee50 0x7fa298108580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa29819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa298100960 0x7fa29819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29819e230 con 0x7fa2980fffa0 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29819e3a0 con 0x7fa298100960 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa29d412700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29819e560 con 0x7fa2980fee50 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa2967fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa29819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa2967fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa29819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49054/0 (socket says 172.21.15.49:49054) 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.145+0000 7fa2967fc700 1 -- 172.21.15.49:0/1101340241 learned_addr learned my addr 172.21.15.49:0/1101340241 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa296ffd700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2980fee50 0x7fa298108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2977fe700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa298100960 0x7fa29819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2967fc700 1 -- 172.21.15.49:0/1101340241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2980fee50 msgr2=0x7fa298108580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2967fc700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2980fee50 0x7fa298108580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2967fc700 1 -- 172.21.15.49:0/1101340241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa298100960 msgr2=0x7fa29819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2967fc700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa298100960 0x7fa29819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2967fc700 1 -- 172.21.15.49:0/1101340241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2800095f0 con 0x7fa2980fffa0 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2977fe700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa298100960 0x7fa29819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa2967fc700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa29819b420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa2800095c0 tx=0x7fa28000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa28ffff700 1 -- 172.21.15.49:0/1101340241 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28001d070 con 0x7fa2980fffa0 2023-10-08T09:14:26.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa28ffff700 1 -- 172.21.15.49:0/1101340241 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa280022470 con 0x7fa2980fffa0 2023-10-08T09:14:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29819e780 con 0x7fa2980fffa0 2023-10-08T09:14:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa28ffff700 1 -- 172.21.15.49:0/1101340241 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280017650 con 0x7fa2980fffa0 2023-10-08T09:14:26.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.146+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29819eba0 con 0x7fa2980fffa0 2023-10-08T09:14:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.147+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa298106210 con 0x7fa2980fffa0 2023-10-08T09:14:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.148+0000 7fa28ffff700 1 -- 172.21.15.49:0/1101340241 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa280022850 con 0x7fa2980fffa0 2023-10-08T09:14:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.149+0000 7fa28ffff700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2840a1510 0x7fa2840a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.149+0000 7fa28ffff700 1 -- 172.21.15.49:0/1101340241 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa2800ce480 con 0x7fa2980fffa0 2023-10-08T09:14:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.149+0000 7fa296ffd700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2840a1510 0x7fa2840a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.149+0000 7fa296ffd700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2840a1510 0x7fa2840a39d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa288005950 tx=0x7fa28800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:26.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.153+0000 7fa28ffff700 1 -- 172.21.15.49:0/1101340241 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa2800978d0 con 0x7fa2980fffa0 2023-10-08T09:14:26.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.355+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa29810b070 con 0x7fa2980fffa0 2023-10-08T09:14:26.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.356+0000 7fa28ffff700 1 -- 172.21.15.49:0/1101340241 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa280027020 con 0x7fa2980fffa0 2023-10-08T09:14:26.356 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:14:26.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2840a1510 msgr2=0x7fa2840a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2840a1510 0x7fa2840a39d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa288005950 tx=0x7fa28800b410 comp rx=0 tx=0).stop 2023-10-08T09:14:26.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 msgr2=0x7fa29819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa29819b420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa2800095c0 tx=0x7fa28000f690 comp rx=0 tx=0).stop 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 shutdown_connections 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2840a1510 0x7fa2840a39d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2980fee50 0x7fa298108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa298100960 0x7fa29819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 --2- 172.21.15.49:0/1101340241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2980fffa0 0x7fa29819b420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 >> 172.21.15.49:0/1101340241 conn(0x7fa2980fa3c0 msgr2=0x7fa298103350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 shutdown_connections 2023-10-08T09:14:26.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.358+0000 7fa29d412700 1 -- 172.21.15.49:0/1101340241 wait complete. 2023-10-08T09:14:26.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:14:26.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 -- 172.21.15.49:0/3065384964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84101dd0 msgr2=0x7f9a841021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 --2- 172.21.15.49:0/3065384964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84101dd0 0x7f9a841021f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009990 tx=0x7f9a78009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 -- 172.21.15.49:0/3065384964 shutdown_connections 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 --2- 172.21.15.49:0/3065384964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a840ff1e0 0x7f9a84108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 --2- 172.21.15.49:0/3065384964 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a84101dd0 0x7f9a841021f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 --2- 172.21.15.49:0/3065384964 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a840fe840 0x7f9a840feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 -- 172.21.15.49:0/3065384964 >> 172.21.15.49:0/3065384964 conn(0x7f9a840fa3a0 msgr2=0x7f9a840fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 -- 172.21.15.49:0/3065384964 shutdown_connections 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.500+0000 7f9a8acce700 1 -- 172.21.15.49:0/3065384964 wait complete. 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 Processor -- start 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 -- start start 2023-10-08T09:14:26.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a840fe840 0x7f9a84196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a840ff1e0 0x7f9a84196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a84101dd0 0x7f9a8419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419e230 con 0x7f9a840ff1e0 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419e370 con 0x7f9a84101dd0 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8acce700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419e500 con 0x7f9a840fe840 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a840ff1e0 0x7f9a84196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a840ff1e0 0x7f9a84196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49068/0 (socket says 172.21.15.49:49068) 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 -- 172.21.15.49:0/2331722874 learned_addr learned my addr 172.21.15.49:0/2331722874 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a88a6a700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a840fe840 0x7f9a84196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a8926b700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a84101dd0 0x7f9a8419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 -- 172.21.15.49:0/2331722874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a840fe840 msgr2=0x7f9a84196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a840fe840 0x7f9a84196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 -- 172.21.15.49:0/2331722874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a84101dd0 msgr2=0x7f9a8419dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a84101dd0 0x7f9a8419dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 -- 172.21.15.49:0/2331722874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780095f0 con 0x7f9a840ff1e0 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.501+0000 7f9a83fff700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a840ff1e0 0x7f9a84196c40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000e9a0 tx=0x7f9a7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.502+0000 7f9a81ffb700 1 -- 172.21.15.49:0/2331722874 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70009790 con 0x7f9a840ff1e0 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.502+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8419e7e0 con 0x7f9a840ff1e0 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.502+0000 7f9a81ffb700 1 -- 172.21.15.49:0/2331722874 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a70010430 con 0x7f9a840ff1e0 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.502+0000 7f9a81ffb700 1 -- 172.21.15.49:0/2331722874 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70018680 con 0x7f9a840ff1e0 2023-10-08T09:14:26.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.502+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8419ecf0 con 0x7f9a840ff1e0 2023-10-08T09:14:26.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.503+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a84107e60 con 0x7f9a840ff1e0 2023-10-08T09:14:26.506 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.504+0000 7f9a81ffb700 1 -- 172.21.15.49:0/2331722874 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9a700105a0 con 0x7f9a840ff1e0 2023-10-08T09:14:26.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.504+0000 7f9a81ffb700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a740a1510 0x7f9a740a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.505+0000 7f9a81ffb700 1 -- 172.21.15.49:0/2331722874 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a70014070 con 0x7f9a840ff1e0 2023-10-08T09:14:26.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.505+0000 7f9a88a6a700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a740a1510 0x7f9a740a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.505+0000 7f9a88a6a700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a740a1510 0x7f9a740a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b680 tx=0x7f9a78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:26.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.508+0000 7f9a81ffb700 1 -- 172.21.15.49:0/2331722874 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9a70096730 con 0x7f9a840ff1e0 2023-10-08T09:14:26.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.709+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9a8410c0c0 con 0x7f9a840ff1e0 2023-10-08T09:14:26.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.709+0000 7f9a81ffb700 1 -- 172.21.15.49:0/2331722874 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9a70099d50 con 0x7f9a840ff1e0 2023-10-08T09:14:26.710 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:26.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a740a1510 msgr2=0x7f9a740a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a740a1510 0x7f9a740a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b680 tx=0x7f9a78005fd0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a840ff1e0 msgr2=0x7f9a84196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a840ff1e0 0x7f9a84196c40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000e9a0 tx=0x7f9a7000ecb0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 shutdown_connections 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a740a1510 0x7f9a740a39d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a840fe840 0x7f9a84196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a84101dd0 0x7f9a8419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 --2- 172.21.15.49:0/2331722874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a840ff1e0 0x7f9a84196c40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 >> 172.21.15.49:0/2331722874 conn(0x7f9a840fa3a0 msgr2=0x7f9a84104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:26.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 shutdown_connections 2023-10-08T09:14:26.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.712+0000 7f9a8acce700 1 -- 172.21.15.49:0/2331722874 wait complete. 2023-10-08T09:14:26.725 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:14:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/3251229369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 msgr2=0x7f6fac108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/3251229369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac108820 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400b0a0 tx=0x7f6fa400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/3251229369 shutdown_connections 2023-10-08T09:14:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/3251229369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac108820 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/3251229369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fac1011b0 0x7f6fac1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/3251229369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fac103ae0 0x7f6fac105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/3251229369 >> 172.21.15.49:0/3251229369 conn(0x7f6fac0fab40 msgr2=0x7f6fac0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/3251229369 shutdown_connections 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.850+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/3251229369 wait complete. 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 Processor -- start 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 -- start start 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fac1011b0 0x7f6fac100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fac103ae0 0x7f6fac0ff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac0ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0ffda0 con 0x7f6fac106410 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac0fff10 con 0x7f6fac1011b0 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fb1ed4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac1a0670 con 0x7f6fac103ae0 2023-10-08T09:14:26.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac0ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fabfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac0ff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49084/0 (socket says 172.21.15.49:49084) 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fabfff700 1 -- 172.21.15.49:0/2823350584 learned_addr learned my addr 172.21.15.49:0/2823350584 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.851+0000 7f6fab7fe700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fac1011b0 0x7f6fac100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6faaffd700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fac103ae0 0x7f6fac0ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fabfff700 1 -- 172.21.15.49:0/2823350584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fac103ae0 msgr2=0x7f6fac0ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fabfff700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fac103ae0 0x7f6fac0ff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fabfff700 1 -- 172.21.15.49:0/2823350584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fac1011b0 msgr2=0x7f6fac100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fabfff700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fac1011b0 0x7f6fac100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fabfff700 1 -- 172.21.15.49:0/2823350584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4009d00 con 0x7f6fac106410 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6faaffd700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fac103ae0 0x7f6fac0ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:26.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fabfff700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac0ff6a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4004d00 tx=0x7f6fa4004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fa8ff9700 1 -- 172.21.15.49:0/2823350584 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa400e040 con 0x7f6fac106410 2023-10-08T09:14:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac1a0870 con 0x7f6fac106410 2023-10-08T09:14:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fa8ff9700 1 -- 172.21.15.49:0/2823350584 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fa4009440 con 0x7f6fac106410 2023-10-08T09:14:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fa8ff9700 1 -- 172.21.15.49:0/2823350584 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa401ba80 con 0x7f6fac106410 2023-10-08T09:14:26.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.852+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac1a0d40 con 0x7f6fac106410 2023-10-08T09:14:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.853+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac0fc6c0 con 0x7f6fac106410 2023-10-08T09:14:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.854+0000 7f6fa8ff9700 1 -- 172.21.15.49:0/2823350584 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f6fa401bbe0 con 0x7f6fac106410 2023-10-08T09:14:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.855+0000 7f6fa8ff9700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6f940a1510 0x7f6f940a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.855+0000 7f6fab7fe700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6f940a1510 0x7f6f940a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.855+0000 7f6fa8ff9700 1 -- 172.21.15.49:0/2823350584 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6fa40cf7e0 con 0x7f6fac106410 2023-10-08T09:14:26.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.855+0000 7f6fab7fe700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6f940a1510 0x7f6f940a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005ea0 tx=0x7f6f9c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:26.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:26.858+0000 7f6fa8ff9700 1 -- 172.21.15.49:0/2823350584 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6fa4098a60 con 0x7f6fac106410 2023-10-08T09:14:26.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:26 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3813044224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:26.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:26 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1101340241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:26 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3813044224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:26 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1101340241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:27.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.106+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fac10bed0 con 0x7f6fac106410 2023-10-08T09:14:27.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.107+0000 7f6fa8ff9700 1 -- 172.21.15.49:0/2823350584 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f6fa409c080 con 0x7f6fac106410 2023-10-08T09:14:27.108 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:27.108 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:14:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6f940a1510 msgr2=0x7f6f940a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6f940a1510 0x7f6f940a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c005ea0 tx=0x7f6f9c005e30 comp rx=0 tx=0).stop 2023-10-08T09:14:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 msgr2=0x7f6fac0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac0ff6a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4004d00 tx=0x7f6fa4004de0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 shutdown_connections 2023-10-08T09:14:27.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f6f940a1510 0x7f6f940a39d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6fac106410 0x7f6fac0ff6a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fac1011b0 0x7f6fac100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 --2- 172.21.15.49:0/2823350584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6fac103ae0 0x7f6fac0ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.109+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 >> 172.21.15.49:0/2823350584 conn(0x7f6fac0fab40 msgr2=0x7f6fac101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.110+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 shutdown_connections 2023-10-08T09:14:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.110+0000 7f6fb1ed4700 1 -- 172.21.15.49:0/2823350584 wait complete. 2023-10-08T09:14:27.111 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:14:27.122 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:26 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3813044224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:27.122 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:26 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1101340241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:27.123 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 -- 172.21.15.49:0/1484404806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8101130 msgr2=0x7f52d81079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 --2- 172.21.15.49:0/1484404806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8101130 0x7f52d81079e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f52d000b0d0 tx=0x7f52d000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 -- 172.21.15.49:0/1484404806 shutdown_connections 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 --2- 172.21.15.49:0/1484404806 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8101130 0x7f52d81079e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 --2- 172.21.15.49:0/1484404806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80ff5a0 0x7f52d80ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 --2- 172.21.15.49:0/1484404806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d8100790 0x7f52d8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 -- 172.21.15.49:0/1484404806 >> 172.21.15.49:0/1484404806 conn(0x7f52d80fab10 msgr2=0x7f52d80fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 -- 172.21.15.49:0/1484404806 shutdown_connections 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 -- 172.21.15.49:0/1484404806 wait complete. 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 Processor -- start 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.248+0000 7f52e048a700 1 -- start start 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52e048a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80ff5a0 0x7f52d8068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52e048a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8100790 0x7f52d819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52e048a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d8101130 0x7f52d819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52e048a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d819e1a0 con 0x7f52d8100790 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52e048a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d819e310 con 0x7f52d80ff5a0 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52e048a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d819e4d0 con 0x7f52d8101130 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8100790 0x7f52d819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8100790 0x7f52d819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49096/0 (socket says 172.21.15.49:49096) 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 -- 172.21.15.49:0/2993221232 learned_addr learned my addr 172.21.15.49:0/2993221232 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dea27700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d8101130 0x7f52d819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52de226700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80ff5a0 0x7f52d8068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 -- 172.21.15.49:0/2993221232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d8101130 msgr2=0x7f52d819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d8101130 0x7f52d819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 -- 172.21.15.49:0/2993221232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80ff5a0 msgr2=0x7f52d8068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80ff5a0 0x7f52d8068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 -- 172.21.15.49:0/2993221232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0009d00 con 0x7f52d8100790 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dea27700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d8101130 0x7f52d819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52dda25700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8100790 0x7f52d819b390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52d400d950 tx=0x7f52d400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52cf7fe700 1 -- 172.21.15.49:0/2993221232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d4009b40 con 0x7f52d8100790 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d819e750 con 0x7f52d8100790 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.249+0000 7f52cf7fe700 1 -- 172.21.15.49:0/2993221232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d4009ca0 con 0x7f52d8100790 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.250+0000 7f52cf7fe700 1 -- 172.21.15.49:0/2993221232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400f620 con 0x7f52d8100790 2023-10-08T09:14:27.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.250+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d819ec60 con 0x7f52d8100790 2023-10-08T09:14:27.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.250+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d8106950 con 0x7f52d8100790 2023-10-08T09:14:27.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.252+0000 7f52cf7fe700 1 -- 172.21.15.49:0/2993221232 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f52d4010460 con 0x7f52d8100790 2023-10-08T09:14:27.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.253+0000 7f52cf7fe700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f52c40a1510 0x7f52c40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.253+0000 7f52cf7fe700 1 -- 172.21.15.49:0/2993221232 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52d40cd5c0 con 0x7f52d8100790 2023-10-08T09:14:27.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.253+0000 7f52de226700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f52c40a1510 0x7f52c40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.253+0000 7f52de226700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f52c40a1510 0x7f52c40a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:27.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.256+0000 7f52cf7fe700 1 -- 172.21.15.49:0/2993221232 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f52d4096750 con 0x7f52d8100790 2023-10-08T09:14:27.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.459+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52d810adc0 con 0x7f52d8100790 2023-10-08T09:14:27.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.461+0000 7f52cf7fe700 1 -- 172.21.15.49:0/2993221232 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24441 (secure 0 0 0) 0x7f52d4016090 con 0x7f52d8100790 2023-10-08T09:14:27.462 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:27.462 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":88,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:13:37.433726+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:14:27.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f52c40a1510 msgr2=0x7f52c40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f52c40a1510 0x7f52c40a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c8005e50 comp rx=0 tx=0).stop 2023-10-08T09:14:27.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8100790 msgr2=0x7f52d819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8100790 0x7f52d819b390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f52d400d950 tx=0x7f52d400dd10 comp rx=0 tx=0).stop 2023-10-08T09:14:27.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 shutdown_connections 2023-10-08T09:14:27.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f52c40a1510 0x7f52c40a39d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d8101130 0x7f52d819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52d80ff5a0 0x7f52d8068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 --2- 172.21.15.49:0/2993221232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f52d8100790 0x7f52d819b390 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 >> 172.21.15.49:0/2993221232 conn(0x7f52d80fab10 msgr2=0x7f52d8103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:27.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 shutdown_connections 2023-10-08T09:14:27.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.463+0000 7f52e048a700 1 -- 172.21.15.49:0/2993221232 wait complete. 2023-10-08T09:14:27.477 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:14:27.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 -- 172.21.15.49:0/2213660285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed41006c0 msgr2=0x7f1ed4100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 --2- 172.21.15.49:0/2213660285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed41006c0 0x7f1ed4100ae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40099c0 tx=0x7f1ec4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 -- 172.21.15.49:0/2213660285 shutdown_connections 2023-10-08T09:14:27.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 --2- 172.21.15.49:0/2213660285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed4102250 0x7f1ed4108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 --2- 172.21.15.49:0/2213660285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed41006c0 0x7f1ed4100ae0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 --2- 172.21.15.49:0/2213660285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ed41018b0 0x7f1ed4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 -- 172.21.15.49:0/2213660285 >> 172.21.15.49:0/2213660285 conn(0x7f1ed40fbc50 msgr2=0x7f1ed40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 -- 172.21.15.49:0/2213660285 shutdown_connections 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 -- 172.21.15.49:0/2213660285 wait complete. 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 Processor -- start 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.602+0000 7f1eda377700 1 -- start start 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1eda377700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41006c0 0x7f1ed4074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1eda377700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ed41018b0 0x7f1ed4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1eda377700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed4102250 0x7f1ed4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1eda377700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40738c0 con 0x7f1ed4102250 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1eda377700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed4073a30 con 0x7f1ed41006c0 2023-10-08T09:14:27.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1eda377700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed41a06b0 con 0x7f1ed41018b0 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed4102250 0x7f1ed4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed4102250 0x7f1ed4073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49112/0 (socket says 172.21.15.49:49112) 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 -- 172.21.15.49:0/1263873206 learned_addr learned my addr 172.21.15.49:0/1263873206 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed37fe700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ed41018b0 0x7f1ed4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed3fff700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41006c0 0x7f1ed4074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 -- 172.21.15.49:0/1263873206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ed41018b0 msgr2=0x7f1ed4072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ed41018b0 0x7f1ed4072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 -- 172.21.15.49:0/1263873206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41006c0 msgr2=0x7f1ed4074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41006c0 0x7f1ed4074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 -- 172.21.15.49:0/1263873206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40095f0 con 0x7f1ed4102250 2023-10-08T09:14:27.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed8914700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed4102250 0x7f1ed4073120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007ee0 tx=0x7f1ecc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:27.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed17fa700 1 -- 172.21.15.49:0/1263873206 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00dd70 con 0x7f1ed4102250 2023-10-08T09:14:27.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1ed17fa700 1 -- 172.21.15.49:0/1263873206 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ecc00f040 con 0x7f1ed4102250 2023-10-08T09:14:27.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.603+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed41a0940 con 0x7f1ed4102250 2023-10-08T09:14:27.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.604+0000 7f1ed17fa700 1 -- 172.21.15.49:0/1263873206 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0048f0 con 0x7f1ed4102250 2023-10-08T09:14:27.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.604+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed41a0e00 con 0x7f1ed4102250 2023-10-08T09:14:27.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.604+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed4107b90 con 0x7f1ed4102250 2023-10-08T09:14:27.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.606+0000 7f1ed17fa700 1 -- 172.21.15.49:0/1263873206 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1ecc003a00 con 0x7f1ed4102250 2023-10-08T09:14:27.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.606+0000 7f1ed17fa700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ebc0a1510 0x7f1ebc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:27.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.606+0000 7f1ed17fa700 1 -- 172.21.15.49:0/1263873206 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1ecc0cd2d0 con 0x7f1ed4102250 2023-10-08T09:14:27.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.606+0000 7f1ed3fff700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ebc0a1510 0x7f1ebc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:27.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.607+0000 7f1ed3fff700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ebc0a1510 0x7f1ebc0a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40036f0 tx=0x7f1ec4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:27.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.610+0000 7f1ed17fa700 1 -- 172.21.15.49:0/1263873206 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ecc0964c0 con 0x7f1ed4102250 2023-10-08T09:14:27.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.857+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed4066bc0 con 0x7f1ed4102250 2023-10-08T09:14:27.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.857+0000 7f1ed17fa700 1 -- 172.21.15.49:0/1263873206 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f1ecc023090 con 0x7f1ed4102250 2023-10-08T09:14:27.859 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:27.859 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:14:27.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.859+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ebc0a1510 msgr2=0x7f1ebc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.859+0000 7f1eda377700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ebc0a1510 0x7f1ebc0a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40036f0 tx=0x7f1ec4005fb0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.859+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed4102250 msgr2=0x7f1ed4073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:27.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.859+0000 7f1eda377700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed4102250 0x7f1ed4073120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc007ee0 tx=0x7f1ecc00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 shutdown_connections 2023-10-08T09:14:27.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ebc0a1510 0x7f1ebc0a39d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1ed4102250 0x7f1ed4073120 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ed41006c0 0x7f1ed4074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 --2- 172.21.15.49:0/1263873206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ed41018b0 0x7f1ed4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:27.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 >> 172.21.15.49:0/1263873206 conn(0x7f1ed40fbc50 msgr2=0x7f1ed4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:27.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 shutdown_connections 2023-10-08T09:14:27.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:27.860+0000 7f1eda377700 1 -- 172.21.15.49:0/1263873206 wait complete. 2023-10-08T09:14:27.862 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:14:27.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:27 smithi049 ceph-mon[99485]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:27.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:27 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2331722874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:27.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:27 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2823350584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:27.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:27 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2993221232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:27.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-08T09:13:02.804952+0000', 'modified': '2023-10-08T09:13:33.418895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24638, 'mds_1': 24616, 'mds_2': 24608, 'mds_3': 24553, 'mds_4': 24545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14721': {'gid': 14721, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6839/3863085428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 3863085428}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 3863085428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14733': {'gid': 14733, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6841/1275166284', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6840', 'nonce': 1275166284}, {'type': 'v1', 'addr': '172.21.15.49:6841', 'nonce': 1275166284}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24545': {'gid': 24545, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.132:6833/3058040154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 3058040154}, {'type': 'v1', 'addr': '172.21.15.132:6833', 'nonce': 3058040154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'l', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.184:6833/3616311663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 3616311663}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 3616311663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24575': {'gid': 24575, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6835/738661164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6834', 'nonce': 738661164}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 738661164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24592': {'gid': 24592, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/356478734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 356478734}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 356478734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24604': {'gid': 24604, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/3605975147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 3605975147}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 3605975147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24608': {'gid': 24608, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.132:6837/2064856316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 2064856316}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 2064856316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24616': {'gid': 24616, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.184:6839/2795080121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 2795080121}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 2795080121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/1629770277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 1629770277}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 1629770277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T09:14:27.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-08T09:14:27.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-10-08T09:14:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:27 smithi184 ceph-mon[103675]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:27 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2331722874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:27 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2823350584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:27 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2993221232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:28.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 -- 172.21.15.49:0/250127489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 msgr2=0x7f0360107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 --2- 172.21.15.49:0/250127489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f0360107ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f035800b0d0 tx=0x7f035800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 -- 172.21.15.49:0/250127489 shutdown_connections 2023-10-08T09:14:28.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 --2- 172.21.15.49:0/250127489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f0360107ae0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 --2- 172.21.15.49:0/250127489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03600ff6a0 0x7f03600ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 --2- 172.21.15.49:0/250127489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0360100890 0x7f0360100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 -- 172.21.15.49:0/250127489 >> 172.21.15.49:0/250127489 conn(0x7f03600fac10 msgr2=0x7f03600fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 -- 172.21.15.49:0/250127489 shutdown_connections 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.001+0000 7f0365f7b700 1 -- 172.21.15.49:0/250127489 wait complete. 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 Processor -- start 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 -- start start 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03600ff6a0 0x7f0360068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0360100890 0x7f036019b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f036019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019e380 con 0x7f0360101230 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019e4f0 con 0x7f0360100890 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.002+0000 7f0365f7b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036019e6b0 con 0x7f03600ff6a0 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f036019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f036019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49126/0 (socket says 172.21.15.49:49126) 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 -- 172.21.15.49:0/2871626127 learned_addr learned my addr 172.21.15.49:0/2871626127 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035f7fe700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03600ff6a0 0x7f0360068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035effd700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0360100890 0x7f036019b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 -- 172.21.15.49:0/2871626127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03600ff6a0 msgr2=0x7f0360068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03600ff6a0 0x7f0360068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 -- 172.21.15.49:0/2871626127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0360100890 msgr2=0x7f036019b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0360100890 0x7f036019b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 -- 172.21.15.49:0/2871626127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358009d00 con 0x7f0360101230 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035ffff700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f036019db90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0358007b60 tx=0x7f0358008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035cff9700 1 -- 172.21.15.49:0/2871626127 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035800e040 con 0x7f0360101230 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036019e8d0 con 0x7f0360101230 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035cff9700 1 -- 172.21.15.49:0/2871626127 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f035801d930 con 0x7f0360101230 2023-10-08T09:14:28.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f035cff9700 1 -- 172.21.15.49:0/2871626127 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035801ba40 con 0x7f0360101230 2023-10-08T09:14:28.009 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.003+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036019ed00 con 0x7f0360101230 2023-10-08T09:14:28.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.005+0000 7f035cff9700 1 -- 172.21.15.49:0/2871626127 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f035801daa0 con 0x7f0360101230 2023-10-08T09:14:28.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.006+0000 7f035cff9700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f034c0a1510 0x7f034c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.006+0000 7f035f7fe700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f034c0a1510 0x7f034c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.006+0000 7f035cff9700 1 -- 172.21.15.49:0/2871626127 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03580ce7a0 con 0x7f0360101230 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f035f7fe700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f034c0a1510 0x7f034c0a39d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f03601006f0 tx=0x7f0350009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 Processor -- start 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 -- start start 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf010 0x559a1dedf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf970 0x559a1dee0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1dee0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1de47410 con 0x559a1dedf970 2023-10-08T09:14:28.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1de38bd0 con 0x559a1dedf010 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f0372164b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1de364f0 con 0x559a1dee07b0 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.007+0000 7f035effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf970 0x559a1dee0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf970 0x559a1dee0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49132/0 (socket says 172.21.15.49:49132) 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035effd700 1 -- 172.21.15.49:0/338364580 learned_addr learned my addr 172.21.15.49:0/338364580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035ffff700 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1dee0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035f7fe700 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf010 0x559a1dedf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035ffff700 1 -- 172.21.15.49:0/338364580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf010 msgr2=0x559a1dedf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035ffff700 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf010 0x559a1dedf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035ffff700 1 -- 172.21.15.49:0/338364580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf970 msgr2=0x559a1dee0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035ffff700 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf970 0x559a1dee0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035ffff700 1 -- 172.21.15.49:0/338364580 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a1dc5cd90 con 0x559a1dee07b0 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035effd700 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf970 0x559a1dee0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f035ffff700 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1dee0c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03580cbe40 tx=0x7f0358097b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e448561880fb28e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f0355ffb700 1 -- 172.21.15.49:0/338364580 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035809bde0 con 0x559a1dee07b0 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f0355ffb700 1 -- 172.21.15.49:0/338364580 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f035809b6e0 con 0x559a1dee07b0 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.008+0000 7f0355ffb700 1 -- 172.21.15.49:0/338364580 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03580cc090 con 0x559a1dee07b0 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 -- 172.21.15.49:0/338364580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 msgr2=0x559a1dee0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1dee0c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03580cbe40 tx=0x7f0358097b10 comp rx=0 tx=0).stop 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 -- 172.21.15.49:0/338364580 shutdown_connections 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1dee0c40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf010 0x559a1dedf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 --2- 172.21.15.49:0/338364580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf970 0x559a1dee0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 -- 172.21.15.49:0/338364580 >> 172.21.15.49:0/338364580 conn(0x559a1de53210 msgr2=0x559a1ded9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 -- 172.21.15.49:0/338364580 shutdown_connections 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 -- 172.21.15.49:0/338364580 wait complete. 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.009+0000 7f0372164b80 1 -- start start 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 1 Processor -- start 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf010 0x559a1dee8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1dee90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dee07b0 0x559a1def00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1de31de0 con 0x559a1dee07b0 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1ddce6c0 con 0x559a1dedf970 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1ddce360 con 0x559a1dedf010 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f0372164b80 10 client.0 _ll_register_callbacks cb 0x559a1dc2b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dee07b0 0x559a1def00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dee07b0 0x559a1def00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49140/0 (socket says 172.21.15.49:49140) 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 -- 172.21.15.49:0/1577717109 learned_addr learned my addr 172.21.15.49:0/1577717109 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035f7fe700 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf010 0x559a1dee8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035effd700 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1dee90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 -- 172.21.15.49:0/1577717109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf010 msgr2=0x559a1dee8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf010 0x559a1dee8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 -- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 msgr2=0x559a1dee90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1dee90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 -- 172.21.15.49:0/1577717109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0348009910 con 0x559a1dee07b0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f035ffff700 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dee07b0 0x559a1def00f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f035809b670 tx=0x7f03580095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7088dfd7921db678 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.010+0000 7f033effd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f033effd700 1 -- 172.21.15.49:0/1577717109 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03580a39e0 con 0x559a1dee07b0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f033effd700 1 -- 172.21.15.49:0/1577717109 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f03580860c0 con 0x559a1dee07b0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a1ddf9e60 con 0x559a1dee07b0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f033effd700 1 -- 172.21.15.49:0/1577717109 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f035809abd0 con 0x559a1dee07b0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 10 client.15327 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f036019ed00 con 0x559a1dee07b0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f033effd700 1 -- 172.21.15.49:0/1577717109 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a1ddf9e60 con 0x559a1dee07b0 2023-10-08T09:14:28.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f033effd700 1 -- 172.21.15.49:0/1577717109 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f035801bba0 con 0x559a1dee07b0 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 10 client.15327 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 20 client.15327 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1def4310 0x559a1def30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 4 client.15327 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 --> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a1de2adc0 con 0x559a1def4310 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.011+0000 7f035f7fe700 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1def4310 0x559a1def30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.012+0000 7f033effd700 10 client.15327 ms_handle_connect on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.012+0000 7f035f7fe700 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1def4310 0x559a1def30c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f0a6fe8b049aa61 server_cookie=a39c56c7cb9be12e in_seq=0 out_seq=0 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f033effd700 1 -- 172.21.15.49:0/1577717109 <== mds.4 v2:172.21.15.132:6832/3058040154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559a1de2adc0 con 0x559a1def4310 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f033effd700 10 client.15327 handle_command_reply: tid=0 2023-10-08T09:14:28.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 client.15327 shutdown 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dee07b0 msgr2=0x559a1def00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dee07b0 0x559a1def00f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f035809b670 tx=0x7f03580095e0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 shutdown_connections 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf010 0x559a1dee8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1def4310 0x559a1def30c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dee07b0 0x559a1def00f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f033effd700 0 client.15327 ms_handle_reset on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 --2- 172.21.15.49:0/1577717109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1dee90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 >> 172.21.15.49:0/1577717109 conn(0x559a1de53210 msgr2=0x559a1ded9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 shutdown_connections 2023-10-08T09:14:28.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 1 -- 172.21.15.49:0/1577717109 wait complete. 2023-10-08T09:14:28.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.013+0000 7f0372164b80 20 client.15327 trim_cache size 0 max 16384 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 Processor -- start 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 -- start start 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1def07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf970 0x559a1def00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dee07b0 0x559a1dee8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1dcb28d0 con 0x559a1dedf010 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1de2adc0 con 0x559a1dee07b0 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f0372164b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1ddf9e60 con 0x559a1dedf970 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1def07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1def07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49144/0 (socket says 172.21.15.49:49144) 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 -- 172.21.15.49:0/4080405446 learned_addr learned my addr 172.21.15.49:0/4080405446 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035effd700 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf970 0x559a1def00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035ffff700 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dee07b0 0x559a1dee8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 -- 172.21.15.49:0/4080405446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf970 msgr2=0x559a1def00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf970 0x559a1def00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 -- 172.21.15.49:0/4080405446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dee07b0 msgr2=0x559a1dee8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dee07b0 0x559a1dee8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.031+0000 7f035f7fe700 1 -- 172.21.15.49:0/4080405446 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a1dc45c10 con 0x559a1dedf010 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f035f7fe700 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1def07a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x559a1def56a0 tx=0x7f0350014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b73b2d96431f5e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f03567fc700 1 -- 172.21.15.49:0/4080405446 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035001db80 con 0x559a1dedf010 2023-10-08T09:14:28.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f03567fc700 1 -- 172.21.15.49:0/4080405446 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0350023040 con 0x559a1dedf010 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f03567fc700 1 -- 172.21.15.49:0/4080405446 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0350003680 con 0x559a1dedf010 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 -- 172.21.15.49:0/4080405446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 msgr2=0x559a1def07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1def07a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x559a1def56a0 tx=0x7f0350014000 comp rx=0 tx=0).stop 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 -- 172.21.15.49:0/4080405446 shutdown_connections 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dee07b0 0x559a1dee8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1def07a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 --2- 172.21.15.49:0/4080405446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dedf970 0x559a1def00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 -- 172.21.15.49:0/4080405446 >> 172.21.15.49:0/4080405446 conn(0x559a1de53210 msgr2=0x559a1ded5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 -- 172.21.15.49:0/4080405446 shutdown_connections 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.032+0000 7f0372164b80 1 -- 172.21.15.49:0/4080405446 wait complete. 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 -- start start 2023-10-08T09:14:28.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 Processor -- start 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1deeb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1deeb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1df47d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1df48240 con 0x559a1dedf010 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1df48380 con 0x559a1dedf970 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a1df484c0 con 0x559a1dee07b0 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f0372164b80 10 client.0 _ll_register_callbacks cb 0x7f03480096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1deeb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1deeb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49150/0 (socket says 172.21.15.49:49150) 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035f7fe700 1 -- 172.21.15.49:0/2137780621 learned_addr learned my addr 172.21.15.49:0/2137780621 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035effd700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1deeb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035ffff700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1df47d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035f7fe700 1 -- 172.21.15.49:0/2137780621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 msgr2=0x559a1df47d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035f7fe700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1df47d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035f7fe700 1 -- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 msgr2=0x559a1deeb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.033+0000 7f035f7fe700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1deeb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f035f7fe700 1 -- 172.21.15.49:0/2137780621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0350017070 con 0x559a1dedf010 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f035effd700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1deeb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f035f7fe700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1deeb320 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03500117d0 tx=0x7f035002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2171af18d7ac64c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f033effd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f033effd700 1 -- 172.21.15.49:0/2137780621 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035002a9f0 con 0x559a1dedf010 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f033effd700 1 -- 172.21.15.49:0/2137780621 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f035002ab50 con 0x559a1dedf010 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f035809b6e0 con 0x559a1dedf010 2023-10-08T09:14:28.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f033effd700 1 -- 172.21.15.49:0/2137780621 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f035002ad10 con 0x559a1dedf010 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 10 client.15333 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559a1dc83200 con 0x559a1dedf010 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f033effd700 1 -- 172.21.15.49:0/2137780621 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f035809b6e0 con 0x559a1dedf010 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f033effd700 1 -- 172.21.15.49:0/2137780621 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0350030030 con 0x559a1dedf010 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 10 client.15333 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 20 client.15333 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1df4d640 0x559a1df49ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 4 client.15333 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.034+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 --> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559a1de2aae0 con 0x559a1df4d640 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f035effd700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1df4d640 0x559a1df49ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f033effd700 10 client.15333 ms_handle_connect on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f035effd700 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1df4d640 0x559a1df49ac0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f6b97ba23be5b3b0 server_cookie=50eed40a1b8bc281 in_seq=0 out_seq=0 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f033effd700 1 -- 172.21.15.49:0/2137780621 <== mds.4 v2:172.21.15.132:6832/3058040154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x559a1de2aae0 con 0x559a1df4d640 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f033effd700 10 client.15333 handle_command_reply: tid=0 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f0372164b80 1 client.15333 shutdown 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 msgr2=0x559a1deeb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.035+0000 7f0372164b80 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1deeb320 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03500117d0 tx=0x7f035002a6a0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 shutdown_connections 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559a1dee07b0 0x559a1df47d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x559a1df4d640 0x559a1df49ac0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559a1dedf010 0x559a1deeb320 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f033effd700 0 client.15333 ms_handle_reset on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 --2- 172.21.15.49:0/2137780621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559a1dedf970 0x559a1deeb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 >> 172.21.15.49:0/2137780621 conn(0x559a1de53210 msgr2=0x559a1dee7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 shutdown_connections 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 1 -- 172.21.15.49:0/2137780621 wait complete. 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0372164b80 20 client.15333 trim_cache size 0 max 16384 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 4, 2023-10-08T09:14:28.045 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24545, 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 55.622846144, 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 114.126942672 2023-10-08T09:14:28.046 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f034c0a1510 msgr2=0x7f034c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f034c0a1510 0x7f034c0a39d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f03601006f0 tx=0x7f0350009260 comp rx=0 tx=0).stop 2023-10-08T09:14:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 msgr2=0x7f036019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f036019db90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0358007b60 tx=0x7f0358008e40 comp rx=0 tx=0).stop 2023-10-08T09:14:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 shutdown_connections 2023-10-08T09:14:28.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f034c0a1510 0x7f034c0a39d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03600ff6a0 0x7f0360068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0360101230 0x7f036019db90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 --2- 172.21.15.49:0/2871626127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0360100890 0x7f036019b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 >> 172.21.15.49:0/2871626127 conn(0x7f03600fac10 msgr2=0x7f0360103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 shutdown_connections 2023-10-08T09:14:28.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.036+0000 7f0365f7b700 1 -- 172.21.15.49:0/2871626127 wait complete. 2023-10-08T09:14:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:27 smithi132 ceph-mon[103389]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:27 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2331722874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:28.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:27 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2823350584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:28.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:27 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2993221232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:28.050 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-10-08T09:14:28.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 -- 172.21.15.49:0/169008550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340ff1c0 msgr2=0x7f2f34108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 --2- 172.21.15.49:0/169008550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340ff1c0 0x7f2f34108d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00b0d0 tx=0x7f2f2c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 -- 172.21.15.49:0/169008550 shutdown_connections 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 --2- 172.21.15.49:0/169008550 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340ff1c0 0x7f2f34108d90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 --2- 172.21.15.49:0/169008550 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f34101db0 0x7f2f341021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 --2- 172.21.15.49:0/169008550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f340fe820 0x7f2f340fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 -- 172.21.15.49:0/169008550 >> 172.21.15.49:0/169008550 conn(0x7f2f340fa3c0 msgr2=0x7f2f340fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 -- 172.21.15.49:0/169008550 shutdown_connections 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.176+0000 7f2f39874700 1 -- 172.21.15.49:0/169008550 wait complete. 2023-10-08T09:14:28.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.177+0000 7f2f39874700 1 Processor -- start 2023-10-08T09:14:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.177+0000 7f2f39874700 1 -- start start 2023-10-08T09:14:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f39874700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340fe820 0x7f2f34196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f39874700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f340ff1c0 0x7f2f34196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f39874700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f34101db0 0x7f2f3419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f39874700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419e370 con 0x7f2f340fe820 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f39874700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419e4b0 con 0x7f2f34101db0 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f39874700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419e640 con 0x7f2f340ff1c0 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340fe820 0x7f2f34196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340fe820 0x7f2f34196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49164/0 (socket says 172.21.15.49:49164) 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2931862236 learned_addr learned my addr 172.21.15.49:0/2931862236 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f327fc700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f340ff1c0 0x7f2f34196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2931862236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f340ff1c0 msgr2=0x7f2f34196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f337fe700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f34101db0 0x7f2f3419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f340ff1c0 0x7f2f34196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2931862236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f34101db0 msgr2=0x7f2f3419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f34101db0 0x7f2f3419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2931862236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c009d00 con 0x7f2f340fe820 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340fe820 0x7f2f34196840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400d980 tx=0x7f2f2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.178+0000 7f2f38872700 1 -- 172.21.15.49:0/2931862236 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24004020 con 0x7f2f340fe820 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.179+0000 7f2f38872700 1 -- 172.21.15.49:0/2931862236 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f24009730 con 0x7f2f340fe820 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.179+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3419e920 con 0x7f2f340fe820 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.179+0000 7f2f38872700 1 -- 172.21.15.49:0/2931862236 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24003a80 con 0x7f2f340fe820 2023-10-08T09:14:28.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.179+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3419ee30 con 0x7f2f340fe820 2023-10-08T09:14:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.181+0000 7f2f38872700 1 -- 172.21.15.49:0/2931862236 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f2f240098a0 con 0x7f2f340fe820 2023-10-08T09:14:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.181+0000 7f2f38872700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.181+0000 7f2f327fc700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.181+0000 7f2f38872700 1 -- 172.21.15.49:0/2931862236 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f240cceb0 con 0x7f2f340fe820 2023-10-08T09:14:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f327fc700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f28005950 tx=0x7f2f2800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 Processor -- start 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 -- start start 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b010 0x55da6072b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6072c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072c7b0 0x55da6072cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da60693410 con 0x55da6072c7b0 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da60684bd0 con 0x55da6072b010 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da606824f0 con 0x55da6072b970 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f337fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072c7b0 0x55da6072cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f337fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072c7b0 0x55da6072cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49176/0 (socket says 172.21.15.49:49176) 2023-10-08T09:14:28.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.182+0000 7f2f337fe700 1 -- 172.21.15.49:0/2071396019 learned_addr learned my addr 172.21.15.49:0/2071396019 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f327fc700 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6072c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b010 0x55da6072b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f337fe700 1 -- 172.21.15.49:0/2071396019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 msgr2=0x55da6072c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f337fe700 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6072c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f337fe700 1 -- 172.21.15.49:0/2071396019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b010 msgr2=0x55da6072b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f337fe700 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b010 0x55da6072b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f337fe700 1 -- 172.21.15.49:0/2071396019 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da604a8d90 con 0x55da6072c7b0 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b010 0x55da6072b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f337fe700 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072c7b0 0x55da6072cc40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009c00 tx=0x7f2f2c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58405cab7207ed32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f19ffb700 1 -- 172.21.15.49:0/2071396019 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00e070 con 0x55da6072c7b0 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f19ffb700 1 -- 172.21.15.49:0/2071396019 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f2c0090d0 con 0x55da6072c7b0 2023-10-08T09:14:28.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.183+0000 7f2f19ffb700 1 -- 172.21.15.49:0/2071396019 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c005040 con 0x55da6072c7b0 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2071396019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072c7b0 msgr2=0x55da6072cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072c7b0 0x55da6072cc40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009c00 tx=0x7f2f2c009c30 comp rx=0 tx=0).stop 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2071396019 shutdown_connections 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072c7b0 0x55da6072cc40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b010 0x55da6072b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2071396019 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6072c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2071396019 >> 172.21.15.49:0/2071396019 conn(0x55da6069f210 msgr2=0x55da60725db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2071396019 shutdown_connections 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2071396019 wait complete. 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 -- start start 2023-10-08T09:14:28.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 Processor -- start 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.184+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60734dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b970 0x55da60735300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072c7b0 0x55da6073c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6067dde0 con 0x55da6072b010 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6061a6c0 con 0x55da6072b970 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da6061a360 con 0x55da6072c7b0 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f45a5db80 10 client.0 _ll_register_callbacks cb 0x55da604777b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60734dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60734dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49178/0 (socket says 172.21.15.49:49178) 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1662047016 learned_addr learned my addr 172.21.15.49:0/1662047016 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f337fe700 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072c7b0 0x55da6073c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f327fc700 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b970 0x55da60735300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072c7b0 msgr2=0x55da6073c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072c7b0 0x55da6073c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1662047016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b970 msgr2=0x55da60735300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b970 0x55da60735300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1662047016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f34109550 con 0x55da6072b010 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60734dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f24099a60 tx=0x7f2f24099d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc1a1f3e5d90bae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1662047016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2409d030 con 0x55da6072b010 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1662047016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f2409deb0 con 0x55da6072b010 2023-10-08T09:14:28.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da60645e60 con 0x55da6072b010 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.185+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1662047016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f240a20f0 con 0x55da6072b010 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 10 client.15345 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55da60491c10 con 0x55da6072b010 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1662047016 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da60645e60 con 0x55da6072b010 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1662047016 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f24003be0 con 0x55da6072b010 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 10 client.15345 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 10 client.15345 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 20 client.15345 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da607408a0 0x55da6073f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 4 client.15345 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:28.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 --> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da60676dc0 con 0x55da607408a0 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.186+0000 7f2f327fc700 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da607408a0 0x55da6073f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.187+0000 7f2f327fc700 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da607408a0 0x55da6073f3b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb48d845e9be2a20 server_cookie=9e0e9f7d2bd9c073 in_seq=0 out_seq=0 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.187+0000 7f2f12ffd700 10 client.15345 ms_handle_connect on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.187+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1662047016 <== mds.3 v2:172.21.15.184:6832/3616311663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55da60676dc0 con 0x55da607408a0 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.187+0000 7f2f12ffd700 10 client.15345 handle_command_reply: tid=0 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.187+0000 7f2f45a5db80 1 client.15345 shutdown 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.187+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 msgr2=0x55da60734dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.187+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60734dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f24099a60 tx=0x7f2f24099d60 comp rx=0 tx=0).stop 2023-10-08T09:14:28.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 shutdown_connections 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072c7b0 0x55da6073c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da607408a0 0x55da6073f3b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60734dc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1662047016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072b970 0x55da60735300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f12ffd700 0 client.15345 ms_handle_reset on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 >> 172.21.15.49:0/1662047016 conn(0x55da6069f210 msgr2=0x55da60725480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 shutdown_connections 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1662047016 wait complete. 2023-10-08T09:14:28.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.188+0000 7f2f45a5db80 20 client.15345 trim_cache size 0 max 16384 2023-10-08T09:14:28.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 Processor -- start 2023-10-08T09:14:28.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 -- start start 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60732d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6073c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da6073f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da604fe8d0 con 0x55da6072b010 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da60720520 con 0x55da6072c7b0 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da60676dc0 con 0x55da6072b970 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.205+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60732d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60732d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49186/0 (socket says 172.21.15.49:49186) 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2775585337 learned_addr learned my addr 172.21.15.49:0/2775585337 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f327fc700 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6073c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f337fe700 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da6073f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2775585337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 msgr2=0x55da6073c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6073c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2775585337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 msgr2=0x55da6073f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da6073f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 -- 172.21.15.49:0/2775585337 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da604cf200 con 0x55da6072b010 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f337fe700 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da6073f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60732d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009520 tx=0x7f2f24099b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b37a1aa75cfaa6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f1a7fc700 1 -- 172.21.15.49:0/2775585337 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2409d070 con 0x55da6072b010 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f1a7fc700 1 -- 172.21.15.49:0/2775585337 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f2409deb0 con 0x55da6072b010 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.206+0000 7f2f1a7fc700 1 -- 172.21.15.49:0/2775585337 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24099250 con 0x55da6072b010 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2775585337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 msgr2=0x55da60732d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60732d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009520 tx=0x7f2f24099b30 comp rx=0 tx=0).stop 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2775585337 shutdown_connections 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da6073f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da60732d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/2775585337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da6073c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2775585337 >> 172.21.15.49:0/2775585337 conn(0x55da6069f210 msgr2=0x55da607342d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2775585337 shutdown_connections 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 -- 172.21.15.49:0/2775585337 wait complete. 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 -- start start 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.207+0000 7f2f45a5db80 1 Processor -- start 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da607391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da60739710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f45a5db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da60794290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da60739d30 con 0x55da6072b010 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da607947d0 con 0x55da6072c7b0 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f45a5db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da60794910 con 0x55da6072b970 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da607391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f45a5db80 10 client.0 _ll_register_callbacks cb 0x7f2f24099770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da607391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49192/0 (socket says 172.21.15.49:49192) 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1190688688 learned_addr learned my addr 172.21.15.49:0/1190688688 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f327fc700 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da60739710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f337fe700 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da60794290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 msgr2=0x55da60739710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da60739710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1190688688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 msgr2=0x55da60794290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da60794290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 -- 172.21.15.49:0/1190688688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da60491c10 con 0x55da6072b010 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f32ffd700 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da607391d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f24099cf0 tx=0x7f2f2409a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789dd4cfa187b818 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.208+0000 7f2f12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1190688688 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2409d070 con 0x55da6072b010 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1190688688 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f240add40 con 0x55da6072b010 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f2c0090d0 con 0x55da6072b010 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1190688688 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f24009cd0 con 0x55da6072b010 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 10 client.15351 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55da604a7260 con 0x55da6072b010 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1190688688 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f2c0090d0 con 0x55da6072b010 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1190688688 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2f24095be0 con 0x55da6072b010 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 10 client.15351 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 10 client.15351 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 20 client.15351 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da60799760 0x55da60795c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 4 client.15351 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 --> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55da60676ae0 con 0x55da60799760 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.209+0000 7f2f327fc700 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da60799760 0x55da60795c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f12ffd700 10 client.15351 ms_handle_connect on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f327fc700 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da60799760 0x55da60795c80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bff8f79664ac671c server_cookie=6a34a64f24543505 in_seq=0 out_seq=0 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f12ffd700 1 -- 172.21.15.49:0/1190688688 <== mds.3 v2:172.21.15.184:6832/3616311663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55da60676ae0 con 0x55da60799760 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f12ffd700 10 client.15351 handle_command_reply: tid=0 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 client.15351 shutdown 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 msgr2=0x55da607391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da607391d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f24099cf0 tx=0x7f2f2409a360 comp rx=0 tx=0).stop 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 shutdown_connections 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55da60799760 0x55da60795c80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55da6072c7b0 0x55da60794290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f12ffd700 0 client.15351 ms_handle_reset on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55da6072b010 0x55da607391d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 --2- 172.21.15.49:0/1190688688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55da6072b970 0x55da60739710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 >> 172.21.15.49:0/1190688688 conn(0x55da6069f210 msgr2=0x55da607329d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 shutdown_connections 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 1 -- 172.21.15.49:0/1190688688 wait complete. 2023-10-08T09:14:28.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.210+0000 7f2f45a5db80 20 client.15351 trim_cache size 0 max 16384 2023-10-08T09:14:28.224 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 3, 2023-10-08T09:14:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24553, 2023-10-08T09:14:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 56.801723078999999, 2023-10-08T09:14:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2023-10-08T09:14:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 116.50724302099999 2023-10-08T09:14:28.225 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:28.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f1c0a1510 msgr2=0x7f2f1c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f28005950 tx=0x7f2f2800a300 comp rx=0 tx=0).stop 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340fe820 msgr2=0x7f2f34196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340fe820 0x7f2f34196840 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f2400d980 tx=0x7f2f2400dc90 comp rx=0 tx=0).stop 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 shutdown_connections 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2f1c0a1510 0x7f2f1c0a39d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f34101db0 0x7f2f3419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2f340fe820 0x7f2f34196840 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 --2- 172.21.15.49:0/2931862236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2f340ff1c0 0x7f2f34196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 >> 172.21.15.49:0/2931862236 conn(0x7f2f340fa3c0 msgr2=0x7f2f34104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 shutdown_connections 2023-10-08T09:14:28.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.211+0000 7f2f39874700 1 -- 172.21.15.49:0/2931862236 wait complete. 2023-10-08T09:14:28.228 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-08T09:14:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 -- 172.21.15.49:0/1906783609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c1008c0 msgr2=0x7fc11c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 --2- 172.21.15.49:0/1906783609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c1008c0 0x7fc11c100d20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0099c0 tx=0x7fc10c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 -- 172.21.15.49:0/1906783609 shutdown_connections 2023-10-08T09:14:28.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 --2- 172.21.15.49:0/1906783609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc11c101260 0x7fc11c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 --2- 172.21.15.49:0/1906783609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c0ff6d0 0x7fc11c0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 --2- 172.21.15.49:0/1906783609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c1008c0 0x7fc11c100d20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 -- 172.21.15.49:0/1906783609 >> 172.21.15.49:0/1906783609 conn(0x7fc11c0fac40 msgr2=0x7fc11c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 -- 172.21.15.49:0/1906783609 shutdown_connections 2023-10-08T09:14:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 -- 172.21.15.49:0/1906783609 wait complete. 2023-10-08T09:14:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 Processor -- start 2023-10-08T09:14:28.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.355+0000 7fc123b75700 1 -- start start 2023-10-08T09:14:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc123b75700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c0ff6d0 0x7fc11c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc123b75700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc11c1008c0 0x7fc11c19b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc123b75700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c101260 0x7fc11c19daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc123b75700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19e240 con 0x7fc11c101260 2023-10-08T09:14:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc123b75700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19e3b0 con 0x7fc11c0ff6d0 2023-10-08T09:14:28.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc123b75700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c19e570 con 0x7fc11c1008c0 2023-10-08T09:14:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c101260 0x7fc11c19daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c101260 0x7fc11c19daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49210/0 (socket says 172.21.15.49:49210) 2023-10-08T09:14:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 -- 172.21.15.49:0/1823849791 learned_addr learned my addr 172.21.15.49:0/1823849791 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc121911700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c0ff6d0 0x7fc11c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc121110700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc11c1008c0 0x7fc11c19b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 -- 172.21.15.49:0/1823849791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc11c1008c0 msgr2=0x7fc11c19b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc11c1008c0 0x7fc11c19b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 -- 172.21.15.49:0/1823849791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c0ff6d0 msgr2=0x7fc11c0694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c0ff6d0 0x7fc11c0694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 -- 172.21.15.49:0/1823849791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0095f0 con 0x7fc11c101260 2023-10-08T09:14:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc122112700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c101260 0x7fc11c19daa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc11400ef30 tx=0x7fc11400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc112ffd700 1 -- 172.21.15.49:0/1823849791 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114018070 con 0x7fc11c101260 2023-10-08T09:14:28.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.356+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c19e7f0 con 0x7fc11c101260 2023-10-08T09:14:28.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.357+0000 7fc112ffd700 1 -- 172.21.15.49:0/1823849791 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc11400f040 con 0x7fc11c101260 2023-10-08T09:14:28.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.357+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c19ecd0 con 0x7fc11c101260 2023-10-08T09:14:28.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.357+0000 7fc112ffd700 1 -- 172.21.15.49:0/1823849791 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400cd00 con 0x7fc11c101260 2023-10-08T09:14:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.359+0000 7fc112ffd700 1 -- 172.21.15.49:0/1823849791 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc114003a00 con 0x7fc11c101260 2023-10-08T09:14:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.359+0000 7fc112ffd700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1590 0x7fc1080a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.359+0000 7fc121911700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1590 0x7fc1080a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.359+0000 7fc112ffd700 1 -- 172.21.15.49:0/1823849791 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc1140ccc60 con 0x7fc11c101260 2023-10-08T09:14:28.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc121911700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1590 0x7fc1080a3a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc11c100720 tx=0x7fc118009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 Processor -- start 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 -- start start 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a715fa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa970 0x558a715fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a715fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a71562410 con 0x558a715fa970 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a71553bd0 con 0x558a715fb7b0 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a715514f0 con 0x558a715fa010 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc121110700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa970 0x558a715fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc121110700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa970 0x558a715fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49216/0 (socket says 172.21.15.49:49216) 2023-10-08T09:14:28.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.360+0000 7fc121110700 1 -- 172.21.15.49:0/455476260 learned_addr learned my addr 172.21.15.49:0/455476260 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc122112700 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a715fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc121911700 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a715fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc121110700 1 -- 172.21.15.49:0/455476260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 msgr2=0x558a715fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc121110700 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a715fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc121110700 1 -- 172.21.15.49:0/455476260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 msgr2=0x558a715fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc121110700 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a715fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc121110700 1 -- 172.21.15.49:0/455476260 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a71377d90 con 0x558a715fa970 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc121110700 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa970 0x558a715fb220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc10c000c00 tx=0x7fc10c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56e7267367b73d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc107fff700 1 -- 172.21.15.49:0/455476260 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c00ba90 con 0x558a715fa970 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc107fff700 1 -- 172.21.15.49:0/455476260 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10c01e040 con 0x558a715fa970 2023-10-08T09:14:28.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.361+0000 7fc107fff700 1 -- 172.21.15.49:0/455476260 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c003bf0 con 0x558a715fa970 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/455476260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa970 msgr2=0x558a715fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa970 0x558a715fb220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc10c000c00 tx=0x7fc10c005bc0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/455476260 shutdown_connections 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a715fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a715fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/455476260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa970 0x558a715fb220 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/455476260 >> 172.21.15.49:0/455476260 conn(0x558a7156e210 msgr2=0x558a715f4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/455476260 shutdown_connections 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/455476260 wait complete. 2023-10-08T09:14:28.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 -- start start 2023-10-08T09:14:28.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.362+0000 7fc12fd5eb80 1 Processor -- start 2023-10-08T09:14:28.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a71603d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fa970 0x558a71604290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fb7b0 0x558a7160b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7154cde0 con 0x558a715fb7b0 2023-10-08T09:14:28.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a714e96c0 con 0x558a715fa970 2023-10-08T09:14:28.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a714e9360 con 0x558a715fa010 2023-10-08T09:14:28.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc12fd5eb80 10 client.0 _ll_register_callbacks cb 0x558a713467b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fb7b0 0x558a7160b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fb7b0 0x558a7160b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49224/0 (socket says 172.21.15.49:49224) 2023-10-08T09:14:28.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 -- 172.21.15.49:0/3507776042 learned_addr learned my addr 172.21.15.49:0/3507776042 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc121911700 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a71603d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc121110700 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fa970 0x558a71604290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 -- 172.21.15.49:0/3507776042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 msgr2=0x558a71603d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a71603d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 -- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fa970 msgr2=0x558a71604290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fa970 0x558a71604290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 -- 172.21.15.49:0/3507776042 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c108460 con 0x558a715fb7b0 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc122112700 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fb7b0 0x558a7160b2d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc114096610 tx=0x7fc1140997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c0b5cedac09693 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.363+0000 7fc1057fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc1057fa700 1 -- 172.21.15.49:0/3507776042 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11401b3a0 con 0x558a715fb7b0 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc1057fa700 1 -- 172.21.15.49:0/3507776042 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc11409a2a0 con 0x558a715fb7b0 2023-10-08T09:14:28.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a71514e60 con 0x558a715fb7b0 2023-10-08T09:14:28.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc1057fa700 1 -- 172.21.15.49:0/3507776042 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc114099ba0 con 0x558a715fb7b0 2023-10-08T09:14:28.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 10 client.15363 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558a71360c10 con 0x558a715fb7b0 2023-10-08T09:14:28.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc1057fa700 1 -- 172.21.15.49:0/3507776042 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a71514e60 con 0x558a715fb7b0 2023-10-08T09:14:28.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc1057fa700 1 -- 172.21.15.49:0/3507776042 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc114099100 con 0x558a715fb7b0 2023-10-08T09:14:28.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 10 client.15363 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T09:14:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 20 client.15363 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a7160f9a0 0x558a7160e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 4 client.15363 mds_command: new command op to 24608 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.364+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 --> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a71545dc0 con 0x558a7160f9a0 2023-10-08T09:14:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.365+0000 7fc121911700 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a7160f9a0 0x558a7160e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.365+0000 7fc121911700 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a7160f9a0 0x558a7160e4c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38e8cdfe94b584bd server_cookie=f3f477f61e115b7b in_seq=0 out_seq=0 2023-10-08T09:14:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.365+0000 7fc1057fa700 10 client.15363 ms_handle_connect on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:28.378 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc1057fa700 1 -- 172.21.15.49:0/3507776042 <== mds.2 v2:172.21.15.132:6836/2064856316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558a71545dc0 con 0x558a7160f9a0 2023-10-08T09:14:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc1057fa700 10 client.15363 handle_command_reply: tid=0 2023-10-08T09:14:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 client.15363 shutdown 2023-10-08T09:14:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fb7b0 msgr2=0x558a7160b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fb7b0 0x558a7160b2d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc114096610 tx=0x7fc1140997e0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 shutdown_connections 2023-10-08T09:14:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa010 0x558a71603d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a7160f9a0 0x558a7160e4c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fb7b0 0x558a7160b2d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/3507776042 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fa970 0x558a71604290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc1057fa700 0 client.15363 ms_handle_reset on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 >> 172.21.15.49:0/3507776042 conn(0x558a7156e210 msgr2=0x558a715f4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 shutdown_connections 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/3507776042 wait complete. 2023-10-08T09:14:28.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.366+0000 7fc12fd5eb80 20 client.15363 trim_cache size 0 max 16384 2023-10-08T09:14:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 Processor -- start 2023-10-08T09:14:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 -- start start 2023-10-08T09:14:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a715fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a7160b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a7160e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a713cd8d0 con 0x558a715fa010 2023-10-08T09:14:28.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7154e070 con 0x558a715fb7b0 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a71548ed0 con 0x558a715fa970 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a715fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a715fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49228/0 (socket says 172.21.15.49:49228) 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 -- 172.21.15.49:0/197074287 learned_addr learned my addr 172.21.15.49:0/197074287 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121110700 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a7160b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 -- 172.21.15.49:0/197074287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 msgr2=0x558a7160b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a7160b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 -- 172.21.15.49:0/197074287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 msgr2=0x558a7160e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc122112700 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a7160e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a7160e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 -- 172.21.15.49:0/197074287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a7139e200 con 0x558a715fa010 2023-10-08T09:14:28.391 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121110700 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a7160b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.384+0000 7fc121911700 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a715fbb90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x558a71610930 tx=0x7fc118014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76a1e53b2353e25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc110d82700 1 -- 172.21.15.49:0/197074287 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11802d070 con 0x558a715fa010 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc110d82700 1 -- 172.21.15.49:0/197074287 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1180044e0 con 0x558a715fa010 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc110d82700 1 -- 172.21.15.49:0/197074287 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118023040 con 0x558a715fa010 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/197074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 msgr2=0x558a715fbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a715fbb90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x558a71610930 tx=0x7fc118014000 comp rx=0 tx=0).stop 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/197074287 shutdown_connections 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a7160e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a715fbb90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/197074287 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a7160b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/197074287 >> 172.21.15.49:0/197074287 conn(0x558a7156e210 msgr2=0x558a715fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/197074287 shutdown_connections 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/197074287 wait complete. 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.385+0000 7fc12fd5eb80 1 -- start start 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 1 Processor -- start 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a71606700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a71606c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a71609ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7160a010 con 0x558a715fa010 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7160a180 con 0x558a715fb7b0 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a7160a2f0 con 0x558a715fa970 2023-10-08T09:14:28.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc12fd5eb80 10 client.0 _ll_register_callbacks cb 0x7fc114099a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a71606700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a71606700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49234/0 (socket says 172.21.15.49:49234) 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 -- 172.21.15.49:0/1221785547 learned_addr learned my addr 172.21.15.49:0/1221785547 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc122112700 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a71609ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121110700 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a71606c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 -- 172.21.15.49:0/1221785547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 msgr2=0x558a71606c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a71606c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 -- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 msgr2=0x558a71609ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a71609ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.386+0000 7fc121911700 1 -- 172.21.15.49:0/1221785547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc118017070 con 0x558a715fa010 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc121911700 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a71606700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1180117d0 tx=0x7fc118003750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d811aacf7f9931b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc1057fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc1057fa700 1 -- 172.21.15.49:0/1221785547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11802d040 con 0x558a715fa010 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc1057fa700 1 -- 172.21.15.49:0/1221785547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc118004160 con 0x558a715fa010 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc10c01e040 con 0x558a715fa010 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc1057fa700 1 -- 172.21.15.49:0/1221785547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc118023040 con 0x558a715fa010 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 10 client.15369 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558a71376260 con 0x558a715fa010 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc1057fa700 1 -- 172.21.15.49:0/1221785547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10c01e040 con 0x558a715fa010 2023-10-08T09:14:28.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc1057fa700 1 -- 172.21.15.49:0/1221785547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc11803a070 con 0x558a715fa010 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 10 client.15369 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 10 client.15369 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 20 client.15369 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a71669130 0x558a716655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 4 client.15369 mds_command: new command op to 24608 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.387+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 --> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558a71545ae0 con 0x558a71669130 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc121110700 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a71669130 0x558a716655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc1057fa700 10 client.15369 ms_handle_connect on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:28.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc121110700 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a71669130 0x558a716655b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=405bbda58c91caf4 server_cookie=d7f49918970f3896 in_seq=0 out_seq=0 2023-10-08T09:14:28.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc1057fa700 1 -- 172.21.15.49:0/1221785547 <== mds.2 v2:172.21.15.132:6836/2064856316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x558a71545ae0 con 0x558a71669130 2023-10-08T09:14:28.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc1057fa700 10 client.15369 handle_command_reply: tid=0 2023-10-08T09:14:28.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc12fd5eb80 1 client.15369 shutdown 2023-10-08T09:14:28.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 msgr2=0x558a71606700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a71606700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc1180117d0 tx=0x7fc118003750 comp rx=0 tx=0).stop 2023-10-08T09:14:28.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 shutdown_connections 2023-10-08T09:14:28.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.388+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x558a71669130 0x558a716655b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558a715fb7b0 0x558a71609ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc1057fa700 0 client.15369 ms_handle_reset on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:28.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558a715fa010 0x558a71606700 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc12fd5eb80 1 --2- 172.21.15.49:0/1221785547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558a715fa970 0x558a71606c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 >> 172.21.15.49:0/1221785547 conn(0x558a7156e210 msgr2=0x558a7160e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 shutdown_connections 2023-10-08T09:14:28.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc12fd5eb80 1 -- 172.21.15.49:0/1221785547 wait complete. 2023-10-08T09:14:28.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc12fd5eb80 20 client.15369 trim_cache size 0 max 16384 2023-10-08T09:14:28.400 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:28.400 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:28.400 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2023-10-08T09:14:28.400 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24608, 2023-10-08T09:14:28.400 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:28.400 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:28.401 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:28.401 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 57.931345505000003, 2023-10-08T09:14:28.401 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:28.401 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2023-10-08T09:14:28.401 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:28.401 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 101.36928988 2023-10-08T09:14:28.401 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1590 msgr2=0x7fc1080a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1590 0x7fc1080a3a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc11c100720 tx=0x7fc118009260 comp rx=0 tx=0).stop 2023-10-08T09:14:28.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c101260 msgr2=0x7fc11c19daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c101260 0x7fc11c19daa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc11400ef30 tx=0x7fc11400c3a0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 shutdown_connections 2023-10-08T09:14:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1590 0x7fc1080a3a50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc11c101260 0x7fc11c19daa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c0ff6d0 0x7fc11c0694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 --2- 172.21.15.49:0/1823849791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc11c1008c0 0x7fc11c19b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 >> 172.21.15.49:0/1823849791 conn(0x7fc11c0fac40 msgr2=0x7fc11c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 shutdown_connections 2023-10-08T09:14:28.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.389+0000 7fc123b75700 1 -- 172.21.15.49:0/1823849791 wait complete. 2023-10-08T09:14:28.405 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-08T09:14:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 -- 172.21.15.49:0/1256628425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 msgr2=0x7fa1fc0fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 --2- 172.21.15.49:0/1256628425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc0fec60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80099c0 tx=0x7fa1f8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 -- 172.21.15.49:0/1256628425 shutdown_connections 2023-10-08T09:14:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 --2- 172.21.15.49:0/1256628425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1fc101d90 0x7fa1fc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 --2- 172.21.15.49:0/1256628425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1fc0ff1a0 0x7fa1fc108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 --2- 172.21.15.49:0/1256628425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc0fec60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 -- 172.21.15.49:0/1256628425 >> 172.21.15.49:0/1256628425 conn(0x7fa1fc0fa3a0 msgr2=0x7fa1fc0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 -- 172.21.15.49:0/1256628425 shutdown_connections 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.532+0000 7fa203510700 1 -- 172.21.15.49:0/1256628425 wait complete. 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 Processor -- start 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 -- start start 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1fc0ff1a0 0x7fa1fc196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1fc101d90 0x7fa1fc19dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc19e260 con 0x7fa1fc0fe800 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc19e3a0 con 0x7fa1fc0ff1a0 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa203510700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc19e530 con 0x7fa1fc101d90 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49248/0 (socket says 172.21.15.49:49248) 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 -- 172.21.15.49:0/4256557319 learned_addr learned my addr 172.21.15.49:0/4256557319 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa200aab700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1fc0ff1a0 0x7fa1fc196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa201aad700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1fc101d90 0x7fa1fc19dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 -- 172.21.15.49:0/4256557319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1fc101d90 msgr2=0x7fa1fc19dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1fc101d90 0x7fa1fc19dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 -- 172.21.15.49:0/4256557319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1fc0ff1a0 msgr2=0x7fa1fc196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1fc0ff1a0 0x7fa1fc196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 -- 172.21.15.49:0/4256557319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f80095f0 con 0x7fa1fc0fe800 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.533+0000 7fa2012ac700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc196730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e950 tx=0x7fa1ec00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.534+0000 7fa1f27fc700 1 -- 172.21.15.49:0/4256557319 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec009770 con 0x7fa1fc0fe800 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.534+0000 7fa1f27fc700 1 -- 172.21.15.49:0/4256557319 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1ec00c7e0 con 0x7fa1fc0fe800 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.534+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc19e810 con 0x7fa1fc0fe800 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.534+0000 7fa1f27fc700 1 -- 172.21.15.49:0/4256557319 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec003cf0 con 0x7fa1fc0fe800 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.534+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc19ecd0 con 0x7fa1fc0fe800 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.536+0000 7fa1f27fc700 1 -- 172.21.15.49:0/4256557319 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa1ec00c950 con 0x7fa1fc0fe800 2023-10-08T09:14:28.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.536+0000 7fa1f27fc700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1e80a1510 0x7fa1e80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.537+0000 7fa1f27fc700 1 -- 172.21.15.49:0/4256557319 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1ec014070 con 0x7fa1fc0fe800 2023-10-08T09:14:28.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.537+0000 7fa200aab700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1e80a1510 0x7fa1e80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.537+0000 7fa200aab700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1e80a1510 0x7fa1e80a39d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009990 tx=0x7fa1f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.537+0000 7fa20f6f9b80 1 Processor -- start 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.537+0000 7fa20f6f9b80 1 -- start start 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.537+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.537+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951ed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951edc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558495154410 con 0x5584951ed7b0 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558495145bd0 con 0x5584951ec010 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584951434f0 con 0x5584951ec970 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951edc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49260/0 (socket says 172.21.15.49:49260) 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 -- 172.21.15.49:0/2079926128 learned_addr learned my addr 172.21.15.49:0/2079926128 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa2012ac700 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa200aab700 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951ed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 -- 172.21.15.49:0/2079926128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 msgr2=0x5584951ed220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951ed220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 -- 172.21.15.49:0/2079926128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 msgr2=0x5584951ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 -- 172.21.15.49:0/2079926128 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558494f69d90 con 0x5584951ed7b0 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa201aad700 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951edc40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4011680 tx=0x7fa1f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a04b8f0f02bf85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa1e77fe700 1 -- 172.21.15.49:0/2079926128 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400e040 con 0x5584951ed7b0 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa1e77fe700 1 -- 172.21.15.49:0/2079926128 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f400f040 con 0x5584951ed7b0 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.538+0000 7fa1e77fe700 1 -- 172.21.15.49:0/2079926128 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4004860 con 0x5584951ed7b0 2023-10-08T09:14:28.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2079926128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 msgr2=0x5584951edc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951edc40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4011680 tx=0x7fa1f4009920 comp rx=0 tx=0).stop 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2079926128 shutdown_connections 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951edc40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2079926128 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951ed220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2079926128 >> 172.21.15.49:0/2079926128 conn(0x558495160210 msgr2=0x5584951e6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2079926128 shutdown_connections 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2079926128 wait complete. 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.539+0000 7fa20f6f9b80 1 -- start start 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 1 Processor -- start 2023-10-08T09:14:28.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849513ede0 con 0x5584951ed7b0 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584950db6c0 con 0x5584951ec010 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584950db360 con 0x5584951ec970 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa20f6f9b80 10 client.0 _ll_register_callbacks cb 0x558494f387b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa201aad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa201aad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49266/0 (socket says 172.21.15.49:49266) 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa201aad700 1 -- 172.21.15.49:0/1621839979 learned_addr learned my addr 172.21.15.49:0/1621839979 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.540+0000 7fa2012ac700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa200aab700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa201aad700 1 -- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 msgr2=0x5584951f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa201aad700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa201aad700 1 -- 172.21.15.49:0/1621839979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 msgr2=0x5584951f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa201aad700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa201aad700 1 -- 172.21.15.49:0/1621839979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f4011330 con 0x5584951ed7b0 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa200aab700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa201aad700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951fd2d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009e40 tx=0x7fa1f4003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41d68897d0149c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa1e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/1621839979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400e030 con 0x5584951ed7b0 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/1621839979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1f40093a0 con 0x5584951ed7b0 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558495106e60 con 0x5584951ed7b0 2023-10-08T09:14:28.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/1621839979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1f4004ce0 con 0x5584951ed7b0 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa20f6f9b80 10 client.15381 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558494f52c10 con 0x5584951ed7b0 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.541+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/1621839979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558495106e60 con 0x5584951ed7b0 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/1621839979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa1f4024020 con 0x5584951ed7b0 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa20f6f9b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa20f6f9b80 10 client.15381 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa20f6f9b80 20 client.15381 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x5584952018a0 0x558495200410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa20f6f9b80 4 client.15381 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:28.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 --> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558495137dc0 con 0x5584952018a0 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa2012ac700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x5584952018a0 0x558495200410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa1e4ff9700 10 client.15381 ms_handle_connect on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.542+0000 7fa2012ac700 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x5584952018a0 0x558495200410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d204135ccb1997cc server_cookie=a13618e5486313bf in_seq=0 out_seq=0 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/1621839979 <== mds.1 v2:172.21.15.184:6838/2795080121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558495137dc0 con 0x5584952018a0 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa1e4ff9700 10 client.15381 handle_command_reply: tid=0 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 client.15381 shutdown 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 msgr2=0x5584951fd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951fd2d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009e40 tx=0x7fa1f4003d60 comp rx=0 tx=0).stop 2023-10-08T09:14:28.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 shutdown_connections 2023-10-08T09:14:28.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x5584952018a0 0x558495200410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ed7b0 0x5584951fd2d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa1e4ff9700 0 client.15381 ms_handle_reset on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:28.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ec010 0x5584951f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/1621839979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 >> 172.21.15.49:0/1621839979 conn(0x558495160210 msgr2=0x5584951e6480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 shutdown_connections 2023-10-08T09:14:28.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.543+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/1621839979 wait complete. 2023-10-08T09:14:28.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.544+0000 7fa20f6f9b80 20 client.15381 trim_cache size 0 max 16384 2023-10-08T09:14:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.561+0000 7fa20f6f9b80 1 Processor -- start 2023-10-08T09:14:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.561+0000 7fa20f6f9b80 1 -- start start 2023-10-08T09:14:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.561+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec010 0x5584951f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.561+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec970 0x5584951f5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.561+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558494fbf8d0 con 0x5584951ec970 2023-10-08T09:14:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584951e3a90 con 0x5584951ed7b0 2023-10-08T09:14:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584951e7ab0 con 0x5584951ec010 2023-10-08T09:14:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec970 0x5584951f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec970 0x5584951f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49272/0 (socket says 172.21.15.49:49272) 2023-10-08T09:14:28.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa2012ac700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec010 0x5584951f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 -- 172.21.15.49:0/2845475326 learned_addr learned my addr 172.21.15.49:0/2845475326 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa201aad700 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 -- 172.21.15.49:0/2845475326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec010 msgr2=0x5584951f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec010 0x5584951f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 -- 172.21.15.49:0/2845475326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 msgr2=0x5584951fd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 -- 172.21.15.49:0/2845475326 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558494f90200 con 0x5584951ec970 2023-10-08T09:14:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa2012ac700 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec010 0x5584951f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa200aab700 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec970 0x5584951f5450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80058e0 tx=0x7fa1f8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1aef4e484071cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa1e7fff700 1 -- 172.21.15.49:0/2845475326 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80039d0 con 0x5584951ec970 2023-10-08T09:14:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa1e7fff700 1 -- 172.21.15.49:0/2845475326 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f802a070 con 0x5584951ec970 2023-10-08T09:14:28.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.562+0000 7fa1e7fff700 1 -- 172.21.15.49:0/2845475326 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800fb50 con 0x5584951ec970 2023-10-08T09:14:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2845475326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec970 msgr2=0x5584951f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec970 0x5584951f5450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80058e0 tx=0x7fa1f8021000 comp rx=0 tx=0).stop 2023-10-08T09:14:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2845475326 shutdown_connections 2023-10-08T09:14:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec010 0x5584951f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/2845475326 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec970 0x5584951f5450 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2845475326 >> 172.21.15.49:0/2845475326 conn(0x558495160210 msgr2=0x5584951f3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2845475326 shutdown_connections 2023-10-08T09:14:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/2845475326 wait complete. 2023-10-08T09:14:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 -- start start 2023-10-08T09:14:28.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.563+0000 7fa20f6f9b80 1 Processor -- start 2023-10-08T09:14:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec010 0x5584951f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa20f6f9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584951fbdd0 con 0x5584951ec010 2023-10-08T09:14:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584951fbf40 con 0x5584951ed7b0 2023-10-08T09:14:28.577 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa20f6f9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584951fc100 con 0x5584951ec970 2023-10-08T09:14:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa20f6f9b80 10 client.0 _ll_register_callbacks cb 0x7fa1f4011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec010 0x5584951f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec010 0x5584951f8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49276/0 (socket says 172.21.15.49:49276) 2023-10-08T09:14:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 -- 172.21.15.49:0/4223274154 learned_addr learned my addr 172.21.15.49:0/4223274154 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa201aad700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa200aab700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 -- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 msgr2=0x5584951f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 -- 172.21.15.49:0/4223274154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 msgr2=0x5584951fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 -- 172.21.15.49:0/4223274154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc19ecd0 con 0x5584951ec010 2023-10-08T09:14:28.579 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa200aab700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa201aad700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa2012ac700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec010 0x5584951f8520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec096ca0 tx=0x7fa1ec09a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b593097ebb1282 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.564+0000 7fa1e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/4223274154 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0a25c0 con 0x5584951ec010 2023-10-08T09:14:28.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/4223274154 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1ec0b2070 con 0x5584951ec010 2023-10-08T09:14:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1f400f040 con 0x5584951ec010 2023-10-08T09:14:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/4223274154 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1ec0985d0 con 0x5584951ec010 2023-10-08T09:14:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 10 client.15387 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa1fc109530 con 0x5584951ec010 2023-10-08T09:14:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/4223274154 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f400f040 con 0x5584951ec010 2023-10-08T09:14:28.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/4223274154 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa1ec09a490 con 0x5584951ec010 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 10 client.15387 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 20 client.15387 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55849525aea0 0x5584952572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 4 client.15387 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.565+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 --> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558495137ae0 con 0x55849525aea0 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa200aab700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55849525aea0 0x5584952572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa1e4ff9700 10 client.15387 ms_handle_connect on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa200aab700 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55849525aea0 0x5584952572f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5ff042bfadd76c6 server_cookie=c7c3905cd16a6da2 in_seq=0 out_seq=0 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa1e4ff9700 1 -- 172.21.15.49:0/4223274154 <== mds.1 v2:172.21.15.184:6838/2795080121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558495137ae0 con 0x55849525aea0 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa1e4ff9700 10 client.15387 handle_command_reply: tid=0 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 client.15387 shutdown 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec010 msgr2=0x5584951f8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec010 0x5584951f8520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec096ca0 tx=0x7fa1ec09a2e0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 shutdown_connections 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55849525aea0 0x5584952572f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5584951ed7b0 0x5584951fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5584951ec010 0x5584951f8520 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa1e4ff9700 0 client.15387 ms_handle_reset on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 --2- 172.21.15.49:0/4223274154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5584951ec970 0x5584951f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.566+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 >> 172.21.15.49:0/4223274154 conn(0x558495160210 msgr2=0x5584951f4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 shutdown_connections 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa20f6f9b80 1 -- 172.21.15.49:0/4223274154 wait complete. 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa20f6f9b80 20 client.15387 trim_cache size 0 max 16384 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:28.584 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24616, 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 59.117448963999998, 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 99.329677047000004 2023-10-08T09:14:28.585 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1e80a1510 msgr2=0x7fa1e80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1e80a1510 0x7fa1e80a39d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009990 tx=0x7fa1f8000bc0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 msgr2=0x7fa1fc196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc196730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00e950 tx=0x7fa1ec00ec60 comp rx=0 tx=0).stop 2023-10-08T09:14:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 shutdown_connections 2023-10-08T09:14:28.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1e80a1510 0x7fa1e80a39d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1fc101d90 0x7fa1fc19dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1fc0fe800 0x7fa1fc196730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 --2- 172.21.15.49:0/4256557319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1fc0ff1a0 0x7fa1fc196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 >> 172.21.15.49:0/4256557319 conn(0x7fa1fc0fa3a0 msgr2=0x7fa1fc104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 shutdown_connections 2023-10-08T09:14:28.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.567+0000 7fa203510700 1 -- 172.21.15.49:0/4256557319 wait complete. 2023-10-08T09:14:28.588 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T09:14:28.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 -- 172.21.15.49:0/4213743650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0101250 msgr2=0x7f75c0103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/4213743650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0101250 0x7f75c0103640 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f75b4009990 tx=0x7f75b4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 -- 172.21.15.49:0/4213743650 shutdown_connections 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/4213743650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75c01064b0 0x7f75c0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/4213743650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0101250 0x7f75c0103640 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/4213743650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c0103b80 0x7f75c0105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 -- 172.21.15.49:0/4213743650 >> 172.21.15.49:0/4213743650 conn(0x7f75c00fac40 msgr2=0x7f75c00fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 -- 172.21.15.49:0/4213743650 shutdown_connections 2023-10-08T09:14:28.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.716+0000 7f75c4e5d700 1 -- 172.21.15.49:0/4213743650 wait complete. 2023-10-08T09:14:28.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 Processor -- start 2023-10-08T09:14:28.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 -- start start 2023-10-08T09:14:28.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c0101250 0x7f75c0196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0103b80 0x7f75c0196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75c01064b0 0x7f75c019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c019e2b0 con 0x7f75c0103b80 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c019e3f0 con 0x7f75c0101250 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75c4e5d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c019e580 con 0x7f75c01064b0 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0103b80 0x7f75c0196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0103b80 0x7f75c0196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49290/0 (socket says 172.21.15.49:49290) 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 -- 172.21.15.49:0/1561329735 learned_addr learned my addr 172.21.15.49:0/1561329735 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75be59c700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c0101250 0x7f75c0196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bed9d700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75c01064b0 0x7f75c019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 -- 172.21.15.49:0/1561329735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75c01064b0 msgr2=0x7f75c019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75c01064b0 0x7f75c019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 -- 172.21.15.49:0/1561329735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c0101250 msgr2=0x7f75c0196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c0101250 0x7f75c0196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.717+0000 7f75bdd9b700 1 -- 172.21.15.49:0/1561329735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40095f0 con 0x7f75c0103b80 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.718+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0103b80 0x7f75c0196cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c920 tx=0x7f75b000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.718+0000 7f75af7fe700 1 -- 172.21.15.49:0/1561329735 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0007b00 con 0x7f75c0103b80 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.718+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c019e860 con 0x7f75c0103b80 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.718+0000 7f75af7fe700 1 -- 172.21.15.49:0/1561329735 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75b000ce70 con 0x7f75c0103b80 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.718+0000 7f75af7fe700 1 -- 172.21.15.49:0/1561329735 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0005460 con 0x7f75c0103b80 2023-10-08T09:14:28.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.718+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c019edc0 con 0x7f75c0103b80 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.720+0000 7f75af7fe700 1 -- 172.21.15.49:0/1561329735 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f75b0004750 con 0x7f75c0103b80 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.721+0000 7f75af7fe700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f75a80a1510 0x7f75a80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.721+0000 7f75af7fe700 1 -- 172.21.15.49:0/1561329735 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f75b00cce70 con 0x7f75c0103b80 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.721+0000 7f75be59c700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f75a80a1510 0x7f75a80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.721+0000 7f75d1046b80 1 Processor -- start 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.721+0000 7f75d1046b80 1 -- start start 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75be59c700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f75a80a1510 0x7f75a80a39d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b2e0 tx=0x7f75b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bad80 0x557f744bb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bbbc0 0x557f744bc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75d1046b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7441e410 con 0x557f744bbbc0 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75d1046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7440fbd0 con 0x557f744bad80 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75d1046b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7440d4f0 con 0x557f744ba420 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bbbc0 0x557f744bc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bbbc0 0x557f744bc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49300/0 (socket says 172.21.15.49:49300) 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 -- 172.21.15.49:0/1897003384 learned_addr learned my addr 172.21.15.49:0/1897003384 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75be59c700 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bad80 0x557f744bb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 -- 172.21.15.49:0/1897003384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 msgr2=0x557f744ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 -- 172.21.15.49:0/1897003384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bad80 msgr2=0x557f744bb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bad80 0x557f744bb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 -- 172.21.15.49:0/1897003384 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f74233d90 con 0x557f744bbbc0 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75be59c700 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bad80 0x557f744bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75bed9d700 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bbbc0 0x557f744bc050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009440 tx=0x7f75b8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef064cc1514479a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.722+0000 7f75acff9700 1 -- 172.21.15.49:0/1897003384 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b800e070 con 0x557f744bbbc0 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75acff9700 1 -- 172.21.15.49:0/1897003384 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75b800f040 con 0x557f744bbbc0 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75acff9700 1 -- 172.21.15.49:0/1897003384 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8015b00 con 0x557f744bbbc0 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 -- 172.21.15.49:0/1897003384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bbbc0 msgr2=0x557f744bc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bbbc0 0x557f744bc050 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009440 tx=0x7f75b8009920 comp rx=0 tx=0).stop 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 -- 172.21.15.49:0/1897003384 shutdown_connections 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bbbc0 0x557f744bc050 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 --2- 172.21.15.49:0/1897003384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bad80 0x557f744bb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 -- 172.21.15.49:0/1897003384 >> 172.21.15.49:0/1897003384 conn(0x557f7442a210 msgr2=0x557f744b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 -- 172.21.15.49:0/1897003384 shutdown_connections 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.723+0000 7f75d1046b80 1 -- 172.21.15.49:0/1897003384 wait complete. 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 -- start start 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 Processor -- start 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744ba420 0x557f744c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bad80 0x557f744c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f74408de0 con 0x557f744ba420 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f743a56c0 con 0x557f744bbbc0 2023-10-08T09:14:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f743a5360 con 0x557f744bad80 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75d1046b80 10 client.0 _ll_register_callbacks cb 0x557f742027b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75be59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744ba420 0x557f744c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75be59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744ba420 0x557f744c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49302/0 (socket says 172.21.15.49:49302) 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75be59c700 1 -- 172.21.15.49:0/2713791952 learned_addr learned my addr 172.21.15.49:0/2713791952 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bad80 0x557f744c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.724+0000 7f75bed9d700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75be59c700 1 -- 172.21.15.49:0/2713791952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bad80 msgr2=0x557f744c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75be59c700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bad80 0x557f744c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75be59c700 1 -- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 msgr2=0x557f744cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75be59c700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75be59c700 1 -- 172.21.15.49:0/2713791952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b80090f0 con 0x557f744ba420 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75bed9d700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:28.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75be59c700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744ba420 0x557f744c40d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b920 tx=0x7f75b4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5caabb72276ff26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2713791952 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400f660 con 0x557f744ba420 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2713791952 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f75b400f7c0 con 0x557f744ba420 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f743d0e60 con 0x557f744ba420 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2713791952 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b4004e80 con 0x557f744ba420 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75d1046b80 10 client.15399 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f7421cc10 con 0x557f744ba420 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.725+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2713791952 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f743d0e60 con 0x557f744ba420 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2713791952 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75b402fa80 con 0x557f744ba420 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75d1046b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75d1046b80 10 client.15399 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:14:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75d1046b80 20 client.15399 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f744cfc80 0x557f744ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75d1046b80 4 client.15399 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f74401dc0 con 0x557f744cfc80 2023-10-08T09:14:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f744cfc80 0x557f744ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75a5ffb700 10 client.15399 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.726+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f744cfc80 0x557f744ce760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=163e5e17582d91c3 server_cookie=81b88eb3f0616afd in_seq=0 out_seq=0 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2713791952 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557f74401dc0 con 0x557f744cfc80 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75a5ffb700 10 client.15399 handle_command_reply: tid=0 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 client.15399 shutdown 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744ba420 msgr2=0x557f744c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744ba420 0x557f744c40d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b920 tx=0x7f75b4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 shutdown_connections 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f744cfc80 0x557f744ce760 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75a5ffb700 0 client.15399 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744ba420 0x557f744c40d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2713791952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bad80 0x557f744c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 >> 172.21.15.49:0/2713791952 conn(0x557f7442a210 msgr2=0x557f744b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 shutdown_connections 2023-10-08T09:14:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 1 -- 172.21.15.49:0/2713791952 wait complete. 2023-10-08T09:14:28.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.727+0000 7f75d1046b80 20 client.15399 trim_cache size 0 max 16384 2023-10-08T09:14:28.750 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 Processor -- start 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 -- start start 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744ba420 0x557f744c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bbbc0 0x557f744bea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f742898d0 con 0x557f744bad80 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f744d0a40 con 0x557f744ba420 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.745+0000 7f75d1046b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f74404ed0 con 0x557f744bbbc0 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49312/0 (socket says 172.21.15.49:49312) 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2135526851 learned_addr learned my addr 172.21.15.49:0/2135526851 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75be59c700 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744ba420 0x557f744c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bed9d700 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bbbc0 0x557f744bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2135526851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bbbc0 msgr2=0x557f744bea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bbbc0 0x557f744bea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2135526851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744ba420 msgr2=0x557f744c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744ba420 0x557f744c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2135526851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f7425a200 con 0x557f744bad80 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75be59c700 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744ba420 0x557f744c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744cb610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f75b00961c0 tx=0x7f75b00996a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1e858b3f2c2d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75ad7fa700 1 -- 172.21.15.49:0/2135526851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0099cc0 con 0x557f744bad80 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75ad7fa700 1 -- 172.21.15.49:0/2135526851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75b009de10 con 0x557f744bad80 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.746+0000 7f75ad7fa700 1 -- 172.21.15.49:0/2135526851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0099f20 con 0x557f744bad80 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 -- 172.21.15.49:0/2135526851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 msgr2=0x557f744cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744cb610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f75b00961c0 tx=0x7f75b00996a0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 -- 172.21.15.49:0/2135526851 shutdown_connections 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744bbbc0 0x557f744bea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744ba420 0x557f744c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2135526851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744cb610 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 -- 172.21.15.49:0/2135526851 >> 172.21.15.49:0/2135526851 conn(0x557f7442a210 msgr2=0x557f744acee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 -- 172.21.15.49:0/2135526851 shutdown_connections 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 -- 172.21.15.49:0/2135526851 wait complete. 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 -- start start 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.747+0000 7f75d1046b80 1 Processor -- start 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f744cb460 con 0x557f744bad80 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f744cb5d0 con 0x557f744bbbc0 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f744cb790 con 0x557f744ba420 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 10 client.0 _ll_register_callbacks cb 0x7f75b4005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49318/0 (socket says 172.21.15.49:49318) 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75be59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2183480762 learned_addr learned my addr 172.21.15.49:0/2183480762 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bed9d700 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2183480762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 msgr2=0x557f744c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 msgr2=0x557f744caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 -- 172.21.15.49:0/2183480762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c019edc0 con 0x557f744bad80 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75bdd9b700 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744c5ef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f75b00b4040 tx=0x7f75b0099b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e838f64e468e59e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2183480762 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00993b0 con 0x557f744bad80 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2183480762 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f75b0098060 con 0x557f744bad80 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.748+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75b800f040 con 0x557f744bad80 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2183480762 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b0099cc0 con 0x557f744bad80 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 10 client.15405 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557f74232260 con 0x557f744bad80 2023-10-08T09:14:28.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2183480762 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75b0083020 con 0x557f744bad80 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2183480762 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f75b00a1bc0 con 0x557f744bad80 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 10 client.15405 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 20 client.15405 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f745280e0 0x557f745245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 4 client.15405 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f74401ae0 con 0x557f745280e0 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.749+0000 7f75be59c700 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f745280e0 0x557f745245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75be59c700 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f745280e0 0x557f745245c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff3cb713d95aecd server_cookie=3fe0b85e754e6ba3 in_seq=0 out_seq=0 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75a5ffb700 10 client.15405 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75a5ffb700 1 -- 172.21.15.49:0/2183480762 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557f74401ae0 con 0x557f745280e0 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75a5ffb700 10 client.15405 handle_command_reply: tid=0 2023-10-08T09:14:28.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 client.15405 shutdown 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 msgr2=0x557f744c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744c5ef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f75b00b4040 tx=0x7f75b0099b90 comp rx=0 tx=0).stop 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 shutdown_connections 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557f745280e0 0x557f745245c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557f744ba420 0x557f744c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557f744bbbc0 0x557f744caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 --2- 172.21.15.49:0/2183480762 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557f744bad80 0x557f744c5ef0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75a5ffb700 0 client.15405 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 >> 172.21.15.49:0/2183480762 conn(0x557f7442a210 msgr2=0x557f744c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 shutdown_connections 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 1 -- 172.21.15.49:0/2183480762 wait complete. 2023-10-08T09:14:28.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.750+0000 7f75d1046b80 20 client.15405 trim_cache size 0 max 16384 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24638, 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 85.030818332999999, 2023-10-08T09:14:28.758 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:28.759 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 88, 2023-10-08T09:14:28.759 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:28.759 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 92.536218321000007 2023-10-08T09:14:28.759 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:28.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f75a80a1510 msgr2=0x7f75a80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f75a80a1510 0x7f75a80a39d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b2e0 tx=0x7f75b4005fb0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0103b80 msgr2=0x7f75c0196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0103b80 0x7f75c0196cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c920 tx=0x7f75b000cc30 comp rx=0 tx=0).stop 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 shutdown_connections 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f75a80a1510 0x7f75a80a39d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75c01064b0 0x7f75c019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75c0101250 0x7f75c0196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 --2- 172.21.15.49:0/1561329735 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0103b80 0x7f75c0196cc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 >> 172.21.15.49:0/1561329735 conn(0x7f75c00fac40 msgr2=0x7f75c01030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 shutdown_connections 2023-10-08T09:14:28.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:28.751+0000 7f75c4e5d700 1 -- 172.21.15.49:0/1561329735 wait complete. 2023-10-08T09:14:28.764 INFO:tasks.cephadm:Setup complete, yielding 2023-10-08T09:14:28.764 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-08T09:14:28.774 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:14:28.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-08T09:14:28.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:28 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1263873206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:28.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:28 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1263873206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:29.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:28 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1263873206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:29.338 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:29 smithi049 ceph-mon[99485]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T09:14:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:29 smithi184 ceph-mon[103675]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T09:14:30.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:29 smithi132 ceph-mon[103389]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-08T09:14:30.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 -- 172.21.15.49:0/1571190900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 msgr2=0x7f258c0ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:30.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 --2- 172.21.15.49:0/1571190900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c0ffb50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f25800099c0 tx=0x7f2580009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 -- 172.21.15.49:0/1571190900 shutdown_connections 2023-10-08T09:14:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 --2- 172.21.15.49:0/1571190900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258c1012d0 0x7f258c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 --2- 172.21.15.49:0/1571190900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c0ffb50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 --2- 172.21.15.49:0/1571190900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f258c100930 0x7f258c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 -- 172.21.15.49:0/1571190900 >> 172.21.15.49:0/1571190900 conn(0x7f258c0facf0 msgr2=0x7f258c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.637+0000 7f2590cd4700 1 -- 172.21.15.49:0/1571190900 shutdown_connections 2023-10-08T09:14:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.638+0000 7f2590cd4700 1 -- 172.21.15.49:0/1571190900 wait complete. 2023-10-08T09:14:30.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 Processor -- start 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 -- start start 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f258c100930 0x7f258c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258c1012d0 0x7f258c19ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1a0520 con 0x7f258c0ff730 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1a0660 con 0x7f258c1012d0 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2590cd4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1a07f0 con 0x7f258c100930 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49328/0 (socket says 172.21.15.49:49328) 2023-10-08T09:14:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 -- 172.21.15.49:0/4059440737 learned_addr learned my addr 172.21.15.49:0/4059440737 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:30.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f2589d9b700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f258c100930 0x7f258c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:30.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258ad9d700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258c1012d0 0x7f258c19ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:30.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 -- 172.21.15.49:0/4059440737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f258c100930 msgr2=0x7f258c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:30.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f258c100930 0x7f258c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 -- 172.21.15.49:0/4059440737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258c1012d0 msgr2=0x7f258c19ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:30.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258c1012d0 0x7f258c19ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258a59c700 1 -- 172.21.15.49:0/4059440737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25800095f0 con 0x7f258c0ff730 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.639+0000 7f258ad9d700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258c1012d0 0x7f258c19ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.640+0000 7f258a59c700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c1989e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2580009f50 tx=0x7f2580005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.640+0000 7f257b7fe700 1 -- 172.21.15.49:0/4059440737 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258001d070 con 0x7f258c0ff730 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.640+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c1a0a70 con 0x7f258c0ff730 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.640+0000 7f257b7fe700 1 -- 172.21.15.49:0/4059440737 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2580004150 con 0x7f258c0ff730 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.640+0000 7f257b7fe700 1 -- 172.21.15.49:0/4059440737 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580005320 con 0x7f258c0ff730 2023-10-08T09:14:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.640+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c1a0ee0 con 0x7f258c0ff730 2023-10-08T09:14:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.641+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c04e4c0 con 0x7f258c0ff730 2023-10-08T09:14:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.642+0000 7f257b7fe700 1 -- 172.21.15.49:0/4059440737 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f25800049e0 con 0x7f258c0ff730 2023-10-08T09:14:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.642+0000 7f257b7fe700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f25740a1510 0x7f25740a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.643+0000 7f2589d9b700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f25740a1510 0x7f25740a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.643+0000 7f257b7fe700 1 -- 172.21.15.49:0/4059440737 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f25800ce0d0 con 0x7f258c0ff730 2023-10-08T09:14:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.643+0000 7f2589d9b700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f25740a1510 0x7f25740a39d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f257c005fd0 tx=0x7f257c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.647+0000 7f257b7fe700 1 -- 172.21.15.49:0/4059440737 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f25800972f0 con 0x7f258c0ff730 2023-10-08T09:14:30.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.867+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f258c0688d0 con 0x7f25740a1510 2023-10-08T09:14:30.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.868+0000 7f257b7fe700 1 -- 172.21.15.49:0/4059440737 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f258c0688d0 con 0x7f25740a1510 2023-10-08T09:14:30.869 INFO:teuthology.orchestra.run.smithi049.stdout:Backend: cephadm 2023-10-08T09:14:30.869 INFO:teuthology.orchestra.run.smithi049.stdout:Available: Yes 2023-10-08T09:14:30.869 INFO:teuthology.orchestra.run.smithi049.stdout:Paused: No 2023-10-08T09:14:30.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f25740a1510 msgr2=0x7f25740a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:30.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f25740a1510 0x7f25740a39d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f257c005fd0 tx=0x7f257c00a560 comp rx=0 tx=0).stop 2023-10-08T09:14:30.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 msgr2=0x7f258c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:30.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c1989e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2580009f50 tx=0x7f2580005af0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 shutdown_connections 2023-10-08T09:14:30.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f25740a1510 0x7f25740a39d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f258c1012d0 0x7f258c19ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f258c0ff730 0x7f258c1989e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.870+0000 7f2590cd4700 1 --2- 172.21.15.49:0/4059440737 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f258c100930 0x7f258c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:30.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.871+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 >> 172.21.15.49:0/4059440737 conn(0x7f258c0facf0 msgr2=0x7f258c0681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:30.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.871+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 shutdown_connections 2023-10-08T09:14:30.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:30.871+0000 7f2590cd4700 1 -- 172.21.15.49:0/4059440737 wait complete. 2023-10-08T09:14:31.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-08T09:14:31.977 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:31.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:31 smithi184 ceph-mon[103675]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-08T09:14:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:31 smithi132 ceph-mon[103389]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-08T09:14:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:31 smithi049 ceph-mon[99485]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-08T09:14:32.807 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:32 smithi049 ceph-mon[99485]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:32.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:32 smithi184 ceph-mon[103675]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:33.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:32 smithi132 ceph-mon[103389]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 -- 172.21.15.49:0/3513044262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5040ff730 msgr2=0x7fe5040ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 --2- 172.21.15.49:0/3513044262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5040ff730 0x7fe5040ffb50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe5000099c0 tx=0x7fe500009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 -- 172.21.15.49:0/3513044262 shutdown_connections 2023-10-08T09:14:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 --2- 172.21.15.49:0/3513044262 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5041012d0 0x7fe504107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 --2- 172.21.15.49:0/3513044262 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5040ff730 0x7fe5040ffb50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 --2- 172.21.15.49:0/3513044262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe504100930 0x7fe504100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 -- 172.21.15.49:0/3513044262 >> 172.21.15.49:0/3513044262 conn(0x7fe5040facf0 msgr2=0x7fe5040fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 -- 172.21.15.49:0/3513044262 shutdown_connections 2023-10-08T09:14:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.279+0000 7fe50b86f700 1 -- 172.21.15.49:0/3513044262 wait complete. 2023-10-08T09:14:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 Processor -- start 2023-10-08T09:14:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 -- start start 2023-10-08T09:14:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5040ff730 0x7fe504069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe504100930 0x7fe50419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041012d0 0x7fe50419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50419e250 con 0x7fe5041012d0 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50419e3c0 con 0x7fe504100930 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50b86f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50419e580 con 0x7fe5040ff730 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe509e0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041012d0 0x7fe50419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe50960b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5040ff730 0x7fe504069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe509e0c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041012d0 0x7fe50419dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49344/0 (socket says 172.21.15.49:49344) 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe509e0c700 1 -- 172.21.15.49:0/921442608 learned_addr learned my addr 172.21.15.49:0/921442608 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.280+0000 7fe508e0a700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe504100930 0x7fe50419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe509e0c700 1 -- 172.21.15.49:0/921442608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5040ff730 msgr2=0x7fe504069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe509e0c700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5040ff730 0x7fe504069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe509e0c700 1 -- 172.21.15.49:0/921442608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe504100930 msgr2=0x7fe50419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe509e0c700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe504100930 0x7fe50419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe509e0c700 1 -- 172.21.15.49:0/921442608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5000095f0 con 0x7fe5041012d0 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe509e0c700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041012d0 0x7fe50419dab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ada0 tx=0x7fe4fc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe4fa7fc700 1 -- 172.21.15.49:0/921442608 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00f070 con 0x7fe5041012d0 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe4fa7fc700 1 -- 172.21.15.49:0/921442608 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4fc007b30 con 0x7fe5041012d0 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50419e860 con 0x7fe5041012d0 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe4fa7fc700 1 -- 172.21.15.49:0/921442608 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc005280 con 0x7fe5041012d0 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.281+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50419ed60 con 0x7fe5041012d0 2023-10-08T09:14:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.282+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe504106cb0 con 0x7fe5041012d0 2023-10-08T09:14:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.283+0000 7fe4fa7fc700 1 -- 172.21.15.49:0/921442608 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe4fc00c8d0 con 0x7fe5041012d0 2023-10-08T09:14:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.284+0000 7fe4fa7fc700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4f00a1510 0x7fe4f00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.284+0000 7fe50960b700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4f00a1510 0x7fe4f00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.284+0000 7fe4fa7fc700 1 -- 172.21.15.49:0/921442608 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe4fc0cc700 con 0x7fe5041012d0 2023-10-08T09:14:33.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.284+0000 7fe50960b700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4f00a1510 0x7fe4f00a39d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe500003820 tx=0x7fe500005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:33.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.288+0000 7fe4fa7fc700 1 -- 172.21.15.49:0/921442608 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4fc0993b0 con 0x7fe5041012d0 2023-10-08T09:14:33.496 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.496+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe504060c30 con 0x7fe4f00a1510 2023-10-08T09:14:33.518 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.518+0000 7fe4fa7fc700 1 -- 172.21.15.49:0/921442608 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fe504060c30 con 0x7fe4f00a1510 2023-10-08T09:14:33.518 INFO:teuthology.orchestra.run.smithi049.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-08T09:14:33.518 INFO:teuthology.orchestra.run.smithi049.stdout:mds.a smithi049 running (100s) 18s ago 99s 19.6M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 4431e06f879a 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.b smithi132 running (97s) 27s ago 97s 30.3M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 05752a862bf7 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.c smithi184 running (108s) 27s ago 108s 21.3M - 18.0.0-6639-g7b3143ba 09a1f3752b5d ebe413d6db30 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.d smithi049 running (115s) 18s ago 115s 16.5M - 18.0.0-6639-g7b3143ba 09a1f3752b5d f31b5393a2a0 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.e smithi132 running (113s) 27s ago 112s 17.5M - 18.0.0-6639-g7b3143ba 09a1f3752b5d fa3baf1f7462 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.f smithi184 running (111s) 27s ago 110s 18.9M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 83239c99ec70 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.g smithi049 running (102s) 18s ago 101s 18.9M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 9b0e6224674a 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.h smithi132 running (106s) 27s ago 106s 18.8M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 3778bda780d4 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.i smithi184 running (104s) 27s ago 104s 21.2M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 47018045fe63 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.j smithi049 running (117s) 18s ago 117s 16.5M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 71c18cc96bf9 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.k smithi132 running (119s) 27s ago 119s 20.2M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 8ac49e162211 2023-10-08T09:14:33.519 INFO:teuthology.orchestra.run.smithi049.stdout:mds.l smithi184 running (2m) 27s ago 2m 21.3M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 08b8cfe48cc1 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.x smithi049 *:9283,8765 running (8m) 18s ago 8m 523M - 18.0.0-6639-g7b3143ba 09a1f3752b5d c5c334124445 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.y smithi132 *:8443,8765 running (6m) 27s ago 6m 451M - 18.0.0-6639-g7b3143ba 09a1f3752b5d 9341b0223e01 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:mgr.z smithi184 *:8443,8765 running (6m) 27s ago 6m 451M - 18.0.0-6639-g7b3143ba 09a1f3752b5d fa9061cb56a3 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a smithi049 running (8m) 18s ago 8m 98.0M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d f54a132ddc75 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b smithi132 running (7m) 27s ago 7m 63.7M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d dc5533f275d6 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c smithi184 running (7m) 27s ago 7m 64.7M 2048M 18.0.0-6639-g7b3143ba 09a1f3752b5d e0f434763804 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 smithi049 running (6m) 18s ago 6m 71.2M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d d7b76ea7187e 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 smithi132 running (5m) 27s ago 5m 78.8M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 0e39d9c1852f 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 smithi184 running (5m) 27s ago 5m 76.7M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 4e571e012cac 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 smithi049 running (5m) 18s ago 5m 66.4M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d a2e95f568949 2023-10-08T09:14:33.520 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 smithi132 running (4m) 27s ago 4m 65.3M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 0a6d413582d1 2023-10-08T09:14:33.521 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 smithi184 running (4m) 27s ago 4m 71.9M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 76a0fd3b4ef2 2023-10-08T09:14:33.521 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 smithi049 running (4m) 18s ago 4m 61.8M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d c23afea048ed 2023-10-08T09:14:33.521 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 smithi132 running (3m) 27s ago 3m 66.2M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 8786da84a39c 2023-10-08T09:14:33.521 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 smithi184 running (3m) 27s ago 3m 62.9M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 765be9bfa4aa 2023-10-08T09:14:33.521 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 smithi049 running (3m) 18s ago 3m 63.9M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 4c627c74bbe6 2023-10-08T09:14:33.521 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 smithi132 running (2m) 27s ago 2m 63.0M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 592fb73fe236 2023-10-08T09:14:33.521 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 smithi184 running (2m) 27s ago 2m 59.0M 4292M 18.0.0-6639-g7b3143ba 09a1f3752b5d 493a0dbee57a 2023-10-08T09:14:33.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4f00a1510 msgr2=0x7fe4f00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:33.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4f00a1510 0x7fe4f00a39d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe500003820 tx=0x7fe500005920 comp rx=0 tx=0).stop 2023-10-08T09:14:33.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041012d0 msgr2=0x7fe50419dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:33.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041012d0 0x7fe50419dab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ada0 tx=0x7fe4fc00c330 comp rx=0 tx=0).stop 2023-10-08T09:14:33.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 shutdown_connections 2023-10-08T09:14:33.522 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe4f00a1510 0x7fe4f00a39d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5040ff730 0x7fe504069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5041012d0 0x7fe50419dab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 --2- 172.21.15.49:0/921442608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe504100930 0x7fe50419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:33.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 >> 172.21.15.49:0/921442608 conn(0x7fe5040facf0 msgr2=0x7fe504103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:33.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 shutdown_connections 2023-10-08T09:14:33.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:33.520+0000 7fe50b86f700 1 -- 172.21.15.49:0/921442608 wait complete. 2023-10-08T09:14:33.870 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:33 smithi049 ceph-mon[99485]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T09:14:33.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-08T09:14:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:33 smithi184 ceph-mon[103675]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T09:14:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:33 smithi132 ceph-mon[103389]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-08T09:14:34.505 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:34.961 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:34 smithi049 ceph-mon[99485]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:34.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:34 smithi184 ceph-mon[103675]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:35.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:34 smithi132 ceph-mon[103389]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 -- 172.21.15.49:0/4144960237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b001012d0 msgr2=0x7f9b00107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 --2- 172.21.15.49:0/4144960237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b001012d0 0x7f9b00107b80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9af800b0d0 tx=0x7f9af800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 -- 172.21.15.49:0/4144960237 shutdown_connections 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 --2- 172.21.15.49:0/4144960237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b000ff730 0x7f9b000ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 --2- 172.21.15.49:0/4144960237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b001012d0 0x7f9b00107b80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 --2- 172.21.15.49:0/4144960237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b00100930 0x7f9b00100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 -- 172.21.15.49:0/4144960237 >> 172.21.15.49:0/4144960237 conn(0x7f9b000facf0 msgr2=0x7f9b000fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:35.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 -- 172.21.15.49:0/4144960237 shutdown_connections 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.844+0000 7f9b06693700 1 -- 172.21.15.49:0/4144960237 wait complete. 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 Processor -- start 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 -- start start 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b000ff730 0x7f9b00198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b00100930 0x7f9b00198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b001012d0 0x7f9b0019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b001a0580 con 0x7f9b00100930 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b001a06c0 con 0x7f9b000ff730 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9b06693700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b001a0850 con 0x7f9b001012d0 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9aff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b00100930 0x7f9b00198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:35.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9aff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b00100930 0x7f9b00198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49354/0 (socket says 172.21.15.49:49354) 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.845+0000 7f9aff7fe700 1 -- 172.21.15.49:0/637206557 learned_addr learned my addr 172.21.15.49:0/637206557 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9afffff700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b000ff730 0x7f9b00198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9b04c30700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b001012d0 0x7f9b0019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9aff7fe700 1 -- 172.21.15.49:0/637206557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b001012d0 msgr2=0x7f9b0019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9aff7fe700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b001012d0 0x7f9b0019ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9aff7fe700 1 -- 172.21.15.49:0/637206557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b000ff730 msgr2=0x7f9b00198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9aff7fe700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b000ff730 0x7f9b00198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9aff7fe700 1 -- 172.21.15.49:0/637206557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8009d00 con 0x7f9b00100930 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9aff7fe700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b00100930 0x7f9b00198f80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9af400d980 tx=0x7f9af400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9afd7fa700 1 -- 172.21.15.49:0/637206557 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40098d0 con 0x7f9b00100930 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9afd7fa700 1 -- 172.21.15.49:0/637206557 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9af4010460 con 0x7f9b00100930 2023-10-08T09:14:35.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.846+0000 7f9afd7fa700 1 -- 172.21.15.49:0/637206557 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af400f620 con 0x7f9b00100930 2023-10-08T09:14:35.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.848+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b001a0b30 con 0x7f9b00100930 2023-10-08T09:14:35.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.848+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b001a0fb0 con 0x7f9b00100930 2023-10-08T09:14:35.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.849+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0004e4c0 con 0x7f9b00100930 2023-10-08T09:14:35.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.850+0000 7f9afd7fa700 1 -- 172.21.15.49:0/637206557 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9af4010860 con 0x7f9b00100930 2023-10-08T09:14:35.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.850+0000 7f9afd7fa700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ae80a1510 0x7f9ae80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:35.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.850+0000 7f9afffff700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ae80a1510 0x7f9ae80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:35.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.850+0000 7f9afd7fa700 1 -- 172.21.15.49:0/637206557 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9af40cd050 con 0x7f9b00100930 2023-10-08T09:14:35.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.851+0000 7f9afffff700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ae80a1510 0x7f9ae80a39d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b00100790 tx=0x7f9af000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:35.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:35.856+0000 7f9afd7fa700 1 -- 172.21.15.49:0/637206557 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9af4096240 con 0x7f9b00100930 2023-10-08T09:14:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:35 smithi049 ceph-mon[99485]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:35.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:35 smithi184 ceph-mon[103675]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:36.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:35 smithi132 ceph-mon[103389]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:36.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.072+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b000fceb0 con 0x7f9ae80a1510 2023-10-08T09:14:36.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.080+0000 7f9afd7fa700 1 -- 172.21.15.49:0/637206557 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f9b000fceb0 con 0x7f9ae80a1510 2023-10-08T09:14:36.081 INFO:teuthology.orchestra.run.smithi049.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-08T09:14:36.081 INFO:teuthology.orchestra.run.smithi049.stdout:mds.all 12/12 30s ago 2m smithi049=a;smithi049=d;smithi049=g;smithi049=j;smithi132=b;smithi132=e;smithi132=h;smithi132=k;smithi184=c;smithi184=f;smithi184=i;smithi184=l;count:12 2023-10-08T09:14:36.081 INFO:teuthology.orchestra.run.smithi049.stdout:mgr 3/3 30s ago 6m smithi049=x;smithi132=y;smithi184=z;count:3 2023-10-08T09:14:36.081 INFO:teuthology.orchestra.run.smithi049.stdout:mon 3/3 30s ago 7m smithi049:172.21.15.49=a;smithi132:172.21.15.132=b;smithi184:172.21.15.184=c;count:3 2023-10-08T09:14:36.081 INFO:teuthology.orchestra.run.smithi049.stdout:osd 12 30s ago - 2023-10-08T09:14:36.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ae80a1510 msgr2=0x7f9ae80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:36.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ae80a1510 0x7f9ae80a39d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b00100790 tx=0x7f9af000b410 comp rx=0 tx=0).stop 2023-10-08T09:14:36.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b00100930 msgr2=0x7f9b00198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b00100930 0x7f9b00198f80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9af400d980 tx=0x7f9af400dc90 comp rx=0 tx=0).stop 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 shutdown_connections 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9ae80a1510 0x7f9ae80a39d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b001012d0 0x7f9b0019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b000ff730 0x7f9b00198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 --2- 172.21.15.49:0/637206557 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9b00100930 0x7f9b00198f80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 >> 172.21.15.49:0/637206557 conn(0x7f9b000facf0 msgr2=0x7f9b000fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:36.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 shutdown_connections 2023-10-08T09:14:36.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:36.082+0000 7f9b06693700 1 -- 172.21.15.49:0/637206557 wait complete. 2023-10-08T09:14:36.667 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-08T09:14:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:37 smithi184 ceph-mon[103675]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:37 smithi184 ceph-mon[103675]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:37 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2023-10-08T09:14:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:37 smithi184 ceph-mon[103675]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-08T09:14:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:37 smithi132 ceph-mon[103389]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:37 smithi132 ceph-mon[103389]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:37 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2023-10-08T09:14:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:37 smithi132 ceph-mon[103389]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-08T09:14:37.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:37 smithi049 ceph-mon[99485]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:37.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:37 smithi049 ceph-mon[99485]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:37.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:37 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2023-10-08T09:14:37.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:37 smithi049 ceph-mon[99485]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-08T09:14:38.177 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:38 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2023-10-08T09:14:38.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:38 smithi184 ceph-mon[103675]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T09:14:38.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:38 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2023-10-08T09:14:38.752 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:38 smithi049 ceph-mon[99485]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T09:14:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:38 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2023-10-08T09:14:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:38 smithi132 ceph-mon[103389]: osdmap e89: 12 total, 12 up, 12 in 2023-10-08T09:14:39.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.520+0000 7f1290406700 1 -- 172.21.15.49:0/3932648902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 msgr2=0x7f1288107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:39.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.520+0000 7f1290406700 1 --2- 172.21.15.49:0/3932648902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f1288107b20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f128000b0d0 tx=0x7f128000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 -- 172.21.15.49:0/3932648902 shutdown_connections 2023-10-08T09:14:39.524 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 --2- 172.21.15.49:0/3932648902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880ff6d0 0x7f12880ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 --2- 172.21.15.49:0/3932648902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f1288107b20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 --2- 172.21.15.49:0/3932648902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12881008d0 0x7f1288100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 -- 172.21.15.49:0/3932648902 >> 172.21.15.49:0/3932648902 conn(0x7f12880fac70 msgr2=0x7f12880fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 -- 172.21.15.49:0/3932648902 shutdown_connections 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 -- 172.21.15.49:0/3932648902 wait complete. 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 Processor -- start 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.521+0000 7f1290406700 1 -- start start 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f1290406700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880ff6d0 0x7f1288068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f1290406700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12881008d0 0x7f128819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f1290406700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f128819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f1290406700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819e290 con 0x7f1288101270 2023-10-08T09:14:39.525 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f1290406700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819e400 con 0x7f12881008d0 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f1290406700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819e5c0 con 0x7f12880ff6d0 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f128819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f128819daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49368/0 (socket says 172.21.15.49:49368) 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 -- 172.21.15.49:0/2881007842 learned_addr learned my addr 172.21.15.49:0/2881007842 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e1a2700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880ff6d0 0x7f1288068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128d9a1700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12881008d0 0x7f128819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 -- 172.21.15.49:0/2881007842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880ff6d0 msgr2=0x7f1288068c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880ff6d0 0x7f1288068c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 -- 172.21.15.49:0/2881007842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12881008d0 msgr2=0x7f128819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12881008d0 0x7f128819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.526 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 -- 172.21.15.49:0/2881007842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280009d00 con 0x7f1288101270 2023-10-08T09:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.522+0000 7f128e9a3700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f128819daf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1280007bd0 tx=0x7f1280009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.523+0000 7f127f7fe700 1 -- 172.21.15.49:0/2881007842 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128000e040 con 0x7f1288101270 2023-10-08T09:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.523+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128819e840 con 0x7f1288101270 2023-10-08T09:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.523+0000 7f127f7fe700 1 -- 172.21.15.49:0/2881007842 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128001dd50 con 0x7f1288101270 2023-10-08T09:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.523+0000 7f127f7fe700 1 -- 172.21.15.49:0/2881007842 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800123f0 con 0x7f1288101270 2023-10-08T09:14:39.527 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.523+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128819ed00 con 0x7f1288101270 2023-10-08T09:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.524+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1288106ab0 con 0x7f1288101270 2023-10-08T09:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.525+0000 7f127f7fe700 1 -- 172.21.15.49:0/2881007842 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f128001d890 con 0x7f1288101270 2023-10-08T09:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.526+0000 7f127f7fe700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f12740a1480 0x7f12740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.526+0000 7f128e1a2700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f12740a1480 0x7f12740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.526+0000 7f127f7fe700 1 -- 172.21.15.49:0/2881007842 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f12800cf690 con 0x7f1288101270 2023-10-08T09:14:39.528 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.526+0000 7f128e1a2700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f12740a1480 0x7f12740a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1288100730 tx=0x7f1284007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:39.531 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.529+0000 7f127f7fe700 1 -- 172.21.15.49:0/2881007842 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f12800989f0 con 0x7f1288101270 2023-10-08T09:14:39.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:39 smithi049 ceph-mon[99485]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T09:14:39.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:39 smithi049 ceph-mon[99485]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T09:14:39.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.737+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f128804e4c0 con 0x7f12740a1480 2023-10-08T09:14:39.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.738+0000 7f127f7fe700 1 -- 172.21.15.49:0/2881007842 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f128804e4c0 con 0x7f12740a1480 2023-10-08T09:14:39.740 INFO:teuthology.orchestra.run.smithi049.stdout:HOST ADDR LABELS STATUS 2023-10-08T09:14:39.740 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 172.21.15.49 2023-10-08T09:14:39.740 INFO:teuthology.orchestra.run.smithi049.stdout:smithi132 172.21.15.132 2023-10-08T09:14:39.741 INFO:teuthology.orchestra.run.smithi049.stdout:smithi184 172.21.15.184 2023-10-08T09:14:39.741 INFO:teuthology.orchestra.run.smithi049.stdout:3 hosts in cluster 2023-10-08T09:14:39.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f12740a1480 msgr2=0x7f12740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:39.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f12740a1480 0x7f12740a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1288100730 tx=0x7f1284007400 comp rx=0 tx=0).stop 2023-10-08T09:14:39.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 msgr2=0x7f128819daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:39.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f128819daf0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1280007bd0 tx=0x7f1280009440 comp rx=0 tx=0).stop 2023-10-08T09:14:39.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 shutdown_connections 2023-10-08T09:14:39.743 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f12740a1480 0x7f12740a3940 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12880ff6d0 0x7f1288068c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1288101270 0x7f128819daf0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 --2- 172.21.15.49:0/2881007842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12881008d0 0x7f128819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:39.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 >> 172.21.15.49:0/2881007842 conn(0x7f12880fac70 msgr2=0x7f1288103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:39.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 shutdown_connections 2023-10-08T09:14:39.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:39.741+0000 7f1290406700 1 -- 172.21.15.49:0/2881007842 wait complete. 2023-10-08T09:14:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:39 smithi184 ceph-mon[103675]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T09:14:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:39 smithi184 ceph-mon[103675]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T09:14:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:39 smithi132 ceph-mon[103389]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-10-08T09:14:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:39 smithi132 ceph-mon[103389]: osdmap e90: 12 total, 12 up, 12 in 2023-10-08T09:14:40.144 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-08T09:14:40.711 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:41 smithi049 ceph-mon[99485]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:41 smithi049 ceph-mon[99485]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-08T09:14:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:41 smithi184 ceph-mon[103675]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:41 smithi184 ceph-mon[103675]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-08T09:14:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:41 smithi132 ceph-mon[103389]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:41 smithi132 ceph-mon[103389]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-08T09:14:42.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 -- 172.21.15.49:0/671394472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac1012d0 msgr2=0x7f26ac107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 --2- 172.21.15.49:0/671394472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac1012d0 0x7f26ac107b80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f26a400b0d0 tx=0x7f26a400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 -- 172.21.15.49:0/671394472 shutdown_connections 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 --2- 172.21.15.49:0/671394472 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac1012d0 0x7f26ac107b80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 --2- 172.21.15.49:0/671394472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0ff730 0x7f26ac0ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 --2- 172.21.15.49:0/671394472 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26ac100930 0x7f26ac100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 -- 172.21.15.49:0/671394472 >> 172.21.15.49:0/671394472 conn(0x7f26ac0facf0 msgr2=0x7f26ac0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 -- 172.21.15.49:0/671394472 shutdown_connections 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.042+0000 7f26b1e74700 1 -- 172.21.15.49:0/671394472 wait complete. 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 Processor -- start 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 -- start start 2023-10-08T09:14:42.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0ff730 0x7f26ac068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac100930 0x7f26ac19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26ac1012d0 0x7f26ac19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac19e260 con 0x7f26ac100930 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac19e3d0 con 0x7f26ac0ff730 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26b1e74700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac19e590 con 0x7f26ac1012d0 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26aaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac100930 0x7f26ac19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26aaffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac100930 0x7f26ac19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49380/0 (socket says 172.21.15.49:49380) 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26aaffd700 1 -- 172.21.15.49:0/99400501 learned_addr learned my addr 172.21.15.49:0/99400501 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26abfff700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26ac1012d0 0x7f26ac19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.043+0000 7f26ab7fe700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0ff730 0x7f26ac068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26aaffd700 1 -- 172.21.15.49:0/99400501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26ac1012d0 msgr2=0x7f26ac19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:42.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26aaffd700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26ac1012d0 0x7f26ac19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26aaffd700 1 -- 172.21.15.49:0/99400501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0ff730 msgr2=0x7f26ac068c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26aaffd700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0ff730 0x7f26ac068c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26aaffd700 1 -- 172.21.15.49:0/99400501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4009d00 con 0x7f26ac100930 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26ab7fe700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0ff730 0x7f26ac068c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26aaffd700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac100930 0x7f26ac19b510 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f269400c9b0 tx=0x7f269400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26a8ff9700 1 -- 172.21.15.49:0/99400501 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694007b00 con 0x7f26ac100930 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac19e870 con 0x7f26ac100930 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26a8ff9700 1 -- 172.21.15.49:0/99400501 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2694007c60 con 0x7f26ac100930 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26a8ff9700 1 -- 172.21.15.49:0/99400501 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694018660 con 0x7f26ac100930 2023-10-08T09:14:42.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.044+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac19edc0 con 0x7f26ac100930 2023-10-08T09:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.045+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac106af0 con 0x7f26ac100930 2023-10-08T09:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.046+0000 7f26a8ff9700 1 -- 172.21.15.49:0/99400501 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f269400f450 con 0x7f26ac100930 2023-10-08T09:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.047+0000 7f26a8ff9700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26980a1480 0x7f26980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.047+0000 7f26ab7fe700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26980a1480 0x7f26980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.047+0000 7f26a8ff9700 1 -- 172.21.15.49:0/99400501 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f26940cd2c0 con 0x7f26ac100930 2023-10-08T09:14:42.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.047+0000 7f26ab7fe700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26980a1480 0x7f26980a3940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f269c005950 tx=0x7f269c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:42.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.050+0000 7f26a8ff9700 1 -- 172.21.15.49:0/99400501 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2694096620 con 0x7f26ac100930 2023-10-08T09:14:42.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.258+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f26ac02cf80 con 0x7f26980a1480 2023-10-08T09:14:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.260+0000 7f26a8ff9700 1 -- 172.21.15.49:0/99400501 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f26ac02cf80 con 0x7f26980a1480 2023-10-08T09:14:42.261 INFO:teuthology.orchestra.run.smithi049.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-08T09:14:42.262 INFO:teuthology.orchestra.run.smithi049.stdout:smithi049 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747500D2480DGN 447G No 3m ago Has a FileSystem, LVM detected 2023-10-08T09:14:42.262 INFO:teuthology.orchestra.run.smithi049.stdout:smithi132 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232000B400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-08T09:14:42.262 INFO:teuthology.orchestra.run.smithi049.stdout:smithi184 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT626500CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-10-08T09:14:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26980a1480 msgr2=0x7f26980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26980a1480 0x7f26980a3940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f269c005950 tx=0x7f269c0058e0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac100930 msgr2=0x7f26ac19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac100930 0x7f26ac19b510 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f269400c9b0 tx=0x7f269400ccc0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 shutdown_connections 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f26980a1480 0x7f26980a3940 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f26ac1012d0 0x7f26ac19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26ac0ff730 0x7f26ac068c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 --2- 172.21.15.49:0/99400501 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f26ac100930 0x7f26ac19b510 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 >> 172.21.15.49:0/99400501 conn(0x7f26ac0facf0 msgr2=0x7f26ac103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 shutdown_connections 2023-10-08T09:14:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:42.262+0000 7f26b1e74700 1 -- 172.21.15.49:0/99400501 wait complete. 2023-10-08T09:14:42.681 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-08T09:14:42.690 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:14:42.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-08T09:14:43.257 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:43.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:43 smithi049 ceph-mon[99485]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-08T09:14:43.708 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:43 smithi049 ceph-mon[99485]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:43.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:43 smithi184 ceph-mon[103675]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-08T09:14:43.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:43 smithi184 ceph-mon[103675]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:43.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:43 smithi132 ceph-mon[103389]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 767 B/s wr, 12 op/s 2023-10-08T09:14:43.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:43 smithi132 ceph-mon[103389]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-08T09:14:44.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 -- 172.21.15.49:0/4018921486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c101360 msgr2=0x7fbe8c103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:44.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 --2- 172.21.15.49:0/4018921486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c101360 0x7fbe8c103750 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009990 tx=0x7fbe88009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 -- 172.21.15.49:0/4018921486 shutdown_connections 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 --2- 172.21.15.49:0/4018921486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe8c1065c0 0x7fbe8c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 --2- 172.21.15.49:0/4018921486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c101360 0x7fbe8c103750 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 --2- 172.21.15.49:0/4018921486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe8c103c90 0x7fbe8c106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 -- 172.21.15.49:0/4018921486 >> 172.21.15.49:0/4018921486 conn(0x7fbe8c0facf0 msgr2=0x7fbe8c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.583+0000 7fbe93a64700 1 -- 172.21.15.49:0/4018921486 shutdown_connections 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 -- 172.21.15.49:0/4018921486 wait complete. 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 Processor -- start 2023-10-08T09:14:44.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 -- start start 2023-10-08T09:14:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe8c101360 0x7fbe8c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe8c103c90 0x7fbe8c196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c1065c0 0x7fbe8c19dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c19e260 con 0x7fbe8c1065c0 2023-10-08T09:14:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c19e3a0 con 0x7fbe8c103c90 2023-10-08T09:14:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe93a64700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c19e530 con 0x7fbe8c101360 2023-10-08T09:14:44.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.584+0000 7fbe92001700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c1065c0 0x7fbe8c19dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c1065c0 0x7fbe8c19dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49394/0 (socket says 172.21.15.49:49394) 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 -- 172.21.15.49:0/2659245867 learned_addr learned my addr 172.21.15.49:0/2659245867 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe90fff700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe8c103c90 0x7fbe8c196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe91800700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe8c101360 0x7fbe8c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 -- 172.21.15.49:0/2659245867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe8c101360 msgr2=0x7fbe8c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe8c101360 0x7fbe8c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 -- 172.21.15.49:0/2659245867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe8c103c90 msgr2=0x7fbe8c196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe8c103c90 0x7fbe8c196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 -- 172.21.15.49:0/2659245867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880095f0 con 0x7fbe8c1065c0 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe91800700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe8c101360 0x7fbe8c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe92001700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c1065c0 0x7fbe8c19dc90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400add0 tx=0x7fbe8400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:44.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe827fc700 1 -- 172.21.15.49:0/2659245867 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400f070 con 0x7fbe8c1065c0 2023-10-08T09:14:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe827fc700 1 -- 172.21.15.49:0/2659245867 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe84007b20 con 0x7fbe8c1065c0 2023-10-08T09:14:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c19e810 con 0x7fbe8c1065c0 2023-10-08T09:14:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe827fc700 1 -- 172.21.15.49:0/2659245867 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84004290 con 0x7fbe8c1065c0 2023-10-08T09:14:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.585+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c19ecc0 con 0x7fbe8c1065c0 2023-10-08T09:14:44.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.586+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c0fc870 con 0x7fbe8c1065c0 2023-10-08T09:14:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.587+0000 7fbe827fc700 1 -- 172.21.15.49:0/2659245867 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbe84003a20 con 0x7fbe8c1065c0 2023-10-08T09:14:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.588+0000 7fbe827fc700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe780a1480 0x7fbe780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.588+0000 7fbe91800700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe780a1480 0x7fbe780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.588+0000 7fbe827fc700 1 -- 172.21.15.49:0/2659245867 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe840cc2b0 con 0x7fbe8c1065c0 2023-10-08T09:14:44.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.589+0000 7fbe91800700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe780a1480 0x7fbe780a3940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbe88006010 tx=0x7fbe880037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:44.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.593+0000 7fbe827fc700 1 -- 172.21.15.49:0/2659245867 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbe840955e0 con 0x7fbe8c1065c0 2023-10-08T09:14:44.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.839+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbe8c04e4c0 con 0x7fbe8c1065c0 2023-10-08T09:14:44.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.839+0000 7fbe827fc700 1 -- 172.21.15.49:0/2659245867 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2979 (secure 0 0 0) 0x7fbe84098c00 con 0x7fbe8c1065c0 2023-10-08T09:14:44.841 INFO:teuthology.orchestra.run.smithi049.stdout:e28 2023-10-08T09:14:44.841 INFO:teuthology.orchestra.run.smithi049.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-08T09:14:44.841 INFO:teuthology.orchestra.run.smithi049.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:14:44.841 INFO:teuthology.orchestra.run.smithi049.stdout:legacy client fscid: 1 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem 'cephfs' (1) 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:fs_name cephfs 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 28 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:created 2023-10-08T09:13:02.804952+0000 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2023-10-08T09:13:33.418895+0000 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:tableserver 0 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:root 0 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:session_timeout 300 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:session_autoclose 300 2023-10-08T09:14:44.842 INFO:teuthology.orchestra.run.smithi049.stdout:max_file_size 1099511627776 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:max_xattr_size 65536 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:required_client_features {} 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure 0 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:last_failure_osd_epoch 0 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:max_mds 5 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:in 0,1,2,3,4 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:up {0=24638,1=24616,2=24608,3=24553,4=24545} 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:failed 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:damaged 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:stopped 2023-10-08T09:14:44.843 INFO:teuthology.orchestra.run.smithi049.stdout:data_pools [3,4] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:metadata_pool 2 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:inline_data disabled 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:balancer 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:bal_rank_mask -1 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:standby_count_wanted 0 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b{0:24638} state up:active seq 3 addr [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c{0:24604} state up:standby-replay seq 1 addr [v2:172.21.15.184:6836/3605975147,v1:172.21.15.184:6837/3605975147] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.i{1:24616} state up:active seq 12 addr [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.f{1:24592} state up:standby-replay seq 1 addr [v2:172.21.15.184:6834/356478734,v1:172.21.15.184:6835/356478734] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.h{2:24608} state up:active seq 12 addr [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.e{2:24575} state up:standby-replay seq 1 addr [v2:172.21.15.132:6834/738661164,v1:172.21.15.132:6835/738661164] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.l{3:24553} state up:active seq 16 addr [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.844 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a{3:14733} state up:standby-replay seq 1 addr [v2:172.21.15.49:6840/1275166284,v1:172.21.15.49:6841/1275166284] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.k{4:24545} state up:active seq 16 addr [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.g{4:14721} state up:standby-replay seq 1 addr [v2:172.21.15.49:6838/3863085428,v1:172.21.15.49:6839/3863085428] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout:Standby daemons: 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.j{-1:14661} state up:standby seq 1 addr [v2:172.21.15.49:6834/3458420640,v1:172.21.15.49:6835/3458420640] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.845 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d{-1:14673} state up:standby seq 1 addr [v2:172.21.15.49:6836/1210414617,v1:172.21.15.49:6837/1210414617] compat {c=[1],r=[1],i=[fff]}] 2023-10-08T09:14:44.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe780a1480 msgr2=0x7fbe780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:44.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe780a1480 0x7fbe780a3940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbe88006010 tx=0x7fbe880037b0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c1065c0 msgr2=0x7fbe8c19dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:44.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c1065c0 0x7fbe8c19dc90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400add0 tx=0x7fbe8400c330 comp rx=0 tx=0).stop 2023-10-08T09:14:44.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 shutdown_connections 2023-10-08T09:14:44.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe780a1480 0x7fbe780a3940 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe8c101360 0x7fbe8c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c1065c0 0x7fbe8c19dc90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 --2- 172.21.15.49:0/2659245867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe8c103c90 0x7fbe8c196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 >> 172.21.15.49:0/2659245867 conn(0x7fbe8c0facf0 msgr2=0x7fbe8c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 shutdown_connections 2023-10-08T09:14:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:44.842+0000 7fbe93a64700 1 -- 172.21.15.49:0/2659245867 wait complete. 2023-10-08T09:14:44.847 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:14:45.314 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-08T09:14:45.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:45 smithi049 ceph-mon[99485]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 8 op/s 2023-10-08T09:14:45.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:45 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2659245867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T09:14:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:45 smithi184 ceph-mon[103675]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 8 op/s 2023-10-08T09:14:45.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:45 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2659245867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T09:14:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:45 smithi132 ceph-mon[103389]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 8 op/s 2023-10-08T09:14:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:45 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2659245867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-08T09:14:45.880 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:47.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1850508775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 msgr2=0x7f6a08100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1850508775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a08100d90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0099c0 tx=0x7f69fc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1850508775 shutdown_connections 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1850508775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a081012d0 0x7f6a08107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1850508775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a080ff730 0x7f6a080ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1850508775 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a08100d90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1850508775 >> 172.21.15.49:0/1850508775 conn(0x7f6a080facf0 msgr2=0x7f6a080fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1850508775 shutdown_connections 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1850508775 wait complete. 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 Processor -- start 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.240+0000 7f6a0e42a700 1 -- start start 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a0e42a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a080ff730 0x7f6a08068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a0e42a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a0819d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a0e42a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a081012d0 0x7f6a0819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a0e42a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081a0400 con 0x7f6a08100930 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a0e42a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081a0570 con 0x7f6a080ff730 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a0e42a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081a0730 con 0x7f6a081012d0 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a0819d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a0819d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49404/0 (socket says 172.21.15.49:49404) 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 -- 172.21.15.49:0/1245834159 learned_addr learned my addr 172.21.15.49:0/1245834159 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a0c9c7700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a081012d0 0x7f6a0819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a07fff700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a080ff730 0x7f6a08068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 -- 172.21.15.49:0/1245834159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a081012d0 msgr2=0x7f6a0819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a081012d0 0x7f6a0819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 -- 172.21.15.49:0/1245834159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a080ff730 msgr2=0x7f6a08068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a080ff730 0x7f6a08068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 -- 172.21.15.49:0/1245834159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc0095f0 con 0x7f6a08100930 2023-10-08T09:14:47.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.241+0000 7f6a077fe700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a0819d890 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0036f0 tx=0x7f69fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:47.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.242+0000 7f6a057fa700 1 -- 172.21.15.49:0/1245834159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc01d070 con 0x7f6a08100930 2023-10-08T09:14:47.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.242+0000 7f6a057fa700 1 -- 172.21.15.49:0/1245834159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69fc004090 con 0x7f6a08100930 2023-10-08T09:14:47.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.242+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a081a09b0 con 0x7f6a08100930 2023-10-08T09:14:47.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.242+0000 7f6a057fa700 1 -- 172.21.15.49:0/1245834159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc022d40 con 0x7f6a08100930 2023-10-08T09:14:47.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.242+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a081a0ea0 con 0x7f6a08100930 2023-10-08T09:14:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.243+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a08106c00 con 0x7f6a08100930 2023-10-08T09:14:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.244+0000 7f6a057fa700 1 -- 172.21.15.49:0/1245834159 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f69fc022470 con 0x7f6a08100930 2023-10-08T09:14:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.244+0000 7f6a057fa700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69f00a1500 0x7f69f00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.244+0000 7f6a07fff700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69f00a1500 0x7f69f00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.244+0000 7f6a057fa700 1 -- 172.21.15.49:0/1245834159 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f69fc0ce4b0 con 0x7f6a08100930 2023-10-08T09:14:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.245+0000 7f6a07fff700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69f00a1500 0x7f69f00a39c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005fd0 tx=0x7f69f8005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.248+0000 7f6a057fa700 1 -- 172.21.15.49:0/1245834159 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f69fc097870 con 0x7f6a08100930 2023-10-08T09:14:47.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.453+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f6a0810af40 con 0x7f6a08100930 2023-10-08T09:14:47.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.454+0000 7f6a057fa700 1 -- 172.21.15.49:0/1245834159 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) v1 ==== 56+0+5629 (secure 0 0 0) 0x7f69fc09ae90 con 0x7f6a08100930 2023-10-08T09:14:47.455 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 90 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:created 2023-10-08T09:05:34.760723+0000 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2023-10-08T09:14:38.423693+0000 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:crush_version 27 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:full_ratio 0.95 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:backfillfull_ratio 0.9 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:nearfull_ratio 0.85 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:require_min_compat_client luminous 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:min_compat_client luminous 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:require_osd_release reef 2023-10-08T09:14:47.456 INFO:teuthology.orchestra.run.smithi049.stdout:stretch_mode_enabled false 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:max_osd 12 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6802/3366428162,v1:172.21.15.49:6803/3366428162] [v2:172.21.15.49:6804/3366428162,v1:172.21.15.49:6805/3366428162] exists,up 78f3f71f-7b0b-4d29-997d-3bca2fd04fd3 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6800/2026160799,v1:172.21.15.132:6801/2026160799] [v2:172.21.15.132:6802/2026160799,v1:172.21.15.132:6803/2026160799] exists,up 7f3cdae9-b298-43b7-a0d0-703f6f7ab094 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6800/2213406776,v1:172.21.15.184:6801/2213406776] [v2:172.21.15.184:6802/2213406776,v1:172.21.15.184:6803/2213406776] exists,up 95cbb767-d840-4487-8b70-2f8381136ef7 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6810/3100042133,v1:172.21.15.49:6811/3100042133] [v2:172.21.15.49:6812/3100042133,v1:172.21.15.49:6813/3100042133] exists,up cc9fe096-1fd1-4de3-b21c-58a18d6cbec6 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 up in weight 1 up_from 32 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6808/2969628103,v1:172.21.15.132:6809/2969628103] [v2:172.21.15.132:6810/2969628103,v1:172.21.15.132:6811/2969628103] exists,up e852c5d9-cf88-4ddf-98e7-e22d644b4a30 2023-10-08T09:14:47.457 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6808/3484166138,v1:172.21.15.184:6809/3484166138] [v2:172.21.15.184:6810/3484166138,v1:172.21.15.184:6811/3484166138] exists,up 95803835-c3f8-469c-bee8-95249eccbe51 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 up in weight 1 up_from 44 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6818/3113806039,v1:172.21.15.49:6819/3113806039] [v2:172.21.15.49:6820/3113806039,v1:172.21.15.49:6821/3113806039] exists,up 811a7eaa-3df2-45a3-9cf3-b6a559db1546 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 up in weight 1 up_from 49 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6816/1427224102,v1:172.21.15.132:6817/1427224102] [v2:172.21.15.132:6818/1427224102,v1:172.21.15.132:6819/1427224102] exists,up d84fc995-e0a5-4655-831b-cacfceee4afa 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.8 up in weight 1 up_from 56 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6816/936308582,v1:172.21.15.184:6817/936308582] [v2:172.21.15.184:6818/936308582,v1:172.21.15.184:6819/936308582] exists,up ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.9 up in weight 1 up_from 64 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6826/2343500504,v1:172.21.15.49:6827/2343500504] [v2:172.21.15.49:6828/2343500504,v1:172.21.15.49:6829/2343500504] exists,up b1a0dc66-f039-4112-93d3-73f6f3a6869e 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.10 up in weight 1 up_from 69 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.132:6824/3556600534,v1:172.21.15.132:6825/3556600534] [v2:172.21.15.132:6826/3556600534,v1:172.21.15.132:6827/3556600534] exists,up 9f90e2ab-e7e8-42b3-966b-7c0d40a89925 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.11 up in weight 1 up_from 76 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.184:6824/613863047,v1:172.21.15.184:6825/613863047] [v2:172.21.15.184:6826/613863047,v1:172.21.15.184:6827/613863047] exists,up 4139ccc6-3973-496b-828c-069a9a0735fb 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.6 [8,11] 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 2.8 [2,11] 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.11 [4,8] 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.29 [3,9] 2023-10-08T09:14:47.458 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.33 [5,9] 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.34 [4,9] 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 3.3a [4,6] 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.b [9,8] 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.13 [9,10] 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.1b [9,8] 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:pg_upmap_items 4.1c [5,8] 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2700533077 expires 2023-10-09T09:06:35.771578+0000 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/656654162 expires 2023-10-09T09:06:35.771578+0000 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/4176953235 expires 2023-10-09T09:06:35.771578+0000 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/907336252 expires 2023-10-09T09:06:08.179468+0000 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/890731290 expires 2023-10-09T09:06:35.771578+0000 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1281967666 expires 2023-10-09T09:06:08.179468+0000 2023-10-08T09:14:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/1812466550 expires 2023-10-09T09:06:08.179468+0000 2023-10-08T09:14:47.460 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6801/907336252 expires 2023-10-09T09:06:08.179468+0000 2023-10-08T09:14:47.460 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:0/2211252163 expires 2023-10-09T09:06:08.179468+0000 2023-10-08T09:14:47.460 INFO:teuthology.orchestra.run.smithi049.stdout:blocklist 172.21.15.49:6800/890731290 expires 2023-10-09T09:06:35.771578+0000 2023-10-08T09:14:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69f00a1500 msgr2=0x7f69f00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.460 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69f00a1500 0x7f69f00a39c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f69f8005fd0 tx=0x7f69f8005dd0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 msgr2=0x7f6a0819d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a0819d890 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0036f0 tx=0x7f69fc0047b0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 shutdown_connections 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69f00a1500 0x7f69f00a39c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a081012d0 0x7f6a0819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6a080ff730 0x7f6a08068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 --2- 172.21.15.49:0/1245834159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6a08100930 0x7f6a0819d890 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 >> 172.21.15.49:0/1245834159 conn(0x7f6a080facf0 msgr2=0x7f6a08103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 shutdown_connections 2023-10-08T09:14:47.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.456+0000 7f6a0e42a700 1 -- 172.21.15.49:0/1245834159 wait complete. 2023-10-08T09:14:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:47 smithi184 ceph-mon[103675]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 726 B/s wr, 13 op/s 2023-10-08T09:14:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:47 smithi132 ceph-mon[103389]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 726 B/s wr, 13 op/s 2023-10-08T09:14:47.850 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:47 smithi049 ceph-mon[99485]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 726 B/s wr, 13 op/s 2023-10-08T09:14:47.851 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-08T09:14:47.862 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 -- 172.21.15.49:0/1455780085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a80ff5a0 msgr2=0x7f08a80ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 --2- 172.21.15.49:0/1455780085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a80ff5a0 0x7f08a80ff9c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f089c0099c0 tx=0x7f089c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 -- 172.21.15.49:0/1455780085 shutdown_connections 2023-10-08T09:14:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 --2- 172.21.15.49:0/1455780085 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8101130 0x7f08a81079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 --2- 172.21.15.49:0/1455780085 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a80ff5a0 0x7f08a80ff9c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 --2- 172.21.15.49:0/1455780085 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a8100790 0x7f08a8100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 -- 172.21.15.49:0/1455780085 >> 172.21.15.49:0/1455780085 conn(0x7f08a80fab10 msgr2=0x7f08a80fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:47.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 -- 172.21.15.49:0/1455780085 shutdown_connections 2023-10-08T09:14:47.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.991+0000 7f08ad17f700 1 -- 172.21.15.49:0/1455780085 wait complete. 2023-10-08T09:14:47.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 Processor -- start 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 -- start start 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a80ff5a0 0x7f08a8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8100790 0x7f08a8198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a8101130 0x7f08a819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a03b0 con 0x7f08a8101130 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a04f0 con 0x7f08a80ff5a0 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08ad17f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a0680 con 0x7f08a8100790 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08a759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a8101130 0x7f08a819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08a759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a8101130 0x7f08a819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49418/0 (socket says 172.21.15.49:49418) 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08a759e700 1 -- 172.21.15.49:0/2079836767 learned_addr learned my addr 172.21.15.49:0/2079836767 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.992+0000 7f08a6d9d700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a80ff5a0 0x7f08a8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08a659c700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8100790 0x7f08a8198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08a759e700 1 -- 172.21.15.49:0/2079836767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8100790 msgr2=0x7f08a8198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08a759e700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8100790 0x7f08a8198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08a759e700 1 -- 172.21.15.49:0/2079836767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a80ff5a0 msgr2=0x7f08a8198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08a759e700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a80ff5a0 0x7f08a8198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08a759e700 1 -- 172.21.15.49:0/2079836767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0095f0 con 0x7f08a8101130 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08a759e700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a8101130 0x7f08a819fde0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f08a0007ee0 tx=0x7f08a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f088ffff700 1 -- 172.21.15.49:0/2079836767 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a000ddd0 con 0x7f08a8101130 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a81a0960 con 0x7f08a8101130 2023-10-08T09:14:47.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f088ffff700 1 -- 172.21.15.49:0/2079836767 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08a000f040 con 0x7f08a8101130 2023-10-08T09:14:47.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f088ffff700 1 -- 172.21.15.49:0/2079836767 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00048e0 con 0x7f08a8101130 2023-10-08T09:14:47.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.993+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a81a0e70 con 0x7f08a8101130 2023-10-08T09:14:47.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.994+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a8067450 con 0x7f08a8101130 2023-10-08T09:14:47.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.996+0000 7f088ffff700 1 -- 172.21.15.49:0/2079836767 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f08a0003a00 con 0x7f08a8101130 2023-10-08T09:14:47.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.996+0000 7f088ffff700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f08900a1480 0x7f08900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:47.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.996+0000 7f088ffff700 1 -- 172.21.15.49:0/2079836767 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f08a00cd1e0 con 0x7f08a8101130 2023-10-08T09:14:47.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.996+0000 7f08a6d9d700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f08900a1480 0x7f08900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:47.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:47.997+0000 7f08a6d9d700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f08900a1480 0x7f08900a3940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f089c006010 tx=0x7f089c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:48.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.000+0000 7f088ffff700 1 -- 172.21.15.49:0/2079836767 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f08a0096510 con 0x7f08a8101130 2023-10-08T09:14:48.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.206+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08a810afd0 con 0x7f08a8101130 2023-10-08T09:14:48.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.207+0000 7f088ffff700 1 -- 172.21.15.49:0/2079836767 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f08a0023090 con 0x7f08a8101130 2023-10-08T09:14:48.209 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:48.209 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:14:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f08900a1480 msgr2=0x7f08900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f08900a1480 0x7f08900a3940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f089c006010 tx=0x7f089c000bc0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a8101130 msgr2=0x7f08a819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a8101130 0x7f08a819fde0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f08a0007ee0 tx=0x7f08a000d3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 shutdown_connections 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f08900a1480 0x7f08900a3940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f08a8101130 0x7f08a819fde0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08a80ff5a0 0x7f08a8198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 --2- 172.21.15.49:0/2079836767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08a8100790 0x7f08a8198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 >> 172.21.15.49:0/2079836767 conn(0x7f08a80fab10 msgr2=0x7f08a80678e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 shutdown_connections 2023-10-08T09:14:48.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.210+0000 7f08ad17f700 1 -- 172.21.15.49:0/2079836767 wait complete. 2023-10-08T09:14:48.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:14:48.225 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:14:48.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 -- 172.21.15.49:0/2570684926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 msgr2=0x7f9088100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 --2- 172.21.15.49:0/2570684926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088100cf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f90840099c0 tx=0x7f9084009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 -- 172.21.15.49:0/2570684926 shutdown_connections 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 --2- 172.21.15.49:0/2570684926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9088101230 0x7f9088107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 --2- 172.21.15.49:0/2570684926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90880ff6a0 0x7f90880ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 --2- 172.21.15.49:0/2570684926 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088100cf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 -- 172.21.15.49:0/2570684926 >> 172.21.15.49:0/2570684926 conn(0x7f90880fac10 msgr2=0x7f90880fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 -- 172.21.15.49:0/2570684926 shutdown_connections 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.353+0000 7f908ffd7700 1 -- 172.21.15.49:0/2570684926 wait complete. 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 Processor -- start 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 -- start start 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90880ff6a0 0x7f9088196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9088101230 0x7f908819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908819e280 con 0x7f9088100890 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908819e3c0 con 0x7f90880ff6a0 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908ffd7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908819e550 con 0x7f9088101230 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49430/0 (socket says 172.21.15.49:49430) 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 -- 172.21.15.49:0/905566060 learned_addr learned my addr 172.21.15.49:0/905566060 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908dd73700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90880ff6a0 0x7f9088196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908e574700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9088101230 0x7f908819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 -- 172.21.15.49:0/905566060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9088101230 msgr2=0x7f908819dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9088101230 0x7f908819dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 -- 172.21.15.49:0/905566060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90880ff6a0 msgr2=0x7f9088196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90880ff6a0 0x7f9088196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 -- 172.21.15.49:0/905566060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840095f0 con 0x7f9088100890 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.354+0000 7f908d572700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088196c90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9084009fd0 tx=0x7f9084005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.355+0000 7f907effd700 1 -- 172.21.15.49:0/905566060 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908401d070 con 0x7f9088100890 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.355+0000 7f907effd700 1 -- 172.21.15.49:0/905566060 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90840059a0 con 0x7f9088100890 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.355+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908819e7d0 con 0x7f9088100890 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.355+0000 7f907effd700 1 -- 172.21.15.49:0/905566060 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084017650 con 0x7f9088100890 2023-10-08T09:14:48.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.355+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908819ec80 con 0x7f9088100890 2023-10-08T09:14:48.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.356+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9088067530 con 0x7f9088100890 2023-10-08T09:14:48.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.357+0000 7f907effd700 1 -- 172.21.15.49:0/905566060 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9084022470 con 0x7f9088100890 2023-10-08T09:14:48.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.357+0000 7f907effd700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90740a1480 0x7f90740a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.357+0000 7f908dd73700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90740a1480 0x7f90740a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.357+0000 7f907effd700 1 -- 172.21.15.49:0/905566060 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f90840ceaf0 con 0x7f9088100890 2023-10-08T09:14:48.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.358+0000 7f908dd73700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90740a1480 0x7f90740a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f90881006f0 tx=0x7f9078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:48.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.361+0000 7f907effd700 1 -- 172.21.15.49:0/905566060 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9084097eb0 con 0x7f9088100890 2023-10-08T09:14:48.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.569+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f908810b0a0 con 0x7f9088100890 2023-10-08T09:14:48.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.569+0000 7f907effd700 1 -- 172.21.15.49:0/905566060 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f908400fe60 con 0x7f9088100890 2023-10-08T09:14:48.571 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:14:48.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90740a1480 msgr2=0x7f90740a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90740a1480 0x7f90740a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f90881006f0 tx=0x7f9078009260 comp rx=0 tx=0).stop 2023-10-08T09:14:48.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 msgr2=0x7f9088196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088196c90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9084009fd0 tx=0x7f9084005f10 comp rx=0 tx=0).stop 2023-10-08T09:14:48.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 shutdown_connections 2023-10-08T09:14:48.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90740a1480 0x7f90740a3940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9088101230 0x7f908819dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90880ff6a0 0x7f9088196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 --2- 172.21.15.49:0/905566060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9088100890 0x7f9088196c90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 >> 172.21.15.49:0/905566060 conn(0x7f90880fac10 msgr2=0x7f90880fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 shutdown_connections 2023-10-08T09:14:48.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.571+0000 7f908ffd7700 1 -- 172.21.15.49:0/905566060 wait complete. 2023-10-08T09:14:48.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:48 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1245834159' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T09:14:48.584 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:48 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2079836767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:48.585 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 -- 172.21.15.49:0/3429701439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc0fee20 msgr2=0x7fb7dc0ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/3429701439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc0fee20 0x7fb7dc0ff240 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009990 tx=0x7fb7d0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 -- 172.21.15.49:0/3429701439 shutdown_connections 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/3429701439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7dc100930 0x7fb7dc107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/3429701439 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc0fee20 0x7fb7dc0ff240 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/3429701439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7dc0fff70 0x7fb7dc1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 -- 172.21.15.49:0/3429701439 >> 172.21.15.49:0/3429701439 conn(0x7fb7dc0fa390 msgr2=0x7fb7dc0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:48.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 -- 172.21.15.49:0/3429701439 shutdown_connections 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.712+0000 7fb7e2f56700 1 -- 172.21.15.49:0/3429701439 wait complete. 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 Processor -- start 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 -- start start 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7dc0fee20 0x7fb7dc1086c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7dc0fff70 0x7fb7dc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc100930 0x7fb7dc19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19e1e0 con 0x7fb7dc100930 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19e350 con 0x7fb7dc0fff70 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e2f56700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc19e510 con 0x7fb7dc0fee20 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e14f3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc100930 0x7fb7dc19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e14f3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc100930 0x7fb7dc19d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49446/0 (socket says 172.21.15.49:49446) 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e14f3700 1 -- 172.21.15.49:0/1629250625 learned_addr learned my addr 172.21.15.49:0/1629250625 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:48.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7e0cf2700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7dc0fee20 0x7fb7dc1086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.713+0000 7fb7dbfff700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7dc0fff70 0x7fb7dc19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e14f3700 1 -- 172.21.15.49:0/1629250625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7dc0fee20 msgr2=0x7fb7dc1086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e14f3700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7dc0fee20 0x7fb7dc1086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e14f3700 1 -- 172.21.15.49:0/1629250625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7dc0fff70 msgr2=0x7fb7dc19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e14f3700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7dc0fff70 0x7fb7dc19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e14f3700 1 -- 172.21.15.49:0/1629250625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d00095f0 con 0x7fb7dc100930 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e14f3700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc100930 0x7fb7dc19d9f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4007d70 tx=0x7fb7d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7d9ffb700 1 -- 172.21.15.49:0/1629250625 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d400ac90 con 0x7fb7dc100930 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7d9ffb700 1 -- 172.21.15.49:0/1629250625 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d400adf0 con 0x7fb7dc100930 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc19e790 con 0x7fb7dc100930 2023-10-08T09:14:48.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7d9ffb700 1 -- 172.21.15.49:0/1629250625 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d40115e0 con 0x7fb7dc100930 2023-10-08T09:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.714+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc19ecf0 con 0x7fb7dc100930 2023-10-08T09:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.715+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7dc1061c0 con 0x7fb7dc100930 2023-10-08T09:14:48.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.716+0000 7fb7d9ffb700 1 -- 172.21.15.49:0/1629250625 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb7d4012420 con 0x7fb7dc100930 2023-10-08T09:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.717+0000 7fb7d9ffb700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7cc0a1500 0x7fb7cc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.717+0000 7fb7e0cf2700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7cc0a1500 0x7fb7cc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.717+0000 7fb7d9ffb700 1 -- 172.21.15.49:0/1629250625 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb7d40cd120 con 0x7fb7dc100930 2023-10-08T09:14:48.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.718+0000 7fb7e0cf2700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7cc0a1500 0x7fb7cc0a39c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b2e0 tx=0x7fb7d000b6f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:48.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.722+0000 7fb7d9ffb700 1 -- 172.21.15.49:0/1629250625 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb7d4096450 con 0x7fb7dc100930 2023-10-08T09:14:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:48 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1245834159' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T09:14:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:48 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2079836767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:48 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1245834159' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-08T09:14:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:48 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2079836767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:48.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.924+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7dc105ba0 con 0x7fb7dc100930 2023-10-08T09:14:48.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.924+0000 7fb7d9ffb700 1 -- 172.21.15.49:0/1629250625 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7d4099a70 con 0x7fb7dc100930 2023-10-08T09:14:48.926 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:48.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7cc0a1500 msgr2=0x7fb7cc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7cc0a1500 0x7fb7cc0a39c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b2e0 tx=0x7fb7d000b6f0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc100930 msgr2=0x7fb7dc19d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc100930 0x7fb7dc19d9f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4007d70 tx=0x7fb7d400a340 comp rx=0 tx=0).stop 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 shutdown_connections 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb7cc0a1500 0x7fb7cc0a39c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7dc0fee20 0x7fb7dc1086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb7dc100930 0x7fb7dc19d9f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 --2- 172.21.15.49:0/1629250625 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7dc0fff70 0x7fb7dc19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 >> 172.21.15.49:0/1629250625 conn(0x7fb7dc0fa390 msgr2=0x7fb7dc103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 shutdown_connections 2023-10-08T09:14:48.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:48.927+0000 7fb7e2f56700 1 -- 172.21.15.49:0/1629250625 wait complete. 2023-10-08T09:14:48.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:14:49.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.068+0000 7f83394ed700 1 -- 172.21.15.49:0/3253015365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83340ff6b0 msgr2=0x7f83340ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.068+0000 7f83394ed700 1 --2- 172.21.15.49:0/3253015365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83340ff6b0 0x7f83340ffad0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f83240099c0 tx=0x7f8324009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.068+0000 7f83394ed700 1 -- 172.21.15.49:0/3253015365 shutdown_connections 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.068+0000 7f83394ed700 1 --2- 172.21.15.49:0/3253015365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8334101240 0x7f8334107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.068+0000 7f83394ed700 1 --2- 172.21.15.49:0/3253015365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83340ff6b0 0x7f83340ffad0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.068+0000 7f83394ed700 1 --2- 172.21.15.49:0/3253015365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83341008a0 0x7f8334100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.068+0000 7f83394ed700 1 -- 172.21.15.49:0/3253015365 >> 172.21.15.49:0/3253015365 conn(0x7f83340fac00 msgr2=0x7f83340fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 -- 172.21.15.49:0/3253015365 shutdown_connections 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 -- 172.21.15.49:0/3253015365 wait complete. 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 Processor -- start 2023-10-08T09:14:49.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 -- start start 2023-10-08T09:14:49.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83340ff6b0 0x7f8334069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83341008a0 0x7f833419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334101240 0x7f833419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419e210 con 0x7f83341008a0 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.069+0000 7f83394ed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419e380 con 0x7f8334101240 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83394ed700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833419e540 con 0x7f83340ff6b0 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83341008a0 0x7f833419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f8332ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83340ff6b0 0x7f8334069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83341008a0 0x7f833419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49456/0 (socket says 172.21.15.49:49456) 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 -- 172.21.15.49:0/1161828715 learned_addr learned my addr 172.21.15.49:0/1161828715 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83337fe700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334101240 0x7f833419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 -- 172.21.15.49:0/1161828715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83340ff6b0 msgr2=0x7f8334069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83340ff6b0 0x7f8334069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 -- 172.21.15.49:0/1161828715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334101240 msgr2=0x7f833419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.074 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334101240 0x7f833419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 -- 172.21.15.49:0/1161828715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240095f0 con 0x7f83341008a0 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83327fc700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83341008a0 0x7f833419b480 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f832800e9a0 tx=0x7f832800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f831bfff700 1 -- 172.21.15.49:0/1161828715 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328009790 con 0x7f83341008a0 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833419e820 con 0x7f83341008a0 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f831bfff700 1 -- 172.21.15.49:0/1161828715 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f832800cbd0 con 0x7f83341008a0 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f831bfff700 1 -- 172.21.15.49:0/1161828715 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328018680 con 0x7f83341008a0 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.070+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833419ed30 con 0x7f83341008a0 2023-10-08T09:14:49.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.071+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8334106b80 con 0x7f83341008a0 2023-10-08T09:14:49.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.072+0000 7f831bfff700 1 -- 172.21.15.49:0/1161828715 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f832800cd40 con 0x7f83341008a0 2023-10-08T09:14:49.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.073+0000 7f831bfff700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f831c0a1480 0x7f831c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.073+0000 7f8332ffd700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f831c0a1480 0x7f831c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.073+0000 7f831bfff700 1 -- 172.21.15.49:0/1161828715 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8328014070 con 0x7f83341008a0 2023-10-08T09:14:49.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.074+0000 7f8332ffd700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f831c0a1480 0x7f831c0a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8324003730 tx=0x7f8324000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:49.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.077+0000 7f831bfff700 1 -- 172.21.15.49:0/1161828715 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f83280968b0 con 0x7f83341008a0 2023-10-08T09:14:49.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.279+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f833410afe0 con 0x7f83341008a0 2023-10-08T09:14:49.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.279+0000 7f831bfff700 1 -- 172.21.15.49:0/1161828715 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8328099ed0 con 0x7f83341008a0 2023-10-08T09:14:49.281 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:14:49.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f831c0a1480 msgr2=0x7f831c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f831c0a1480 0x7f831c0a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8324003730 tx=0x7f8324000bc0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83341008a0 msgr2=0x7f833419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83341008a0 0x7f833419b480 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f832800e9a0 tx=0x7f832800ecb0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 shutdown_connections 2023-10-08T09:14:49.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f831c0a1480 0x7f831c0a3940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83340ff6b0 0x7f8334069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8334101240 0x7f833419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 --2- 172.21.15.49:0/1161828715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f83341008a0 0x7f833419b480 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 >> 172.21.15.49:0/1161828715 conn(0x7f83340fac00 msgr2=0x7f8334103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:49.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 shutdown_connections 2023-10-08T09:14:49.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.281+0000 7f83394ed700 1 -- 172.21.15.49:0/1161828715 wait complete. 2023-10-08T09:14:49.296 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:14:49.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 -- 172.21.15.49:0/1975998538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 msgr2=0x7fd07c101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 --2- 172.21.15.49:0/1975998538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c101f50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd064009990 tx=0x7fd064009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 -- 172.21.15.49:0/1975998538 shutdown_connections 2023-10-08T09:14:49.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 --2- 172.21.15.49:0/1975998538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c0fea80 0x7fd07c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 --2- 172.21.15.49:0/1975998538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd07c102490 0x7fd07c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 --2- 172.21.15.49:0/1975998538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c101f50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 -- 172.21.15.49:0/1975998538 >> 172.21.15.49:0/1975998538 conn(0x7fd07c0fa3d0 msgr2=0x7fd07c0fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:49.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 -- 172.21.15.49:0/1975998538 shutdown_connections 2023-10-08T09:14:49.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 -- 172.21.15.49:0/1975998538 wait complete. 2023-10-08T09:14:49.428 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 Processor -- start 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.424+0000 7fd08177d700 1 -- start start 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd08177d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd07c0fea80 0x7fd07c196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd08177d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd08177d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c102490 0x7fd07c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd08177d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c19e270 con 0x7fd07c101af0 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd08177d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c19e3b0 con 0x7fd07c0fea80 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd08177d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c19e540 con 0x7fd07c102490 2023-10-08T09:14:49.429 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49470/0 (socket says 172.21.15.49:49470) 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 -- 172.21.15.49:0/1493765235 learned_addr learned my addr 172.21.15.49:0/1493765235 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd07b7fe700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c102490 0x7fd07c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd07affd700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd07c0fea80 0x7fd07c196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 -- 172.21.15.49:0/1493765235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c102490 msgr2=0x7fd07c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c102490 0x7fd07c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 -- 172.21.15.49:0/1493765235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd07c0fea80 msgr2=0x7fd07c196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd07c0fea80 0x7fd07c196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 -- 172.21.15.49:0/1493765235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640095f0 con 0x7fd07c101af0 2023-10-08T09:14:49.430 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.425+0000 7fd073fff700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c196c80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd064000c00 tx=0x7fd064004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:49.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.426+0000 7fd078ff9700 1 -- 172.21.15.49:0/1493765235 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06401d070 con 0x7fd07c101af0 2023-10-08T09:14:49.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.426+0000 7fd078ff9700 1 -- 172.21.15.49:0/1493765235 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0640054f0 con 0x7fd07c101af0 2023-10-08T09:14:49.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.426+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c19e7c0 con 0x7fd07c101af0 2023-10-08T09:14:49.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.426+0000 7fd078ff9700 1 -- 172.21.15.49:0/1493765235 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064004bd0 con 0x7fd07c101af0 2023-10-08T09:14:49.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.426+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c19eb80 con 0x7fd07c101af0 2023-10-08T09:14:49.431 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.427+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07c107dd0 con 0x7fd07c101af0 2023-10-08T09:14:49.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.428+0000 7fd078ff9700 1 -- 172.21.15.49:0/1493765235 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd06400fb90 con 0x7fd07c101af0 2023-10-08T09:14:49.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.428+0000 7fd078ff9700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0680a1480 0x7fd0680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.429+0000 7fd07affd700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0680a1480 0x7fd0680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.429+0000 7fd078ff9700 1 -- 172.21.15.49:0/1493765235 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd0640ce450 con 0x7fd07c101af0 2023-10-08T09:14:49.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.429+0000 7fd07affd700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0680a1480 0x7fd0680a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd07c068a70 tx=0x7fd06c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:49.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.432+0000 7fd078ff9700 1 -- 172.21.15.49:0/1493765235 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd0640977b0 con 0x7fd07c101af0 2023-10-08T09:14:49.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:49 smithi049 ceph-mon[99485]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2023-10-08T09:14:49.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:49 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/905566060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:49 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1629250625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:49 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1161828715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.637 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.636+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd07c10bfc0 con 0x7fd07c101af0 2023-10-08T09:14:49.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.637+0000 7fd078ff9700 1 -- 172.21.15.49:0/1493765235 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd06409add0 con 0x7fd07c101af0 2023-10-08T09:14:49.638 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:49.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.638+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0680a1480 msgr2=0x7fd0680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.638+0000 7fd08177d700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0680a1480 0x7fd0680a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd07c068a70 tx=0x7fd06c00b410 comp rx=0 tx=0).stop 2023-10-08T09:14:49.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.638+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 msgr2=0x7fd07c196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.638+0000 7fd08177d700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c196c80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd064000c00 tx=0x7fd064004730 comp rx=0 tx=0).stop 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.638+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 shutdown_connections 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.638+0000 7fd08177d700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0680a1480 0x7fd0680a3940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.638+0000 7fd08177d700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd07c102490 0x7fd07c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.639+0000 7fd08177d700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd07c0fea80 0x7fd07c196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.639+0000 7fd08177d700 1 --2- 172.21.15.49:0/1493765235 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd07c101af0 0x7fd07c196c80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.639+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 >> 172.21.15.49:0/1493765235 conn(0x7fd07c0fa3d0 msgr2=0x7fd07c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.639+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 shutdown_connections 2023-10-08T09:14:49.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.639+0000 7fd08177d700 1 -- 172.21.15.49:0/1493765235 wait complete. 2023-10-08T09:14:49.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:14:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:49 smithi184 ceph-mon[103675]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2023-10-08T09:14:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:49 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/905566060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:49 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1629250625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:49 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1161828715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 -- 172.21.15.49:0/2244244520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 msgr2=0x7fe8180ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 --2- 172.21.15.49:0/2244244520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8180ff9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe8080099c0 tx=0x7fe808009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 -- 172.21.15.49:0/2244244520 shutdown_connections 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 --2- 172.21.15.49:0/2244244520 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe818101130 0x7fe8181079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 --2- 172.21.15.49:0/2244244520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8180ff9c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 --2- 172.21.15.49:0/2244244520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe818100790 0x7fe818100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 -- 172.21.15.49:0/2244244520 >> 172.21.15.49:0/2244244520 conn(0x7fe8180fab10 msgr2=0x7fe8180fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 -- 172.21.15.49:0/2244244520 shutdown_connections 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.776+0000 7fe81df40700 1 -- 172.21.15.49:0/2244244520 wait complete. 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 Processor -- start 2023-10-08T09:14:49.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 -- start start 2023-10-08T09:14:49.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8181034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe818100790 0x7fe818101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe818101130 0x7fe8181020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe818102850 con 0x7fe8180ff5a0 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8181029c0 con 0x7fe818101130 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe81df40700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8181a0600 con 0x7fe818100790 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe8177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8181034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe8177fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8181034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49482/0 (socket says 172.21.15.49:49482) 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe8177fe700 1 -- 172.21.15.49:0/611942355 learned_addr learned my addr 172.21.15.49:0/611942355 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe816ffd700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe818100790 0x7fe818101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.777+0000 7fe817fff700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe818101130 0x7fe8181020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe8177fe700 1 -- 172.21.15.49:0/611942355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe818100790 msgr2=0x7fe818101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe8177fe700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe818100790 0x7fe818101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe8177fe700 1 -- 172.21.15.49:0/611942355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe818101130 msgr2=0x7fe8181020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe8177fe700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe818101130 0x7fe8181020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe8177fe700 1 -- 172.21.15.49:0/611942355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8080095f0 con 0x7fe8180ff5a0 2023-10-08T09:14:49.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe8177fe700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8181034e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe808003820 tx=0x7fe808004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:49.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe814ff9700 1 -- 172.21.15.49:0/611942355 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80801d070 con 0x7fe8180ff5a0 2023-10-08T09:14:49.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe814ff9700 1 -- 172.21.15.49:0/611942355 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8080054f0 con 0x7fe8180ff5a0 2023-10-08T09:14:49.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe814ff9700 1 -- 172.21.15.49:0/611942355 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808004a00 con 0x7fe8180ff5a0 2023-10-08T09:14:49.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8181a0830 con 0x7fe8180ff5a0 2023-10-08T09:14:49.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.778+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8181a0c30 con 0x7fe8180ff5a0 2023-10-08T09:14:49.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.779+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe818067450 con 0x7fe8180ff5a0 2023-10-08T09:14:49.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.780+0000 7fe814ff9700 1 -- 172.21.15.49:0/611942355 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe808003b50 con 0x7fe8180ff5a0 2023-10-08T09:14:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.781+0000 7fe814ff9700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe8040a1480 0x7fe8040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.781+0000 7fe816ffd700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe8040a1480 0x7fe8040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.781+0000 7fe814ff9700 1 -- 172.21.15.49:0/611942355 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe8080cde90 con 0x7fe8180ff5a0 2023-10-08T09:14:49.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.781+0000 7fe816ffd700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe8040a1480 0x7fe8040a3940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe80000ac50 tx=0x7fe80000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:49.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:49.785+0000 7fe814ff9700 1 -- 172.21.15.49:0/611942355 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe808097250 con 0x7fe8180ff5a0 2023-10-08T09:14:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:49 smithi132 ceph-mon[103389]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2023-10-08T09:14:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:49 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/905566060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:49 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1629250625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:49 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1161828715' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:50.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.033+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe81810ade0 con 0x7fe8180ff5a0 2023-10-08T09:14:50.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.034+0000 7fe814ff9700 1 -- 172.21.15.49:0/611942355 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7fe80809a870 con 0x7fe8180ff5a0 2023-10-08T09:14:50.035 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:50.035 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:14:50.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe8040a1480 msgr2=0x7fe8040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe8040a1480 0x7fe8040a3940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe80000ac50 tx=0x7fe80000a380 comp rx=0 tx=0).stop 2023-10-08T09:14:50.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 msgr2=0x7fe8181034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8181034e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe808003820 tx=0x7fe808004730 comp rx=0 tx=0).stop 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 shutdown_connections 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe8040a1480 0x7fe8040a3940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe818101130 0x7fe8181020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8180ff5a0 0x7fe8181034e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 --2- 172.21.15.49:0/611942355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe818100790 0x7fe818101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 >> 172.21.15.49:0/611942355 conn(0x7fe8180fab10 msgr2=0x7fe8180fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 shutdown_connections 2023-10-08T09:14:50.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.036+0000 7fe81df40700 1 -- 172.21.15.49:0/611942355 wait complete. 2023-10-08T09:14:50.039 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:14:50.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 -- 172.21.15.49:0/1016318222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c0ff5b0 msgr2=0x7fb56c0ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 --2- 172.21.15.49:0/1016318222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c0ff5b0 0x7fb56c0ff9d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0099c0 tx=0x7fb55c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 -- 172.21.15.49:0/1016318222 shutdown_connections 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 --2- 172.21.15.49:0/1016318222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb56c101140 0x7fb56c1079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 --2- 172.21.15.49:0/1016318222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c0ff5b0 0x7fb56c0ff9d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 --2- 172.21.15.49:0/1016318222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c1007a0 0x7fb56c100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 -- 172.21.15.49:0/1016318222 >> 172.21.15.49:0/1016318222 conn(0x7fb56c0fab40 msgr2=0x7fb56c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 -- 172.21.15.49:0/1016318222 shutdown_connections 2023-10-08T09:14:50.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.175+0000 7fb571093700 1 -- 172.21.15.49:0/1016318222 wait complete. 2023-10-08T09:14:50.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 Processor -- start 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 -- start start 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0ff5b0 0x7fb56c0695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c1007a0 0x7fb56c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb56c101140 0x7fb56c19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c19e1e0 con 0x7fb56c1007a0 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c19e350 con 0x7fb56c101140 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb571093700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c19e510 con 0x7fb56c0ff5b0 2023-10-08T09:14:50.180 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb56a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c1007a0 0x7fb56c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb56a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c1007a0 0x7fb56c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49498/0 (socket says 172.21.15.49:49498) 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb56a59c700 1 -- 172.21.15.49:0/1737242757 learned_addr learned my addr 172.21.15.49:0/1737242757 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb56ad9d700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0ff5b0 0x7fb56c0695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.176+0000 7fb56b59e700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb56c101140 0x7fb56c19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb56a59c700 1 -- 172.21.15.49:0/1737242757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0ff5b0 msgr2=0x7fb56c0695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb56a59c700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0ff5b0 0x7fb56c0695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb56a59c700 1 -- 172.21.15.49:0/1737242757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb56c101140 msgr2=0x7fb56c19d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb56a59c700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb56c101140 0x7fb56c19d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb56a59c700 1 -- 172.21.15.49:0/1737242757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c0095f0 con 0x7fb56c1007a0 2023-10-08T09:14:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb56a59c700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c1007a0 0x7fb56c19b3d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d950 tx=0x7fb55400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb563fff700 1 -- 172.21.15.49:0/1737242757 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55400f890 con 0x7fb56c1007a0 2023-10-08T09:14:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb563fff700 1 -- 172.21.15.49:0/1737242757 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb554010460 con 0x7fb56c1007a0 2023-10-08T09:14:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c19e790 con 0x7fb56c1007a0 2023-10-08T09:14:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb563fff700 1 -- 172.21.15.49:0/1737242757 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55400e610 con 0x7fb56c1007a0 2023-10-08T09:14:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.177+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c19eca0 con 0x7fb56c1007a0 2023-10-08T09:14:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.178+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c106a30 con 0x7fb56c1007a0 2023-10-08T09:14:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.179+0000 7fb563fff700 1 -- 172.21.15.49:0/1737242757 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb5540108d0 con 0x7fb56c1007a0 2023-10-08T09:14:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.180+0000 7fb563fff700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5580a1480 0x7fb5580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.180+0000 7fb563fff700 1 -- 172.21.15.49:0/1737242757 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb5540cce00 con 0x7fb56c1007a0 2023-10-08T09:14:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.180+0000 7fb56ad9d700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5580a1480 0x7fb5580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.180+0000 7fb56ad9d700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5580a1480 0x7fb5580a3940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006010 tx=0x7fb55c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.183+0000 7fb563fff700 1 -- 172.21.15.49:0/1737242757 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb554096130 con 0x7fb56c1007a0 2023-10-08T09:14:50.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.386+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb56c066bc0 con 0x7fb56c1007a0 2023-10-08T09:14:50.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.388+0000 7fb563fff700 1 -- 172.21.15.49:0/1737242757 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7fb554016090 con 0x7fb56c1007a0 2023-10-08T09:14:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:50.390 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5580a1480 msgr2=0x7fb5580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5580a1480 0x7fb5580a3940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb55c006010 tx=0x7fb55c00b540 comp rx=0 tx=0).stop 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c1007a0 msgr2=0x7fb56c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c1007a0 0x7fb56c19b3d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d950 tx=0x7fb55400dc60 comp rx=0 tx=0).stop 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 shutdown_connections 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb5580a1480 0x7fb5580a3940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0ff5b0 0x7fb56c0695d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb56c101140 0x7fb56c19d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 --2- 172.21.15.49:0/1737242757 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb56c1007a0 0x7fb56c19b3d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.392 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 >> 172.21.15.49:0/1737242757 conn(0x7fb56c0fab40 msgr2=0x7fb56c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:50.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 shutdown_connections 2023-10-08T09:14:50.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.390+0000 7fb571093700 1 -- 172.21.15.49:0/1737242757 wait complete. 2023-10-08T09:14:50.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:14:50.405 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:14:50.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 -- 172.21.15.49:0/1299047831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb01019b0 msgr2=0x7fbdb0101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 --2- 172.21.15.49:0/1299047831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb01019b0 0x7fbdb0101e10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbda40099c0 tx=0x7fbda4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 -- 172.21.15.49:0/1299047831 shutdown_connections 2023-10-08T09:14:50.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 --2- 172.21.15.49:0/1299047831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb0102350 0x7fbdb0108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 --2- 172.21.15.49:0/1299047831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdb01007c0 0x7fbdb0100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 --2- 172.21.15.49:0/1299047831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb01019b0 0x7fbdb0101e10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 -- 172.21.15.49:0/1299047831 >> 172.21.15.49:0/1299047831 conn(0x7fbdb00fbd30 msgr2=0x7fbdb00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:50.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 -- 172.21.15.49:0/1299047831 shutdown_connections 2023-10-08T09:14:50.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.533+0000 7fbdb5908700 1 -- 172.21.15.49:0/1299047831 wait complete. 2023-10-08T09:14:50.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 Processor -- start 2023-10-08T09:14:50.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 -- start start 2023-10-08T09:14:50.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb01007c0 0x7fbdb0196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdb01019b0 0x7fbdb0196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb0102350 0x7fbdb019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb019e270 con 0x7fbdb0102350 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb019e3b0 con 0x7fbdb01019b0 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdb5908700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb019e540 con 0x7fbdb01007c0 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdaf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb0102350 0x7fbdb019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdaeffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb01007c0 0x7fbdb0196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdaf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb0102350 0x7fbdb019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49514/0 (socket says 172.21.15.49:49514) 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdaf7fe700 1 -- 172.21.15.49:0/3436568299 learned_addr learned my addr 172.21.15.49:0/3436568299 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:50.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdaeffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb01007c0 0x7fbdb0196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:36544/0 (socket says 172.21.15.49:36544) 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.534+0000 7fbdae7fc700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdb01019b0 0x7fbdb0196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdaf7fe700 1 -- 172.21.15.49:0/3436568299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb01007c0 msgr2=0x7fbdb0196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdaf7fe700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb01007c0 0x7fbdb0196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdaf7fe700 1 -- 172.21.15.49:0/3436568299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdb01019b0 msgr2=0x7fbdb0196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdaf7fe700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdb01019b0 0x7fbdb0196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdaf7fe700 1 -- 172.21.15.49:0/3436568299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40095f0 con 0x7fbdb0102350 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdaf7fe700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb0102350 0x7fbdb019dca0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d980 tx=0x7fbda800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdb4906700 1 -- 172.21.15.49:0/3436568299 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8004a30 con 0x7fbdb0102350 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdb4906700 1 -- 172.21.15.49:0/3436568299 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbda80054e0 con 0x7fbdb0102350 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb019e820 con 0x7fbdb0102350 2023-10-08T09:14:50.540 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdb4906700 1 -- 172.21.15.49:0/3436568299 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda800fbc0 con 0x7fbdb0102350 2023-10-08T09:14:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.535+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb019ece0 con 0x7fbdb0102350 2023-10-08T09:14:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.536+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb0107bc0 con 0x7fbdb0102350 2023-10-08T09:14:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.537+0000 7fbdb4906700 1 -- 172.21.15.49:0/3436568299 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbda8004c20 con 0x7fbdb0102350 2023-10-08T09:14:50.541 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.538+0000 7fbdb4906700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd980a1480 0x7fbd980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.538+0000 7fbdaeffd700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd980a1480 0x7fbd980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.538+0000 7fbdb4906700 1 -- 172.21.15.49:0/3436568299 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbda80cd230 con 0x7fbdb0102350 2023-10-08T09:14:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.538+0000 7fbdaeffd700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd980a1480 0x7fbd980a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0101810 tx=0x7fbda000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:50.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.541+0000 7fbdb4906700 1 -- 172.21.15.49:0/3436568299 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbda8096500 con 0x7fbdb0102350 2023-10-08T09:14:50.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.742+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdb010c130 con 0x7fbdb0102350 2023-10-08T09:14:50.744 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.742+0000 7fbdb4906700 1 -- 172.21.15.49:0/3436568299 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbda80cae60 con 0x7fbdb0102350 2023-10-08T09:14:50.744 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:14:50.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd980a1480 msgr2=0x7fbd980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd980a1480 0x7fbd980a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0101810 tx=0x7fbda000b040 comp rx=0 tx=0).stop 2023-10-08T09:14:50.746 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb0102350 msgr2=0x7fbdb019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb0102350 0x7fbdb019dca0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbda800d980 tx=0x7fbda800dc90 comp rx=0 tx=0).stop 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 shutdown_connections 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd980a1480 0x7fbd980a3940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbdb01007c0 0x7fbdb0196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbdb0102350 0x7fbdb019dca0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 --2- 172.21.15.49:0/3436568299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdb01019b0 0x7fbdb0196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 >> 172.21.15.49:0/3436568299 conn(0x7fbdb00fbd30 msgr2=0x7fbdb0104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 shutdown_connections 2023-10-08T09:14:50.747 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.745+0000 7fbdb5908700 1 -- 172.21.15.49:0/3436568299 wait complete. 2023-10-08T09:14:50.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:50 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1493765235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:50.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:50 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/611942355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:50.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:50 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1737242757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:50.757 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:50 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1493765235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:50.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:50 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/611942355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:50.758 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:50 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1737242757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:50.759 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:14:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:50 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1493765235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:50 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/611942355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:50 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1737242757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:50.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2449288073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b101008d0 msgr2=0x7f3b10100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2449288073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b101008d0 0x7f3b10100d50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009990 tx=0x7f3b00009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2449288073 shutdown_connections 2023-10-08T09:14:50.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2449288073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b10101290 0x7f3b10107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2449288073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b100ffbc0 0x7f3b100fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2449288073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b101008d0 0x7f3b10100d50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2449288073 >> 172.21.15.49:0/2449288073 conn(0x7f3b100fa390 msgr2=0x7f3b100fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2449288073 shutdown_connections 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2449288073 wait complete. 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.886+0000 7f3b15ff6700 1 Processor -- start 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b15ff6700 1 -- start start 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b15ff6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b100ffbc0 0x7f3b100ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b15ff6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b101008d0 0x7f3b1019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b15ff6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b10101290 0x7f3b1019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b15ff6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1019e260 con 0x7f3b10101290 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b15ff6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1019e3d0 con 0x7f3b100ffbc0 2023-10-08T09:14:50.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b15ff6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1019e590 con 0x7f3b101008d0 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b10101290 0x7f3b1019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b10101290 0x7f3b1019da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49528/0 (socket says 172.21.15.49:49528) 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 -- 172.21.15.49:0/2129367144 learned_addr learned my addr 172.21.15.49:0/2129367144 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0f7fe700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b100ffbc0 0x7f3b100ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0effd700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b101008d0 0x7f3b1019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 -- 172.21.15.49:0/2129367144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b101008d0 msgr2=0x7f3b1019b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b101008d0 0x7f3b1019b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 -- 172.21.15.49:0/2129367144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b100ffbc0 msgr2=0x7f3b100ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b100ffbc0 0x7f3b100ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 -- 172.21.15.49:0/2129367144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000095f0 con 0x7f3b10101290 2023-10-08T09:14:50.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0f7fe700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b100ffbc0 0x7f3b100ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0effd700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b101008d0 0x7f3b1019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0ffff700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b10101290 0x7f3b1019da70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3b08007d70 tx=0x7f3b0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.887+0000 7f3b0cff9700 1 -- 172.21.15.49:0/2129367144 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800ad40 con 0x7f3b10101290 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.888+0000 7f3b0cff9700 1 -- 172.21.15.49:0/2129367144 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b080128e0 con 0x7f3b10101290 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.888+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1019e810 con 0x7f3b10101290 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.888+0000 7f3b0cff9700 1 -- 172.21.15.49:0/2129367144 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08011780 con 0x7f3b10101290 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.888+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1019ed20 con 0x7f3b10101290 2023-10-08T09:14:50.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.888+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b10106a90 con 0x7f3b10101290 2023-10-08T09:14:50.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.890+0000 7f3b0cff9700 1 -- 172.21.15.49:0/2129367144 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3b08012a50 con 0x7f3b10101290 2023-10-08T09:14:50.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.891+0000 7f3b0cff9700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3afc0a1480 0x7f3afc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:50.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.891+0000 7f3b0f7fe700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3afc0a1480 0x7f3afc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:50.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.891+0000 7f3b0cff9700 1 -- 172.21.15.49:0/2129367144 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3b080cd7a0 con 0x7f3b10101290 2023-10-08T09:14:50.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.891+0000 7f3b0f7fe700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3afc0a1480 0x7f3afc0a3940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b100673a0 tx=0x7f3af800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:50.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:50.894+0000 7f3b0cff9700 1 -- 172.21.15.49:0/2129367144 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3b08096a70 con 0x7f3b10101290 2023-10-08T09:14:51.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.097+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b1010b040 con 0x7f3b10101290 2023-10-08T09:14:51.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.098+0000 7f3b0cff9700 1 -- 172.21.15.49:0/2129367144 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b0809a090 con 0x7f3b10101290 2023-10-08T09:14:51.100 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:51.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3afc0a1480 msgr2=0x7f3afc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3afc0a1480 0x7f3afc0a3940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b100673a0 tx=0x7f3af800b670 comp rx=0 tx=0).stop 2023-10-08T09:14:51.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b10101290 msgr2=0x7f3b1019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b10101290 0x7f3b1019da70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3b08007d70 tx=0x7f3b0800a340 comp rx=0 tx=0).stop 2023-10-08T09:14:51.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 shutdown_connections 2023-10-08T09:14:51.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3afc0a1480 0x7f3afc0a3940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3b10101290 0x7f3b1019da70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b100ffbc0 0x7f3b100ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 --2- 172.21.15.49:0/2129367144 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b101008d0 0x7f3b1019b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 >> 172.21.15.49:0/2129367144 conn(0x7f3b100fa390 msgr2=0x7f3b10103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:51.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 shutdown_connections 2023-10-08T09:14:51.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.100+0000 7f3b15ff6700 1 -- 172.21.15.49:0/2129367144 wait complete. 2023-10-08T09:14:51.114 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:14:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 -- 172.21.15.49:0/1296307285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe240ff6a0 msgr2=0x7efe240ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 --2- 172.21.15.49:0/1296307285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe240ff6a0 0x7efe240ffac0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efe180099c0 tx=0x7efe18009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 -- 172.21.15.49:0/1296307285 shutdown_connections 2023-10-08T09:14:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 --2- 172.21.15.49:0/1296307285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe24101230 0x7efe24107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 --2- 172.21.15.49:0/1296307285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe240ff6a0 0x7efe240ffac0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 --2- 172.21.15.49:0/1296307285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe24100890 0x7efe24100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 -- 172.21.15.49:0/1296307285 >> 172.21.15.49:0/1296307285 conn(0x7efe240fac10 msgr2=0x7efe240fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 -- 172.21.15.49:0/1296307285 shutdown_connections 2023-10-08T09:14:51.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 -- 172.21.15.49:0/1296307285 wait complete. 2023-10-08T09:14:51.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.242+0000 7efe2af3b700 1 Processor -- start 2023-10-08T09:14:51.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe2af3b700 1 -- start start 2023-10-08T09:14:51.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe2af3b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe240ff6a0 0x7efe24196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe2af3b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe24100890 0x7efe24196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe2af3b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe24101230 0x7efe2419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe2af3b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419e250 con 0x7efe24100890 2023-10-08T09:14:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe2af3b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419e390 con 0x7efe24101230 2023-10-08T09:14:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe2af3b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2419e520 con 0x7efe240ff6a0 2023-10-08T09:14:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe24100890 0x7efe24196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe24100890 0x7efe24196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49540/0 (socket says 172.21.15.49:49540) 2023-10-08T09:14:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 -- 172.21.15.49:0/360174358 learned_addr learned my addr 172.21.15.49:0/360174358 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe28cd7700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe240ff6a0 0x7efe24196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe294d8700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe24101230 0x7efe2419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 -- 172.21.15.49:0/360174358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe240ff6a0 msgr2=0x7efe24196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe240ff6a0 0x7efe24196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 -- 172.21.15.49:0/360174358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe24101230 msgr2=0x7efe2419dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe24101230 0x7efe2419dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 -- 172.21.15.49:0/360174358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180095f0 con 0x7efe24100890 2023-10-08T09:14:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe28cd7700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe240ff6a0 0x7efe24196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe23fff700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe24100890 0x7efe24196c60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c9b0 tx=0x7efe1000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.243+0000 7efe21ffb700 1 -- 172.21.15.49:0/360174358 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10007b00 con 0x7efe24100890 2023-10-08T09:14:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.244+0000 7efe21ffb700 1 -- 172.21.15.49:0/360174358 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe10007c60 con 0x7efe24100890 2023-10-08T09:14:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.244+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2419e800 con 0x7efe24100890 2023-10-08T09:14:51.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.244+0000 7efe21ffb700 1 -- 172.21.15.49:0/360174358 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10018660 con 0x7efe24100890 2023-10-08T09:14:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.244+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2419ed60 con 0x7efe24100890 2023-10-08T09:14:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.245+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe24067530 con 0x7efe24100890 2023-10-08T09:14:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.246+0000 7efe21ffb700 1 -- 172.21.15.49:0/360174358 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efe10018880 con 0x7efe24100890 2023-10-08T09:14:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.246+0000 7efe21ffb700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efe140a1500 0x7efe140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.246+0000 7efe21ffb700 1 -- 172.21.15.49:0/360174358 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efe100cd020 con 0x7efe24100890 2023-10-08T09:14:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.246+0000 7efe28cd7700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efe140a1500 0x7efe140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.247+0000 7efe28cd7700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efe140a1500 0x7efe140a39c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b580 tx=0x7efe18005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:51.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.251+0000 7efe21ffb700 1 -- 172.21.15.49:0/360174358 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efe10096350 con 0x7efe24100890 2023-10-08T09:14:51.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.457+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe2419f080 con 0x7efe24100890 2023-10-08T09:14:51.459 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.458+0000 7efe21ffb700 1 -- 172.21.15.49:0/360174358 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe10099970 con 0x7efe24100890 2023-10-08T09:14:51.459 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:14:51.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efe140a1500 msgr2=0x7efe140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efe140a1500 0x7efe140a39c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b580 tx=0x7efe18005fb0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe24100890 msgr2=0x7efe24196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe24100890 0x7efe24196c60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efe1000c9b0 tx=0x7efe1000cd70 comp rx=0 tx=0).stop 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 shutdown_connections 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efe140a1500 0x7efe140a39c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe240ff6a0 0x7efe24196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efe24101230 0x7efe2419dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 --2- 172.21.15.49:0/360174358 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efe24100890 0x7efe24196c60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 >> 172.21.15.49:0/360174358 conn(0x7efe240fac10 msgr2=0x7efe24067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 shutdown_connections 2023-10-08T09:14:51.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.460+0000 7efe2af3b700 1 -- 172.21.15.49:0/360174358 wait complete. 2023-10-08T09:14:51.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:14:51.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 -- 172.21.15.49:0/1722291014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 msgr2=0x7f93d40ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 --2- 172.21.15.49:0/1722291014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d40ffac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b0d0 tx=0x7f93d000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 -- 172.21.15.49:0/1722291014 shutdown_connections 2023-10-08T09:14:51.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 --2- 172.21.15.49:0/1722291014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d4101230 0x7f93d4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 --2- 172.21.15.49:0/1722291014 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d40ffac0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 --2- 172.21.15.49:0/1722291014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d4100890 0x7f93d4100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 -- 172.21.15.49:0/1722291014 >> 172.21.15.49:0/1722291014 conn(0x7f93d40fac10 msgr2=0x7f93d40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.601+0000 7f93db28b700 1 -- 172.21.15.49:0/1722291014 shutdown_connections 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.602+0000 7f93db28b700 1 -- 172.21.15.49:0/1722291014 wait complete. 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.602+0000 7f93db28b700 1 Processor -- start 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.602+0000 7f93db28b700 1 -- start start 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.602+0000 7f93db28b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d4195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93db28b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d4100890 0x7f93d4195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93db28b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d4101230 0x7f93d419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93db28b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d419d370 con 0x7f93d40ff6a0 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93db28b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d419d4b0 con 0x7f93d4100890 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93db28b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d419d640 con 0x7f93d4101230 2023-10-08T09:14:51.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d4195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d4195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49552/0 (socket says 172.21.15.49:49552) 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 -- 172.21.15.49:0/1555231914 learned_addr learned my addr 172.21.15.49:0/1555231914 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9828700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d4101230 0x7f93d419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d8826700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d4100890 0x7f93d4195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 -- 172.21.15.49:0/1555231914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d4101230 msgr2=0x7f93d419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d4101230 0x7f93d419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 -- 172.21.15.49:0/1555231914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d4100890 msgr2=0x7f93d4195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d4100890 0x7f93d4195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 -- 172.21.15.49:0/1555231914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d0009d00 con 0x7f93d40ff6a0 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d8826700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d4100890 0x7f93d4195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93d9027700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d4195840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f93d0007a00 tx=0x7f93d00094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:51.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93ca7fc700 1 -- 172.21.15.49:0/1555231914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d000e040 con 0x7f93d40ff6a0 2023-10-08T09:14:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93ca7fc700 1 -- 172.21.15.49:0/1555231914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d001ad90 con 0x7f93d40ff6a0 2023-10-08T09:14:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93ca7fc700 1 -- 172.21.15.49:0/1555231914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0023390 con 0x7f93d40ff6a0 2023-10-08T09:14:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d4067830 con 0x7f93d40ff6a0 2023-10-08T09:14:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.603+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d4067d40 con 0x7f93d40ff6a0 2023-10-08T09:14:51.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.604+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d4067530 con 0x7f93d40ff6a0 2023-10-08T09:14:51.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.605+0000 7f93ca7fc700 1 -- 172.21.15.49:0/1555231914 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f93d001a8d0 con 0x7f93d40ff6a0 2023-10-08T09:14:51.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.606+0000 7f93ca7fc700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93c00a1500 0x7f93c00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.606+0000 7f93d8826700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93c00a1500 0x7f93c00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.606+0000 7f93ca7fc700 1 -- 172.21.15.49:0/1555231914 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f93d00d02d0 con 0x7f93d40ff6a0 2023-10-08T09:14:51.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.607+0000 7f93d8826700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93c00a1500 0x7f93c00a39c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f93c40099d0 tx=0x7f93c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:51.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.611+0000 7f93ca7fc700 1 -- 172.21.15.49:0/1555231914 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f93d0099730 con 0x7f93d40ff6a0 2023-10-08T09:14:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:51 smithi184 ceph-mon[103675]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 536 B/s wr, 12 op/s 2023-10-08T09:14:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:51 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3436568299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:51 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2129367144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:51 smithi132 ceph-mon[103389]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 536 B/s wr, 12 op/s 2023-10-08T09:14:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:51 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3436568299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:51 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2129367144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:51.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.812+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f93d410b0c0 con 0x7f93d40ff6a0 2023-10-08T09:14:51.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.813+0000 7f93ca7fc700 1 -- 172.21.15.49:0/1555231914 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f93d009cd50 con 0x7f93d40ff6a0 2023-10-08T09:14:51.814 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:51.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93c00a1500 msgr2=0x7f93c00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93c00a1500 0x7f93c00a39c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f93c40099d0 tx=0x7f93c4008040 comp rx=0 tx=0).stop 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 msgr2=0x7f93d4195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d4195840 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f93d0007a00 tx=0x7f93d00094c0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 shutdown_connections 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93c00a1500 0x7f93c00a39c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93d4101230 0x7f93d419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93d40ff6a0 0x7f93d4195840 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 --2- 172.21.15.49:0/1555231914 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93d4100890 0x7f93d4195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 >> 172.21.15.49:0/1555231914 conn(0x7f93d40fac10 msgr2=0x7f93d4103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 shutdown_connections 2023-10-08T09:14:51.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.815+0000 7f93db28b700 1 -- 172.21.15.49:0/1555231914 wait complete. 2023-10-08T09:14:51.828 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:51 smithi049 ceph-mon[99485]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 536 B/s wr, 12 op/s 2023-10-08T09:14:51.828 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:51 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3436568299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:51.828 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:51 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2129367144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:51.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:14:51.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 -- 172.21.15.49:0/3879143873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568101130 msgr2=0x7fe5681079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 --2- 172.21.15.49:0/3879143873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568101130 0x7fe5681079e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b0d0 tx=0x7fe56000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 -- 172.21.15.49:0/3879143873 shutdown_connections 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 --2- 172.21.15.49:0/3879143873 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5680ff5a0 0x7fe5680ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 --2- 172.21.15.49:0/3879143873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568101130 0x7fe5681079e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 --2- 172.21.15.49:0/3879143873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe568100790 0x7fe568100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 -- 172.21.15.49:0/3879143873 >> 172.21.15.49:0/3879143873 conn(0x7fe5680fab10 msgr2=0x7fe5680fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 -- 172.21.15.49:0/3879143873 shutdown_connections 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 -- 172.21.15.49:0/3879143873 wait complete. 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.955+0000 7fe56f482700 1 Processor -- start 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56f482700 1 -- start start 2023-10-08T09:14:51.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56f482700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5680ff5a0 0x7fe56810c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56f482700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568100790 0x7fe56810a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56f482700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe568101130 0x7fe56810af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56f482700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56810b6a0 con 0x7fe568100790 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56f482700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56810b810 con 0x7fe5680ff5a0 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56f482700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681a06c0 con 0x7fe568101130 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568100790 0x7fe56810a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568100790 0x7fe56810a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49568/0 (socket says 172.21.15.49:49568) 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 -- 172.21.15.49:0/2097302896 learned_addr learned my addr 172.21.15.49:0/2097302896 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56da1f700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe568101130 0x7fe56810af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56d21e700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5680ff5a0 0x7fe56810c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 -- 172.21.15.49:0/2097302896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe568101130 msgr2=0x7fe56810af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe568101130 0x7fe56810af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 -- 172.21.15.49:0/2097302896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5680ff5a0 msgr2=0x7fe56810c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5680ff5a0 0x7fe56810c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 -- 172.21.15.49:0/2097302896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560009d00 con 0x7fe568100790 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56da1f700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe568101130 0x7fe56810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56d21e700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5680ff5a0 0x7fe56810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe56ca1d700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568100790 0x7fe56810a9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe56400d950 tx=0x7fe56400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.956+0000 7fe55e7fc700 1 -- 172.21.15.49:0/2097302896 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564009ae0 con 0x7fe568100790 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.957+0000 7fe55e7fc700 1 -- 172.21.15.49:0/2097302896 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe564009c40 con 0x7fe568100790 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.957+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5681a0950 con 0x7fe568100790 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.957+0000 7fe55e7fc700 1 -- 172.21.15.49:0/2097302896 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56400f620 con 0x7fe568100790 2023-10-08T09:14:51.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.957+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5681a0e60 con 0x7fe568100790 2023-10-08T09:14:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.958+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5681068f0 con 0x7fe568100790 2023-10-08T09:14:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.959+0000 7fe55e7fc700 1 -- 172.21.15.49:0/2097302896 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe564010460 con 0x7fe568100790 2023-10-08T09:14:51.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.960+0000 7fe55e7fc700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe5540a1480 0x7fe5540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.960+0000 7fe56d21e700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe5540a1480 0x7fe5540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.960+0000 7fe55e7fc700 1 -- 172.21.15.49:0/2097302896 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe5640ccf80 con 0x7fe568100790 2023-10-08T09:14:51.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.960+0000 7fe56d21e700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe5540a1480 0x7fe5540a3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe558005fd0 tx=0x7fe558005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:51.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:51.963+0000 7fe55e7fc700 1 -- 172.21.15.49:0/2097302896 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe5640962b0 con 0x7fe568100790 2023-10-08T09:14:52.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.212+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe568067450 con 0x7fe568100790 2023-10-08T09:14:52.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.212+0000 7fe55e7fc700 1 -- 172.21.15.49:0/2097302896 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7fe564016090 con 0x7fe568100790 2023-10-08T09:14:52.214 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:52.214 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:14:52.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.214+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe5540a1480 msgr2=0x7fe5540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.214+0000 7fe56f482700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe5540a1480 0x7fe5540a3940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe558005fd0 tx=0x7fe558005e40 comp rx=0 tx=0).stop 2023-10-08T09:14:52.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.214+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568100790 msgr2=0x7fe56810a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.214+0000 7fe56f482700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568100790 0x7fe56810a9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe56400d950 tx=0x7fe56400dd10 comp rx=0 tx=0).stop 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 shutdown_connections 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe5540a1480 0x7fe5540a3940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe568101130 0x7fe56810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5680ff5a0 0x7fe56810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 --2- 172.21.15.49:0/2097302896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe568100790 0x7fe56810a9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 >> 172.21.15.49:0/2097302896 conn(0x7fe5680fab10 msgr2=0x7fe568103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 shutdown_connections 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.215+0000 7fe56f482700 1 -- 172.21.15.49:0/2097302896 wait complete. 2023-10-08T09:14:52.217 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:14:52.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.353+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/4246394746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 msgr2=0x7f2e28108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.353+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/4246394746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e28108b00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000b0d0 tx=0x7f2e2000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.353+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/4246394746 shutdown_connections 2023-10-08T09:14:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/4246394746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e28108b00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/4246394746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e281006c0 0x7f2e28100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/4246394746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e281018b0 0x7f2e28101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/4246394746 >> 172.21.15.49:0/4246394746 conn(0x7f2e280fbc30 msgr2=0x7f2e280fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/4246394746 shutdown_connections 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/4246394746 wait complete. 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 Processor -- start 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 -- start start 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e281006c0 0x7f2e28196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e281018b0 0x7f2e28196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e2819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2819e2a0 con 0x7f2e28102250 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2819e3e0 con 0x7f2e281018b0 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.354+0000 7f2e2ecfb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2819e570 con 0x7f2e281006c0 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e2819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e2819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49584/0 (socket says 172.21.15.49:49584) 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 -- 172.21.15.49:0/2568830679 learned_addr learned my addr 172.21.15.49:0/2568830679 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2ca97700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e281006c0 0x7f2e28196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e27fff700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e281018b0 0x7f2e28196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 -- 172.21.15.49:0/2568830679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e281006c0 msgr2=0x7f2e28196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e281006c0 0x7f2e28196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 -- 172.21.15.49:0/2568830679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e281018b0 msgr2=0x7f2e28196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e281018b0 0x7f2e28196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 -- 172.21.15.49:0/2568830679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20009d00 con 0x7f2e28102250 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e27fff700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e281018b0 0x7f2e28196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2d298700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e2819dcd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2e20007bd0 tx=0x7f2e200094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e25ffb700 1 -- 172.21.15.49:0/2568830679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000e040 con 0x7f2e28102250 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e25ffb700 1 -- 172.21.15.49:0/2568830679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e20012dc0 con 0x7f2e28102250 2023-10-08T09:14:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2819e7f0 con 0x7f2e28102250 2023-10-08T09:14:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e25ffb700 1 -- 172.21.15.49:0/2568830679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2001b450 con 0x7f2e28102250 2023-10-08T09:14:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.355+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2819ecc0 con 0x7f2e28102250 2023-10-08T09:14:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.356+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28107a90 con 0x7f2e28102250 2023-10-08T09:14:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.358+0000 7f2e25ffb700 1 -- 172.21.15.49:0/2568830679 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2e2001d610 con 0x7f2e28102250 2023-10-08T09:14:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.359+0000 7f2e25ffb700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e180a1500 0x7f2e180a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.359+0000 7f2e25ffb700 1 -- 172.21.15.49:0/2568830679 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2e200cf210 con 0x7f2e28102250 2023-10-08T09:14:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.359+0000 7f2e2ca97700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e180a1500 0x7f2e180a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.359+0000 7f2e2ca97700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e180a1500 0x7f2e180a39c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c005950 tx=0x7f2e1c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.362+0000 7f2e25ffb700 1 -- 172.21.15.49:0/2568830679 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2e20098570 con 0x7f2e28102250 2023-10-08T09:14:52.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.568+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e2810bf30 con 0x7f2e28102250 2023-10-08T09:14:52.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.570+0000 7f2e25ffb700 1 -- 172.21.15.49:0/2568830679 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f2e2009bb90 con 0x7f2e28102250 2023-10-08T09:14:52.571 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:52.572 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:14:52.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e180a1500 msgr2=0x7f2e180a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e180a1500 0x7f2e180a39c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c005950 tx=0x7f2e1c0058e0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 msgr2=0x7f2e2819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e2819dcd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2e20007bd0 tx=0x7f2e200094f0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 shutdown_connections 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e180a1500 0x7f2e180a39c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e281006c0 0x7f2e28196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e28102250 0x7f2e2819dcd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 --2- 172.21.15.49:0/2568830679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e281018b0 0x7f2e28196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 >> 172.21.15.49:0/2568830679 conn(0x7f2e280fbc30 msgr2=0x7f2e28104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 shutdown_connections 2023-10-08T09:14:52.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.572+0000 7f2e2ecfb700 1 -- 172.21.15.49:0/2568830679 wait complete. 2023-10-08T09:14:52.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:52 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/360174358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:52.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:52 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1555231914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:52.585 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:52 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2097302896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:52.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-08T09:13:02.804952+0000', 'modified': '2023-10-08T09:13:33.418895+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24638, 'mds_1': 24616, 'mds_2': 24608, 'mds_3': 24553, 'mds_4': 24545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14721': {'gid': 14721, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6839/3863085428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6838', 'nonce': 3863085428}, {'type': 'v1', 'addr': '172.21.15.49:6839', 'nonce': 3863085428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14733': {'gid': 14733, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.49:6841/1275166284', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6840', 'nonce': 1275166284}, {'type': 'v1', 'addr': '172.21.15.49:6841', 'nonce': 1275166284}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24545': {'gid': 24545, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.132:6833/3058040154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6832', 'nonce': 3058040154}, {'type': 'v1', 'addr': '172.21.15.132:6833', 'nonce': 3058040154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24553': {'gid': 24553, 'name': 'l', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.184:6833/3616311663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 3616311663}, {'type': 'v1', 'addr': '172.21.15.184:6833', 'nonce': 3616311663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24575': {'gid': 24575, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.132:6835/738661164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6834', 'nonce': 738661164}, {'type': 'v1', 'addr': '172.21.15.132:6835', 'nonce': 738661164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24592': {'gid': 24592, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6835/356478734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6834', 'nonce': 356478734}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 356478734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24604': {'gid': 24604, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.184:6837/3605975147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6836', 'nonce': 3605975147}, {'type': 'v1', 'addr': '172.21.15.184:6837', 'nonce': 3605975147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24608': {'gid': 24608, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.132:6837/2064856316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6836', 'nonce': 2064856316}, {'type': 'v1', 'addr': '172.21.15.132:6837', 'nonce': 2064856316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24616': {'gid': 24616, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.184:6839/2795080121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6838', 'nonce': 2795080121}, {'type': 'v1', 'addr': '172.21.15.184:6839', 'nonce': 2795080121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.132:6839/1629770277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.132:6838', 'nonce': 1629770277}, {'type': 'v1', 'addr': '172.21.15.132:6839', 'nonce': 1629770277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T09:14:52.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-08T09:14:52.588 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-10-08T09:14:52.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 -- 172.21.15.49:0/3115119351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 msgr2=0x7f0480107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 --2- 172.21.15.49:0/3115119351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f0480107b10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f047800b0d0 tx=0x7f047800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 -- 172.21.15.49:0/3115119351 shutdown_connections 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 --2- 172.21.15.49:0/3115119351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04800ff6d0 0x7f04800ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 --2- 172.21.15.49:0/3115119351 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f0480107b10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 --2- 172.21.15.49:0/3115119351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04801008c0 0x7f0480100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 -- 172.21.15.49:0/3115119351 >> 172.21.15.49:0/3115119351 conn(0x7f04800fac40 msgr2=0x7f04800fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 -- 172.21.15.49:0/3115119351 shutdown_connections 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 -- 172.21.15.49:0/3115119351 wait complete. 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.714+0000 7f04872d7700 1 Processor -- start 2023-10-08T09:14:52.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f04872d7700 1 -- start start 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f04872d7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04800ff6d0 0x7f0480198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f04872d7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04801008c0 0x7f0480198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f04872d7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f048019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f04872d7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04801a04e0 con 0x7f0480101260 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f04872d7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04801a0620 con 0x7f04800ff6d0 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f04872d7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04801a07b0 con 0x7f04801008c0 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f048019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f048019ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49598/0 (socket says 172.21.15.49:49598) 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 -- 172.21.15.49:0/3488468763 learned_addr learned my addr 172.21.15.49:0/3488468763 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485073700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04800ff6d0 0x7f0480198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0484872700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04801008c0 0x7f0480198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 -- 172.21.15.49:0/3488468763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04801008c0 msgr2=0x7f0480198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04801008c0 0x7f0480198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 -- 172.21.15.49:0/3488468763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04800ff6d0 msgr2=0x7f0480198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04800ff6d0 0x7f0480198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 -- 172.21.15.49:0/3488468763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0478009d00 con 0x7f0480101260 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.715+0000 7f0485874700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f048019ff10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0478007bd0 tx=0x7f04780094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.716+0000 7f04767fc700 1 -- 172.21.15.49:0/3488468763 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047800e040 con 0x7f0480101260 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.716+0000 7f04767fc700 1 -- 172.21.15.49:0/3488468763 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0478004e60 con 0x7f0480101260 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.716+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04801a0a30 con 0x7f0480101260 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.716+0000 7f04767fc700 1 -- 172.21.15.49:0/3488468763 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478012a60 con 0x7f0480101260 2023-10-08T09:14:52.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.716+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04801a0e50 con 0x7f0480101260 2023-10-08T09:14:52.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.718+0000 7f04767fc700 1 -- 172.21.15.49:0/3488468763 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f04780056f0 con 0x7f0480101260 2023-10-08T09:14:52.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.718+0000 7f04767fc700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f046c0a1480 0x7f046c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.719+0000 7f04767fc700 1 -- 172.21.15.49:0/3488468763 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f04780ce670 con 0x7f0480101260 2023-10-08T09:14:52.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.719+0000 7f0485073700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f046c0a1480 0x7f046c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.719+0000 7f0485073700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f046c0a1480 0x7f046c0a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0480100720 tx=0x7f047000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 Processor -- start 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 -- start start 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1010 0x55df43bc1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bc2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43bc2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43b29410 con 0x55df43bc1970 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43b1abd0 con 0x55df43bc1010 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f04934c0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43b184f0 con 0x55df43bc27b0 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bc2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bc2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49604/0 (socket says 172.21.15.49:49604) 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 -- 172.21.15.49:0/1643514825 learned_addr learned my addr 172.21.15.49:0/1643514825 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0485073700 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1010 0x55df43bc1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0485874700 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43bc2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 -- 172.21.15.49:0/1643514825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 msgr2=0x55df43bc2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43bc2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 -- 172.21.15.49:0/1643514825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1010 msgr2=0x55df43bc1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1010 0x55df43bc1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.720+0000 7f0484872700 1 -- 172.21.15.49:0/1643514825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df4393ed90 con 0x55df43bc1970 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f0484872700 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bc2220 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f047c008c60 tx=0x7f047c008f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c4589a5c1a691a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f046b7fe700 1 -- 172.21.15.49:0/1643514825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ed70 con 0x55df43bc1970 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f046b7fe700 1 -- 172.21.15.49:0/1643514825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f047c00eed0 con 0x55df43bc1970 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f046b7fe700 1 -- 172.21.15.49:0/1643514825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c010690 con 0x55df43bc1970 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f04934c0b80 1 -- 172.21.15.49:0/1643514825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 msgr2=0x55df43bc2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f04934c0b80 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bc2220 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f047c008c60 tx=0x7f047c008f70 comp rx=0 tx=0).stop 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f04934c0b80 1 -- 172.21.15.49:0/1643514825 shutdown_connections 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f04934c0b80 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43bc2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f04934c0b80 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1010 0x55df43bc1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f04934c0b80 1 --2- 172.21.15.49:0/1643514825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bc2220 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.721+0000 7f04934c0b80 1 -- 172.21.15.49:0/1643514825 >> 172.21.15.49:0/1643514825 conn(0x55df43b35210 msgr2=0x55df43bbbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 -- 172.21.15.49:0/1643514825 shutdown_connections 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 -- 172.21.15.49:0/1643514825 wait complete. 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 -- start start 2023-10-08T09:14:52.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 Processor -- start 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bcae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bcb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc27b0 0x55df43bd2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43b13de0 con 0x55df43bc1970 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43ab06c0 con 0x55df43bc27b0 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43ab0360 con 0x55df43bc1010 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.722+0000 7f04934c0b80 10 client.0 _ll_register_callbacks cb 0x55df4390d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bcb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bcb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49610/0 (socket says 172.21.15.49:49610) 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 -- 172.21.15.49:0/639873123 learned_addr learned my addr 172.21.15.49:0/639873123 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0485874700 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc27b0 0x55df43bd2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0485073700 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bcae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 -- 172.21.15.49:0/639873123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 msgr2=0x55df43bcae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bcae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 -- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc27b0 msgr2=0x55df43bd2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc27b0 0x55df43bd2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 -- 172.21.15.49:0/639873123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c008890 con 0x55df43bc1970 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0484872700 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bcb350 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f047c008c30 tx=0x7f047c004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d71f7b174b21fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0468ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:52.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0468ff9700 1 -- 172.21.15.49:0/639873123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c004440 con 0x55df43bc1970 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0468ff9700 1 -- 172.21.15.49:0/639873123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f047c008230 con 0x55df43bc1970 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df43adbe60 con 0x55df43bc1970 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f0468ff9700 1 -- 172.21.15.49:0/639873123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f047c0045a0 con 0x55df43bc1970 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f04934c0b80 10 client.15537 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.723+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55df43927c10 con 0x55df43bc1970 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f0468ff9700 1 -- 172.21.15.49:0/639873123 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df43adbe60 con 0x55df43bc1970 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f0468ff9700 1 -- 172.21.15.49:0/639873123 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f047c003c20 con 0x55df43bc1970 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f04934c0b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f04934c0b80 10 client.15537 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f04934c0b80 20 client.15537 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f04934c0b80 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43bd68f0 0x55df43bd5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f04934c0b80 4 client.15537 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 --> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df43b0cdc0 con 0x55df43bd68f0 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.724+0000 7f0485073700 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43bd68f0 0x55df43bd5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f0468ff9700 10 client.15537 ms_handle_connect on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f0485073700 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43bd68f0 0x55df43bd5420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2020b5a60690962 server_cookie=b645595d2cc975af in_seq=0 out_seq=0 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f0468ff9700 1 -- 172.21.15.49:0/639873123 <== mds.4 v2:172.21.15.132:6832/3058040154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df43b0cdc0 con 0x55df43bd68f0 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f0468ff9700 10 client.15537 handle_command_reply: tid=0 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 client.15537 shutdown 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 msgr2=0x55df43bcb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bcb350 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f047c008c30 tx=0x7f047c004980 comp rx=0 tx=0).stop 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 shutdown_connections 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bcae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43bd68f0 0x55df43bd5420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc27b0 0x55df43bd2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f0468ff9700 0 client.15537 ms_handle_reset on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:52.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 --2- 172.21.15.49:0/639873123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1970 0x55df43bcb350 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.725+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 >> 172.21.15.49:0/639873123 conn(0x55df43b35210 msgr2=0x55df43bbb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.726+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 shutdown_connections 2023-10-08T09:14:52.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.726+0000 7f04934c0b80 1 -- 172.21.15.49:0/639873123 wait complete. 2023-10-08T09:14:52.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.726+0000 7f04934c0b80 20 client.15537 trim_cache size 0 max 16384 2023-10-08T09:14:52.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:52 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/360174358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:52.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:52 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1555231914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:52.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:52 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2097302896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.743+0000 7f04934c0b80 1 Processor -- start 2023-10-08T09:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.743+0000 7f04934c0b80 1 -- start start 2023-10-08T09:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.743+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bb7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.743+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bd2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc27b0 0x55df43bca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.751 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f04934c0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df439948d0 con 0x55df43bc27b0 2023-10-08T09:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f04934c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43aba4d0 con 0x55df43bc1970 2023-10-08T09:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f04934c0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43ad2590 con 0x55df43bc1010 2023-10-08T09:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc27b0 0x55df43bca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc27b0 0x55df43bca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49620/0 (socket says 172.21.15.49:49620) 2023-10-08T09:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 -- 172.21.15.49:0/3725293990 learned_addr learned my addr 172.21.15.49:0/3725293990 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.752 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485073700 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bb7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0484872700 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bd2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 -- 172.21.15.49:0/3725293990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 msgr2=0x55df43bb7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bb7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 -- 172.21.15.49:0/3725293990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 msgr2=0x55df43bd2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.753 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bd2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 -- 172.21.15.49:0/3725293990 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df43965200 con 0x55df43bc27b0 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f0485874700 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc27b0 0x55df43bca320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0478005a40 tx=0x7f04780bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ac06826b11605a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f046bfff700 1 -- 172.21.15.49:0/3725293990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047809bd90 con 0x55df43bc27b0 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f046bfff700 1 -- 172.21.15.49:0/3725293990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f047809b690 con 0x55df43bc27b0 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.744+0000 7f046bfff700 1 -- 172.21.15.49:0/3725293990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478099b10 con 0x55df43bc27b0 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 -- 172.21.15.49:0/3725293990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc27b0 msgr2=0x55df43bca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc27b0 0x55df43bca320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0478005a40 tx=0x7f04780bb040 comp rx=0 tx=0).stop 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 -- 172.21.15.49:0/3725293990 shutdown_connections 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc1010 0x55df43bb7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc27b0 0x55df43bca320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 --2- 172.21.15.49:0/3725293990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bd2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 -- 172.21.15.49:0/3725293990 >> 172.21.15.49:0/3725293990 conn(0x55df43b35210 msgr2=0x55df43bbbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 -- 172.21.15.49:0/3725293990 shutdown_connections 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 -- 172.21.15.49:0/3725293990 wait complete. 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 -- start start 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.745+0000 7f04934c0b80 1 Processor -- start 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1010 0x55df43bcef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bcf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43c2a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43bd2420 con 0x55df43bc1010 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43bd2590 con 0x55df43bc1970 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df43bd2750 con 0x55df43bc27b0 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 10 client.0 _ll_register_callbacks cb 0x7f047000a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:52.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1010 0x55df43bcef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1010 0x55df43bcef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49622/0 (socket says 172.21.15.49:49622) 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 -- 172.21.15.49:0/110679067 learned_addr learned my addr 172.21.15.49:0/110679067 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485874700 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43c2a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0484872700 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bcf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 -- 172.21.15.49:0/110679067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 msgr2=0x55df43c2a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43c2a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 -- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 msgr2=0x55df43bcf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bcf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 -- 172.21.15.49:0/110679067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04801a0e50 con 0x55df43bc1010 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0485073700 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1010 0x55df43bcef10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f047000fa30 tx=0x7f0470012070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7ed9342fd1690e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0468ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:52.756 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0468ff9700 1 -- 172.21.15.49:0/110679067 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470022040 con 0x55df43bc1010 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f0468ff9700 1 -- 172.21.15.49:0/110679067 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0470010da0 con 0x55df43bc1010 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.746+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f047c00eed0 con 0x55df43bc1010 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f0468ff9700 1 -- 172.21.15.49:0/110679067 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0470027900 con 0x55df43bc1010 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 10 client.15543 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55df4393d260 con 0x55df43bc1010 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f0468ff9700 1 -- 172.21.15.49:0/110679067 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f047c00eed0 con 0x55df43bc1010 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f0468ff9700 1 -- 172.21.15.49:0/110679067 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f047002d020 con 0x55df43bc1010 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 10 client.15543 resolve_mds: resolved name 'k' to daemon mds.k 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 20 client.15543 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43c2faa0 0x55df43c2bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 4 client.15543 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 --> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55df43b0cae0 con 0x55df43c2faa0 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.747+0000 7f0484872700 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43c2faa0 0x55df43c2bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f0484872700 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43c2faa0 0x55df43c2bf40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7af7145f7ada887b server_cookie=4e1395ba974ad9b in_seq=0 out_seq=0 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f0468ff9700 10 client.15543 ms_handle_connect on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f0468ff9700 1 -- 172.21.15.49:0/110679067 <== mds.4 v2:172.21.15.132:6832/3058040154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55df43b0cae0 con 0x55df43c2faa0 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f0468ff9700 10 client.15543 handle_command_reply: tid=0 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 client.15543 shutdown 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1010 msgr2=0x55df43bcef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1010 0x55df43bcef10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f047000fa30 tx=0x7f0470012070 comp rx=0 tx=0).stop 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 shutdown_connections 2023-10-08T09:14:52.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df43bc27b0 0x55df43c2a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x55df43c2faa0 0x55df43c2bf40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df43bc1010 0x55df43bcef10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f0468ff9700 0 client.15543 ms_handle_reset on v2:172.21.15.132:6832/3058040154 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 --2- 172.21.15.49:0/110679067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df43bc1970 0x55df43bcf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 >> 172.21.15.49:0/110679067 conn(0x55df43b35210 msgr2=0x55df43bbbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 shutdown_connections 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 1 -- 172.21.15.49:0/110679067 wait complete. 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.748+0000 7f04934c0b80 20 client.15543 trim_cache size 0 max 16384 2023-10-08T09:14:52.759 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 4, 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24545, 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 80.336524432999994, 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 90, 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:52.760 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 138.84061977799999 2023-10-08T09:14:52.761 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:52.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f046c0a1480 msgr2=0x7f046c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f046c0a1480 0x7f046c0a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0480100720 tx=0x7f047000a380 comp rx=0 tx=0).stop 2023-10-08T09:14:52.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 msgr2=0x7f048019ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f048019ff10 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0478007bd0 tx=0x7f04780094c0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 shutdown_connections 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f046c0a1480 0x7f046c0a3940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0480101260 0x7f048019ff10 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04800ff6d0 0x7f0480198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 --2- 172.21.15.49:0/3488468763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04801008c0 0x7f0480198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 >> 172.21.15.49:0/3488468763 conn(0x7f04800fac40 msgr2=0x7f0480108460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 shutdown_connections 2023-10-08T09:14:52.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.749+0000 7f04872d7700 1 -- 172.21.15.49:0/3488468763 wait complete. 2023-10-08T09:14:52.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2023-10-08T09:14:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:52 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/360174358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:52 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1555231914' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:14:52.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:52 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2097302896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:14:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.888+0000 7efd291f1700 1 -- 172.21.15.49:0/1551755863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 msgr2=0x7efd240ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.888+0000 7efd291f1700 1 --2- 172.21.15.49:0/1551755863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd240ffac0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efd180099c0 tx=0x7efd18009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 -- 172.21.15.49:0/1551755863 shutdown_connections 2023-10-08T09:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 --2- 172.21.15.49:0/1551755863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd24101230 0x7efd24107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 --2- 172.21.15.49:0/1551755863 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd240ffac0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 --2- 172.21.15.49:0/1551755863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd24100890 0x7efd24100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 -- 172.21.15.49:0/1551755863 >> 172.21.15.49:0/1551755863 conn(0x7efd240fac10 msgr2=0x7efd240fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 -- 172.21.15.49:0/1551755863 shutdown_connections 2023-10-08T09:14:52.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 -- 172.21.15.49:0/1551755863 wait complete. 2023-10-08T09:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.889+0000 7efd291f1700 1 Processor -- start 2023-10-08T09:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd291f1700 1 -- start start 2023-10-08T09:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd291f1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd24196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd291f1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd24100890 0x7efd24196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd291f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd24101230 0x7efd2419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd291f1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2419e270 con 0x7efd240ff6a0 2023-10-08T09:14:52.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd291f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2419e3b0 con 0x7efd24101230 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd291f1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2419e540 con 0x7efd24100890 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd24196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd24196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49636/0 (socket says 172.21.15.49:49636) 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd22d9d700 1 -- 172.21.15.49:0/1739258209 learned_addr learned my addr 172.21.15.49:0/1739258209 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd2259c700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd24100890 0x7efd24196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd2359e700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd24101230 0x7efd2419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd22d9d700 1 -- 172.21.15.49:0/1739258209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd24100890 msgr2=0x7efd24196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.890+0000 7efd22d9d700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd24100890 0x7efd24196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd22d9d700 1 -- 172.21.15.49:0/1739258209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd24101230 msgr2=0x7efd2419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd22d9d700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd24101230 0x7efd2419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd22d9d700 1 -- 172.21.15.49:0/1739258209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd180095f0 con 0x7efd240ff6a0 2023-10-08T09:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd22d9d700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd24196740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efd18003820 tx=0x7efd18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd0bfff700 1 -- 172.21.15.49:0/1739258209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1801d070 con 0x7efd240ff6a0 2023-10-08T09:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2419e7c0 con 0x7efd240ff6a0 2023-10-08T09:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd0bfff700 1 -- 172.21.15.49:0/1739258209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd18004ca0 con 0x7efd240ff6a0 2023-10-08T09:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd0bfff700 1 -- 172.21.15.49:0/1739258209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18003d30 con 0x7efd240ff6a0 2023-10-08T09:14:52.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.891+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2419ec70 con 0x7efd240ff6a0 2023-10-08T09:14:52.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.893+0000 7efd0bfff700 1 -- 172.21.15.49:0/1739258209 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efd18005030 con 0x7efd240ff6a0 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.894+0000 7efd0bfff700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd0c0a1480 0x7efd0c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.894+0000 7efd2259c700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd0c0a1480 0x7efd0c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.894+0000 7efd0bfff700 1 -- 172.21.15.49:0/1739258209 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efd180cde60 con 0x7efd240ff6a0 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.894+0000 7efd2259c700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd0c0a1480 0x7efd0c0a3940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efd14005950 tx=0x7efd1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.894+0000 7efd353dab80 1 Processor -- start 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.894+0000 7efd353dab80 1 -- start start 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd353dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f488430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd353dab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f489220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd353dab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f4897b0 0x56259f489c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd353dab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f3f0410 con 0x56259f4897b0 2023-10-08T09:14:52.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd353dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f3e1bd0 con 0x56259f488010 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd353dab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f3df4f0 con 0x56259f488970 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f4897b0 0x56259f489c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f4897b0 0x56259f489c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49648/0 (socket says 172.21.15.49:49648) 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 -- 172.21.15.49:0/618274247 learned_addr learned my addr 172.21.15.49:0/618274247 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd22d9d700 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f488430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2259c700 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f489220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 -- 172.21.15.49:0/618274247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 msgr2=0x56259f489220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f489220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 -- 172.21.15.49:0/618274247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 msgr2=0x56259f488430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f488430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 -- 172.21.15.49:0/618274247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56259f205d90 con 0x56259f4897b0 2023-10-08T09:14:52.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd22d9d700 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f488430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd2359e700 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f4897b0 0x56259f489c40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0098f0 tx=0x7efd1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47fd551d7200a2a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd097fa700 1 -- 172.21.15.49:0/618274247 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c019070 con 0x56259f4897b0 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.895+0000 7efd097fa700 1 -- 172.21.15.49:0/618274247 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1c00f040 con 0x56259f4897b0 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd097fa700 1 -- 172.21.15.49:0/618274247 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c007b60 con 0x56259f4897b0 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 -- 172.21.15.49:0/618274247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f4897b0 msgr2=0x56259f489c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f4897b0 0x56259f489c40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0098f0 tx=0x7efd1c009c00 comp rx=0 tx=0).stop 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 -- 172.21.15.49:0/618274247 shutdown_connections 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f4897b0 0x56259f489c40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f488430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 --2- 172.21.15.49:0/618274247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f489220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 -- 172.21.15.49:0/618274247 >> 172.21.15.49:0/618274247 conn(0x56259f3fc210 msgr2=0x56259f482db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 -- 172.21.15.49:0/618274247 shutdown_connections 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.896+0000 7efd353dab80 1 -- 172.21.15.49:0/618274247 wait complete. 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 -- start start 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 Processor -- start 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f491d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488970 0x56259f492290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f4992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f3dade0 con 0x56259f488010 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f3776c0 con 0x56259f488970 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f377360 con 0x56259f4897b0 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd353dab80 10 client.0 _ll_register_callbacks cb 0x56259f1d47b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f491d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f491d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49650/0 (socket says 172.21.15.49:49650) 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 -- 172.21.15.49:0/210702328 learned_addr learned my addr 172.21.15.49:0/210702328 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd2259c700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488970 0x56259f492290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 -- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 msgr2=0x56259f4992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd2359e700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f4992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f4992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 -- 172.21.15.49:0/210702328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488970 msgr2=0x56259f492290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488970 0x56259f492290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd22d9d700 1 -- 172.21.15.49:0/210702328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c003c50 con 0x56259f488010 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.897+0000 7efd2259c700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488970 0x56259f492290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd22d9d700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f491d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd1809ae10 tx=0x7efd18005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e5ec45105efbb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:52.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd06ff5700 1 -- 172.21.15.49:0/210702328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1809b580 con 0x56259f488010 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd06ff5700 1 -- 172.21.15.49:0/210702328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efd1809ae80 con 0x56259f488010 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56259f3a2e60 con 0x56259f488010 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd06ff5700 1 -- 172.21.15.49:0/210702328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd1809efd0 con 0x56259f488010 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 10 client.15555 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56259f1eec10 con 0x56259f488010 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd06ff5700 1 -- 172.21.15.49:0/210702328 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd180b9030 con 0x56259f488010 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd06ff5700 1 -- 172.21.15.49:0/210702328 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efd180a2d80 con 0x56259f488010 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 10 client.15555 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 20 client.15555 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f49d5f0 0x56259f49c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 4 client.15555 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:52.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.898+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 --> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56259f3d3dc0 con 0x56259f49d5f0 2023-10-08T09:14:52.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.899+0000 7efd2259c700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f49d5f0 0x56259f49c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.899+0000 7efd06ff5700 10 client.15555 ms_handle_connect on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:52.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.899+0000 7efd2259c700 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f49d5f0 0x56259f49c2a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19f56001110e9f91 server_cookie=9fb8b01c2d83ed1e in_seq=0 out_seq=0 2023-10-08T09:14:52.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd06ff5700 1 -- 172.21.15.49:0/210702328 <== mds.3 v2:172.21.15.184:6832/3616311663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56259f3d3dc0 con 0x56259f49d5f0 2023-10-08T09:14:52.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd06ff5700 10 client.15555 handle_command_reply: tid=0 2023-10-08T09:14:52.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 client.15555 shutdown 2023-10-08T09:14:52.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 msgr2=0x56259f491d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f491d50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd1809ae10 tx=0x7efd18005fb0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 shutdown_connections 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f4992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f49d5f0 0x56259f49c2a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f491d50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 --2- 172.21.15.49:0/210702328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488970 0x56259f492290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd06ff5700 0 client.15555 ms_handle_reset on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 >> 172.21.15.49:0/210702328 conn(0x56259f3fc210 msgr2=0x56259f482480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 shutdown_connections 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 1 -- 172.21.15.49:0/210702328 wait complete. 2023-10-08T09:14:52.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.900+0000 7efd353dab80 20 client.15555 trim_cache size 0 max 16384 2023-10-08T09:14:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 Processor -- start 2023-10-08T09:14:52.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 -- start start 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f4883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f499290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f4897b0 0x56259f48c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f25b8d0 con 0x56259f488010 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f47fa90 con 0x56259f4897b0 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd353dab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f488de0 con 0x56259f488970 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f4883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f4883f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49658/0 (socket says 172.21.15.49:49658) 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 -- 172.21.15.49:0/598930325 learned_addr learned my addr 172.21.15.49:0/598930325 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd2359e700 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f4897b0 0x56259f48c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 -- 172.21.15.49:0/598930325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 msgr2=0x56259f499290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd2259c700 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f499290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f499290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 -- 172.21.15.49:0/598930325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f4897b0 msgr2=0x56259f48c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f4897b0 0x56259f48c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd22d9d700 1 -- 172.21.15.49:0/598930325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56259f22c200 con 0x56259f488010 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.918+0000 7efd2359e700 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f4897b0 0x56259f48c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd22d9d700 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f4883f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efd18000c00 tx=0x7efd180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17be0e2b1e30736f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd09ffb700 1 -- 172.21.15.49:0/598930325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18009290 con 0x56259f488010 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd09ffb700 1 -- 172.21.15.49:0/598930325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd180b9040 con 0x56259f488010 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd09ffb700 1 -- 172.21.15.49:0/598930325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd180a3180 con 0x56259f488010 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 -- 172.21.15.49:0/598930325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 msgr2=0x56259f4883f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f4883f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efd18000c00 tx=0x7efd180057f0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 -- 172.21.15.49:0/598930325 shutdown_connections 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f4897b0 0x56259f48c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488010 0x56259f4883f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 --2- 172.21.15.49:0/598930325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f488970 0x56259f499290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 -- 172.21.15.49:0/598930325 >> 172.21.15.49:0/598930325 conn(0x56259f3fc210 msgr2=0x56259f491260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 -- 172.21.15.49:0/598930325 shutdown_connections 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.919+0000 7efd353dab80 1 -- 172.21.15.49:0/598930325 wait complete. 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 -- start start 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 Processor -- start 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f494460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488970 0x56259f499280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f494b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f483ab0 con 0x56259f488970 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f49afe0 con 0x56259f488010 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56259f4f1090 con 0x56259f4897b0 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd353dab80 10 client.0 _ll_register_callbacks cb 0x7efd1809ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488970 0x56259f499280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488970 0x56259f499280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49666/0 (socket says 172.21.15.49:49666) 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 -- 172.21.15.49:0/3040781774 learned_addr learned my addr 172.21.15.49:0/3040781774 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd22d9d700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f494460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2359e700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f494b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 -- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 msgr2=0x56259f494b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f494b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 -- 172.21.15.49:0/3040781774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 msgr2=0x56259f494460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f494460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd2259c700 1 -- 172.21.15.49:0/3040781774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56259f1eec10 con 0x56259f488970 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.920+0000 7efd22d9d700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f494460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd2259c700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488970 0x56259f499280 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efd14005e10 tx=0x7efd140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2c59042c628a85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd06ff5700 1 -- 172.21.15.49:0/3040781774 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400be70 con 0x56259f488970 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd1c00f040 con 0x56259f488970 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd06ff5700 1 -- 172.21.15.49:0/3040781774 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efd14017070 con 0x56259f488970 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd06ff5700 1 -- 172.21.15.49:0/3040781774 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd140261c0 con 0x56259f488970 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 10 client.15561 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56259f204260 con 0x56259f488970 2023-10-08T09:14:52.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd06ff5700 1 -- 172.21.15.49:0/3040781774 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1c00f040 con 0x56259f488970 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd06ff5700 1 -- 172.21.15.49:0/3040781774 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efd1402c020 con 0x56259f488970 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 10 client.15561 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 10 client.15561 resolve_mds: resolved name 'l' to daemon mds.l 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 20 client.15561 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f4f6380 0x56259f4f2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 4 client.15561 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.921+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 --> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56259f3d3ae0 con 0x56259f4f6380 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd22d9d700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f4f6380 0x56259f4f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd06ff5700 10 client.15561 ms_handle_connect on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd22d9d700 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f4f6380 0x56259f4f2800 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12f864929746ef8 server_cookie=197280f2f315faff in_seq=0 out_seq=0 2023-10-08T09:14:52.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd06ff5700 1 -- 172.21.15.49:0/3040781774 <== mds.3 v2:172.21.15.184:6832/3616311663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56259f3d3ae0 con 0x56259f4f6380 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd06ff5700 10 client.15561 handle_command_reply: tid=0 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 client.15561 shutdown 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488970 msgr2=0x56259f499280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488970 0x56259f499280 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7efd14005e10 tx=0x7efd140047c0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 shutdown_connections 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56259f4897b0 0x56259f494b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x56259f4f6380 0x56259f4f2800 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56259f488010 0x56259f494460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd06ff5700 0 client.15561 ms_handle_reset on v2:172.21.15.184:6832/3616311663 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 --2- 172.21.15.49:0/3040781774 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56259f488970 0x56259f499280 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.922+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 >> 172.21.15.49:0/3040781774 conn(0x56259f3fc210 msgr2=0x56259f48c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 shutdown_connections 2023-10-08T09:14:52.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd353dab80 1 -- 172.21.15.49:0/3040781774 wait complete. 2023-10-08T09:14:52.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd353dab80 20 client.15561 trim_cache size 0 max 16384 2023-10-08T09:14:52.932 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:52.932 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:52.932 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 3, 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24553, 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 81.514938529999995, 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 90, 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 141.22045815000001 2023-10-08T09:14:52.933 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:52.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd0c0a1480 msgr2=0x7efd0c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd0c0a1480 0x7efd0c0a3940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efd14005950 tx=0x7efd1400b410 comp rx=0 tx=0).stop 2023-10-08T09:14:52.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 msgr2=0x7efd24196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:52.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd24196740 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efd18003820 tx=0x7efd18004730 comp rx=0 tx=0).stop 2023-10-08T09:14:52.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 shutdown_connections 2023-10-08T09:14:52.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd0c0a1480 0x7efd0c0a3940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd24101230 0x7efd2419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efd240ff6a0 0x7efd24196740 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 --2- 172.21.15.49:0/1739258209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd24100890 0x7efd24196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:52.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 >> 172.21.15.49:0/1739258209 conn(0x7efd240fac10 msgr2=0x7efd24103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:52.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 shutdown_connections 2023-10-08T09:14:52.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:52.923+0000 7efd291f1700 1 -- 172.21.15.49:0/1739258209 wait complete. 2023-10-08T09:14:52.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-08T09:14:53.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 -- 172.21.15.49:0/3336860443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb4101240 msgr2=0x7f7bb4107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/3336860443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb4101240 0x7f7bb4107af0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00b0d0 tx=0x7f7bac00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 -- 172.21.15.49:0/3336860443 shutdown_connections 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/3336860443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40ff6b0 0x7f7bb40ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/3336860443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb4101240 0x7f7bb4107af0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/3336860443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb41008a0 0x7f7bb4100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 -- 172.21.15.49:0/3336860443 >> 172.21.15.49:0/3336860443 conn(0x7f7bb40fac40 msgr2=0x7f7bb40fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 -- 172.21.15.49:0/3336860443 shutdown_connections 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 -- 172.21.15.49:0/3336860443 wait complete. 2023-10-08T09:14:53.065 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 Processor -- start 2023-10-08T09:14:53.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.063+0000 7f7bb9de5700 1 -- start start 2023-10-08T09:14:53.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb9de5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40ff6b0 0x7f7bb41968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb9de5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb41008a0 0x7f7bb4196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb9de5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4101240 0x7f7bb419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.067 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb9de5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb419e390 con 0x7f7bb41008a0 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb9de5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb419e4d0 con 0x7f7bb4101240 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb9de5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb419e660 con 0x7f7bb40ff6b0 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb41008a0 0x7f7bb4196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb41008a0 0x7f7bb4196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49678/0 (socket says 172.21.15.49:49678) 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/1195522664 learned_addr learned my addr 172.21.15.49:0/1195522664 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40ff6b0 0x7f7bb41968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4101240 0x7f7bb419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/1195522664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40ff6b0 msgr2=0x7f7bb41968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40ff6b0 0x7f7bb41968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/1195522664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4101240 msgr2=0x7f7bb419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4101240 0x7f7bb419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/1195522664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac009d00 con 0x7f7bb41008a0 2023-10-08T09:14:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb41008a0 0x7f7bb4196df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800d950 tx=0x7f7ba800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb0ff9700 1 -- 172.21.15.49:0/1195522664 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800f890 con 0x7f7bb41008a0 2023-10-08T09:14:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb0ff9700 1 -- 172.21.15.49:0/1195522664 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba8010460 con 0x7f7bb41008a0 2023-10-08T09:14:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb0ff9700 1 -- 172.21.15.49:0/1195522664 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800e610 con 0x7f7bb41008a0 2023-10-08T09:14:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.064+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb419e940 con 0x7f7bb41008a0 2023-10-08T09:14:53.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.065+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb419ee00 con 0x7f7bb41008a0 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.067+0000 7f7bb0ff9700 1 -- 172.21.15.49:0/1195522664 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7ba80108d0 con 0x7f7bb41008a0 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.067+0000 7f7bb0ff9700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b9c0a1480 0x7f7b9c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.067+0000 7f7bb0ff9700 1 -- 172.21.15.49:0/1195522664 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7ba80cc810 con 0x7f7bb41008a0 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.067+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b9c0a1480 0x7f7b9c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b9c0a1480 0x7f7b9c0a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4100700 tx=0x7f7ba400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 Processor -- start 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 -- start start 2023-10-08T09:14:53.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a13840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a14630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a14bc0 0x55a910a15050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910977410 con 0x55a910a13d80 2023-10-08T09:14:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910968bd0 con 0x55a910a13420 2023-10-08T09:14:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9109664f0 con 0x55a910a14bc0 2023-10-08T09:14:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a14630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a14630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49686/0 (socket says 172.21.15.49:49686) 2023-10-08T09:14:53.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2139724924 learned_addr learned my addr 172.21.15.49:0/2139724924 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.068+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a13840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2139724924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a14bc0 msgr2=0x55a910a15050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a14bc0 0x55a910a15050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2139724924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 msgr2=0x55a910a13840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a13840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2139724924 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a91078cd90 con 0x55a910a13d80 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a14630 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8096270 tx=0x7f7ba8005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f248118aec1a65 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7b99ffb700 1 -- 172.21.15.49:0/2139724924 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80a1b90 con 0x55a910a13d80 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7b99ffb700 1 -- 172.21.15.49:0/2139724924 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba8097ba0 con 0x55a910a13d80 2023-10-08T09:14:53.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.069+0000 7f7b99ffb700 1 -- 172.21.15.49:0/2139724924 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8099cc0 con 0x55a910a13d80 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2139724924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 msgr2=0x55a910a14630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a14630 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8096270 tx=0x7f7ba8005910 comp rx=0 tx=0).stop 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2139724924 shutdown_connections 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a14bc0 0x55a910a15050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a13840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2139724924 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a14630 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2139724924 >> 172.21.15.49:0/2139724924 conn(0x55a910983210 msgr2=0x55a910a09db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2139724924 shutdown_connections 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2139724924 wait complete. 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 -- start start 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.070+0000 7f7bc5fceb80 1 Processor -- start 2023-10-08T09:14:53.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a1d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910961de0 con 0x55a910a13d80 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9108fe6c0 con 0x55a910a14bc0 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9108fe360 con 0x55a910a13420 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 10 client.0 _ll_register_callbacks cb 0x55a91075b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a1d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49692/0 (socket says 172.21.15.49:49692) 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2783363102 learned_addr learned my addr 172.21.15.49:0/2783363102 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a1d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2783363102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 msgr2=0x55a910a1d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a1d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 msgr2=0x55a910a24750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a24750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/2783363102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb419ee00 con 0x55a910a13d80 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a1d760 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80840f0 tx=0x7f7ba8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9c27a0ab6c509f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7b92ffd700 1 -- 172.21.15.49:0/2783363102 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8098ef0 con 0x55a910a13d80 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7b92ffd700 1 -- 172.21.15.49:0/2783363102 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7ba80a1b90 con 0x55a910a13d80 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.071+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a910929e60 con 0x55a910a13d80 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7b92ffd700 1 -- 172.21.15.49:0/2783363102 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba8099a60 con 0x55a910a13d80 2023-10-08T09:14:53.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 10 client.15573 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a910775c10 con 0x55a910a13d80 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7b92ffd700 1 -- 172.21.15.49:0/2783363102 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a910929e60 con 0x55a910a13d80 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7b92ffd700 1 -- 172.21.15.49:0/2783363102 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ba80955f0 con 0x55a910a13d80 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 10 client.15573 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 20 client.15573 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a288c0 0x55a910a27720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 4 client.15573 mds_command: new command op to 24608 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 --> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a91095adc0 con 0x55a910a288c0 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.072+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a288c0 0x55a910a27720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.073+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a288c0 0x55a910a27720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1dd953d3d6b8178d server_cookie=cc7972244086cca2 in_seq=0 out_seq=0 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.073+0000 7f7b92ffd700 10 client.15573 ms_handle_connect on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.073+0000 7f7b92ffd700 1 -- 172.21.15.49:0/2783363102 <== mds.2 v2:172.21.15.132:6836/2064856316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a91095adc0 con 0x55a910a288c0 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.073+0000 7f7b92ffd700 10 client.15573 handle_command_reply: tid=0 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.073+0000 7f7bc5fceb80 1 client.15573 shutdown 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 msgr2=0x55a910a1d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a1d760 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80840f0 tx=0x7f7ba8000bc0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 shutdown_connections 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a1d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a288c0 0x55a910a27720 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7b92ffd700 0 client.15573 ms_handle_reset on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:53.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/2783363102 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a1d760 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 >> 172.21.15.49:0/2783363102 conn(0x55a910983210 msgr2=0x55a910a09480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 shutdown_connections 2023-10-08T09:14:53.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/2783363102 wait complete. 2023-10-08T09:14:53.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.074+0000 7f7bc5fceb80 20 client.15573 trim_cache size 0 max 16384 2023-10-08T09:14:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 Processor -- start 2023-10-08T09:14:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 -- start start 2023-10-08T09:14:53.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13d80 0x55a910a24710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a14bc0 0x55a910a05ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9107e28d0 con 0x55a910a14bc0 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910963070 con 0x55a910a13420 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91095ded0 con 0x55a910a13d80 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bb3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a14bc0 0x55a910a05ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bb3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a14bc0 0x55a910a05ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49702/0 (socket says 172.21.15.49:49702) 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.091+0000 7f7bb3fff700 1 -- 172.21.15.49:0/4115489012 learned_addr learned my addr 172.21.15.49:0/4115489012 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a24e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13d80 0x55a910a24710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb3fff700 1 -- 172.21.15.49:0/4115489012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13d80 msgr2=0x55a910a24710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13d80 0x55a910a24710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb3fff700 1 -- 172.21.15.49:0/4115489012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 msgr2=0x55a910a24e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a24e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb3fff700 1 -- 172.21.15.49:0/4115489012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9107b3200 con 0x55a910a14bc0 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a24e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a14bc0 0x55a910a05ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7bac015000 tx=0x7f7bac007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38526eff76a1916d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7b9a7fc700 1 -- 172.21.15.49:0/4115489012 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00e070 con 0x55a910a14bc0 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7b9a7fc700 1 -- 172.21.15.49:0/4115489012 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bac009b80 con 0x55a910a14bc0 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.092+0000 7f7b9a7fc700 1 -- 172.21.15.49:0/4115489012 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0055a0 con 0x55a910a14bc0 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4115489012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a14bc0 msgr2=0x55a910a05ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a14bc0 0x55a910a05ee0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7bac015000 tx=0x7f7bac007600 comp rx=0 tx=0).stop 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4115489012 shutdown_connections 2023-10-08T09:14:53.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a14bc0 0x55a910a05ee0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a13420 0x55a910a24e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4115489012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13d80 0x55a910a24710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4115489012 >> 172.21.15.49:0/4115489012 conn(0x55a910983210 msgr2=0x55a910a09d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4115489012 shutdown_connections 2023-10-08T09:14:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4115489012 wait complete. 2023-10-08T09:14:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- start start 2023-10-08T09:14:53.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 Processor -- start 2023-10-08T09:14:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a214e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a21a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a7c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910a24900 con 0x55a910a13d80 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910a24a70 con 0x55a910a14bc0 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.093+0000 7f7bc5fceb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a910a7c7f0 con 0x55a910a13420 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bc5fceb80 10 client.0 _ll_register_callbacks cb 0x55a91075b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a21a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a21a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49706/0 (socket says 172.21.15.49:49706) 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4037178375 learned_addr learned my addr 172.21.15.49:0/4037178375 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb3fff700 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a7c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4037178375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 msgr2=0x55a910a214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 msgr2=0x55a910a7c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a7c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 -- 172.21.15.49:0/4037178375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb4108440 con 0x55a910a13d80 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bb2ffd700 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a21a20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80ca350 tx=0x7f7ba80ca050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c601e813b6526bbb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7b92ffd700 1 -- 172.21.15.49:0/4037178375 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8084120 con 0x55a910a13d80 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7b92ffd700 1 -- 172.21.15.49:0/4037178375 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7ba80a1b90 con 0x55a910a13d80 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a91095aae0 con 0x55a910a13d80 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7b92ffd700 1 -- 172.21.15.49:0/4037178375 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ba8098cf0 con 0x55a910a13d80 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bc5fceb80 10 client.15579 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:53.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.094+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a91078b260 con 0x55a910a13d80 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7b92ffd700 1 -- 172.21.15.49:0/4037178375 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ba807c080 con 0x55a910a13d80 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7b92ffd700 1 -- 172.21.15.49:0/4037178375 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7ba8098840 con 0x55a910a13d80 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7bc5fceb80 10 client.15579 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7bc5fceb80 10 client.15579 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7bc5fceb80 20 client.15579 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a81530 0x55a910a7d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7bc5fceb80 4 client.15579 mds_command: new command op to 24608 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 --> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a9107adf20 con 0x55a910a81530 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.095+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a81530 0x55a910a7d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7b92ffd700 10 client.15579 ms_handle_connect on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bb37fe700 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a81530 0x55a910a7d9c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50dd05ec79ed1568 server_cookie=8174d01bb2448ac8 in_seq=0 out_seq=0 2023-10-08T09:14:53.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7b92ffd700 1 -- 172.21.15.49:0/4037178375 <== mds.2 v2:172.21.15.132:6836/2064856316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a9107adf20 con 0x55a910a81530 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7b92ffd700 10 client.15579 handle_command_reply: tid=0 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 client.15579 shutdown 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 msgr2=0x55a910a21a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a21a20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80ca350 tx=0x7f7ba80ca050 comp rx=0 tx=0).stop 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 shutdown_connections 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a910a13420 0x55a910a214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x55a910a81530 0x55a910a7d9c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a910a14bc0 0x55a910a7c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 --2- 172.21.15.49:0/4037178375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a910a13d80 0x55a910a21a20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7b92ffd700 0 client.15579 ms_handle_reset on v2:172.21.15.132:6836/2064856316 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 >> 172.21.15.49:0/4037178375 conn(0x55a910983210 msgr2=0x55a910a09d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 shutdown_connections 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 1 -- 172.21.15.49:0/4037178375 wait complete. 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.096+0000 7f7bc5fceb80 20 client.15579 trim_cache size 0 max 16384 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24608, 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:53.105 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:53.106 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:53.106 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 82.639921252999997, 2023-10-08T09:14:53.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:53.106 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 90, 2023-10-08T09:14:53.106 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:53.106 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 126.07786576700001 2023-10-08T09:14:53.106 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b9c0a1480 msgr2=0x7f7b9c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b9c0a1480 0x7f7b9c0a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4100700 tx=0x7f7ba400b410 comp rx=0 tx=0).stop 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb41008a0 msgr2=0x7f7bb4196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb41008a0 0x7f7bb4196df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800d950 tx=0x7f7ba800dc60 comp rx=0 tx=0).stop 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 shutdown_connections 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b9c0a1480 0x7f7b9c0a3940 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7bb40ff6b0 0x7f7bb41968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7bb4101240 0x7f7bb419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 --2- 172.21.15.49:0/1195522664 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bb41008a0 0x7f7bb4196df0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 >> 172.21.15.49:0/1195522664 conn(0x7f7bb40fac40 msgr2=0x7f7bb40fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 shutdown_connections 2023-10-08T09:14:53.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.097+0000 7f7bb9de5700 1 -- 172.21.15.49:0/1195522664 wait complete. 2023-10-08T09:14:53.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-08T09:14:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 -- 172.21.15.49:0/4096498311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 msgr2=0x7fd418100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 --2- 172.21.15.49:0/4096498311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418100be0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0099c0 tx=0x7fd40c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 -- 172.21.15.49:0/4096498311 shutdown_connections 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 --2- 172.21.15.49:0/4096498311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd418102350 0x7fd418108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 --2- 172.21.15.49:0/4096498311 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418100be0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 --2- 172.21.15.49:0/4096498311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4181019b0 0x7fd418101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 -- 172.21.15.49:0/4096498311 >> 172.21.15.49:0/4096498311 conn(0x7fd4180fbd30 msgr2=0x7fd4180fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 -- 172.21.15.49:0/4096498311 shutdown_connections 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 -- 172.21.15.49:0/4096498311 wait complete. 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 Processor -- start 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.235+0000 7fd41f142700 1 -- start start 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41f142700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41f142700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4181019b0 0x7fd418072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41f142700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd418102350 0x7fd418073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41f142700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180738c0 con 0x7fd4181007c0 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41f142700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418073a30 con 0x7fd4181019b0 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41f142700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181a0610 con 0x7fd418102350 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49718/0 (socket says 172.21.15.49:49718) 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 -- 172.21.15.49:0/2286810830 learned_addr learned my addr 172.21.15.49:0/2286810830 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41d6df700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd418102350 0x7fd418073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd417fff700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4181019b0 0x7fd418072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 -- 172.21.15.49:0/2286810830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd418102350 msgr2=0x7fd418073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.240 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd418102350 0x7fd418073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 -- 172.21.15.49:0/2286810830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4181019b0 msgr2=0x7fd418072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4181019b0 0x7fd418072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 -- 172.21.15.49:0/2286810830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0095f0 con 0x7fd4181007c0 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41d6df700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd418102350 0x7fd418073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41cede700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418074550 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009f50 tx=0x7fd40c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd415ffb700 1 -- 172.21.15.49:0/2286810830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c01d070 con 0x7fd4181007c0 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd415ffb700 1 -- 172.21.15.49:0/2286810830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c022470 con 0x7fd4181007c0 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4181a0840 con 0x7fd4181007c0 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.236+0000 7fd415ffb700 1 -- 172.21.15.49:0/2286810830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c022ea0 con 0x7fd4181007c0 2023-10-08T09:14:53.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.237+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4181a0c90 con 0x7fd4181007c0 2023-10-08T09:14:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.240+0000 7fd415ffb700 1 -- 172.21.15.49:0/2286810830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd40c022800 con 0x7fd4181007c0 2023-10-08T09:14:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.240+0000 7fd415ffb700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4080a1480 0x7fd4080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.240+0000 7fd417fff700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4080a1480 0x7fd4080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.240+0000 7fd415ffb700 1 -- 172.21.15.49:0/2286810830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd40c0ce530 con 0x7fd4181007c0 2023-10-08T09:14:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 Processor -- start 2023-10-08T09:14:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 -- start start 2023-10-08T09:14:53.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd417fff700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4080a1480 0x7fd4080a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd404005ea0 tx=0x7fd404005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d6e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6e970 0x55d5e2d6f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d6fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2cd6410 con 0x55d5e2d6e010 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2cc7bd0 con 0x55d5e2d6e970 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2cc54f0 con 0x55d5e2d6f7b0 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d6e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d6e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49726/0 (socket says 172.21.15.49:49726) 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 -- 172.21.15.49:0/3505641652 learned_addr learned my addr 172.21.15.49:0/3505641652 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd417fff700 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6e970 0x55d5e2d6f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 -- 172.21.15.49:0/3505641652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6f7b0 msgr2=0x55d5e2d6fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d6fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 -- 172.21.15.49:0/3505641652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6e970 msgr2=0x55d5e2d6f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6e970 0x55d5e2d6f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 -- 172.21.15.49:0/3505641652 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5e2aebd90 con 0x55d5e2d6e010 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd417fff700 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6e970 0x55d5e2d6f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.241+0000 7fd41cede700 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d6e430 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd40c097a50 tx=0x7fd40c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cc23deaf60a38e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd402ffd700 1 -- 172.21.15.49:0/3505641652 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09bc50 con 0x55d5e2d6e010 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd402ffd700 1 -- 172.21.15.49:0/3505641652 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c09b550 con 0x55d5e2d6e010 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd402ffd700 1 -- 172.21.15.49:0/3505641652 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09aa40 con 0x55d5e2d6e010 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3505641652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 msgr2=0x55d5e2d6e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d6e430 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd40c097a50 tx=0x7fd40c005a20 comp rx=0 tx=0).stop 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3505641652 shutdown_connections 2023-10-08T09:14:53.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d6fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d6e430 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3505641652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6e970 0x55d5e2d6f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3505641652 >> 172.21.15.49:0/3505641652 conn(0x55d5e2ce2210 msgr2=0x55d5e2d68db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3505641652 shutdown_connections 2023-10-08T09:14:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.242+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3505641652 wait complete. 2023-10-08T09:14:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 -- start start 2023-10-08T09:14:53.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 Processor -- start 2023-10-08T09:14:53.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e010 0x55d5e2d77ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e970 0x55d5e2d78020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2cc0de0 con 0x55d5e2d6e970 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2c5d6c0 con 0x55d5e2d6f7b0 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2c5d360 con 0x55d5e2d6e010 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd42b32bb80 10 client.0 _ll_register_callbacks cb 0x55d5e2aba7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd417fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e970 0x55d5e2d78020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd417fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e970 0x55d5e2d78020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49732/0 (socket says 172.21.15.49:49732) 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd417fff700 1 -- 172.21.15.49:0/915164369 learned_addr learned my addr 172.21.15.49:0/915164369 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.243+0000 7fd41d6df700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd41cede700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e010 0x55d5e2d77ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd417fff700 1 -- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e010 msgr2=0x55d5e2d77ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd417fff700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e010 0x55d5e2d77ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd417fff700 1 -- 172.21.15.49:0/915164369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 msgr2=0x55d5e2d7f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd417fff700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd417fff700 1 -- 172.21.15.49:0/915164369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40c097040 con 0x55d5e2d6e970 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd41cede700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e010 0x55d5e2d77ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd417fff700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e970 0x55d5e2d78020 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd404010730 tx=0x7fd4040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a33a86fc221c5f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd3f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd3f7fff700 1 -- 172.21.15.49:0/915164369 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400f850 con 0x55d5e2d6e970 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd3f7fff700 1 -- 172.21.15.49:0/915164369 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd40400f9b0 con 0x55d5e2d6e970 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5e2c88e60 con 0x55d5e2d6e970 2023-10-08T09:14:53.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd3f7fff700 1 -- 172.21.15.49:0/915164369 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd40400fbb0 con 0x55d5e2d6e970 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd42b32bb80 10 client.15591 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d5e2ad4c10 con 0x55d5e2d6e970 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd3f7fff700 1 -- 172.21.15.49:0/915164369 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5e2c88e60 con 0x55d5e2d6e970 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd3f7fff700 1 -- 172.21.15.49:0/915164369 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd404023020 con 0x55d5e2d6e970 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.244+0000 7fd42b32bb80 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd42b32bb80 10 client.15591 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd42b32bb80 20 client.15591 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2d836b0 0x55d5e2d82180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd42b32bb80 4 client.15591 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 --> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5e2cb9dc0 con 0x55d5e2d836b0 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd41cede700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2d836b0 0x55d5e2d82180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd41cede700 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2d836b0 0x55d5e2d82180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8b410de08b144d2 server_cookie=166ab8491726f8bf in_seq=0 out_seq=0 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.245+0000 7fd3f7fff700 10 client.15591 ms_handle_connect on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd3f7fff700 1 -- 172.21.15.49:0/915164369 <== mds.1 v2:172.21.15.184:6838/2795080121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d5e2cb9dc0 con 0x55d5e2d836b0 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd3f7fff700 10 client.15591 handle_command_reply: tid=0 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 client.15591 shutdown 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e970 msgr2=0x55d5e2d78020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e970 0x55d5e2d78020 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd404010730 tx=0x7fd4040047c0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 shutdown_connections 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2d836b0 0x55d5e2d82180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e010 0x55d5e2d77ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/915164369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e970 0x55d5e2d78020 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd3f7fff700 0 client.15591 ms_handle_reset on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 >> 172.21.15.49:0/915164369 conn(0x55d5e2ce2210 msgr2=0x55d5e2d68480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 shutdown_connections 2023-10-08T09:14:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 1 -- 172.21.15.49:0/915164369 wait complete. 2023-10-08T09:14:53.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.246+0000 7fd42b32bb80 20 client.15591 trim_cache size 0 max 16384 2023-10-08T09:14:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 Processor -- start 2023-10-08T09:14:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 -- start start 2023-10-08T09:14:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d76ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 0x55d5e2d7efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2b418d0 con 0x55d5e2d6e010 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2cc54f0 con 0x55d5e2d6f7b0 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2d6a180 con 0x55d5e2d6e970 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d76ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d76ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49740/0 (socket says 172.21.15.49:49740) 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 -- 172.21.15.49:0/2642499395 learned_addr learned my addr 172.21.15.49:0/2642499395 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41d6df700 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 -- 172.21.15.49:0/2642499395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 msgr2=0x55d5e2d7efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 0x55d5e2d7efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 -- 172.21.15.49:0/2642499395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 msgr2=0x55d5e2d767b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d767b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41cede700 1 -- 172.21.15.49:0/2642499395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5e2b12200 con 0x55d5e2d6e010 2023-10-08T09:14:53.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.264+0000 7fd41d6df700 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd41cede700 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d76ff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd40c09b4e0 tx=0x7fd40c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8caec5ea1a06fbd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd4037fe700 1 -- 172.21.15.49:0/2642499395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09bc50 con 0x55d5e2d6e010 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd4037fe700 1 -- 172.21.15.49:0/2642499395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c09f720 con 0x55d5e2d6e010 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd4037fe700 1 -- 172.21.15.49:0/2642499395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09aca0 con 0x55d5e2d6e010 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 -- 172.21.15.49:0/2642499395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 msgr2=0x55d5e2d76ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d76ff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd40c09b4e0 tx=0x7fd40c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 -- 172.21.15.49:0/2642499395 shutdown_connections 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d76ff0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/2642499395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 0x55d5e2d7efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 -- 172.21.15.49:0/2642499395 >> 172.21.15.49:0/2642499395 conn(0x55d5e2ce2210 msgr2=0x55d5e2d64ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 -- 172.21.15.49:0/2642499395 shutdown_connections 2023-10-08T09:14:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.265+0000 7fd42b32bb80 1 -- 172.21.15.49:0/2642499395 wait complete. 2023-10-08T09:14:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 -- start start 2023-10-08T09:14:53.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 Processor -- start 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d7ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 0x55d5e2d7ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2cd6410 con 0x55d5e2d6e010 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2d7c600 con 0x55d5e2d6f7b0 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5e2dd70a0 con 0x55d5e2d6e970 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd42b32bb80 10 client.0 _ll_register_callbacks cb 0x7fd40400ed10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:53.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d7ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd41cede700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d7ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49746/0 (socket says 172.21.15.49:49746) 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd41cede700 1 -- 172.21.15.49:0/3681458710 learned_addr learned my addr 172.21.15.49:0/3681458710 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd417fff700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 0x55d5e2d7ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.266+0000 7fd41d6df700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd41cede700 1 -- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 msgr2=0x55d5e2d7ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd41cede700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 0x55d5e2d7ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd41cede700 1 -- 172.21.15.49:0/3681458710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 msgr2=0x55d5e2d7bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd41cede700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd41cede700 1 -- 172.21.15.49:0/3681458710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5e2aebd90 con 0x55d5e2d6e010 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd41d6df700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:53.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd41cede700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d7ee90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0a3320 tx=0x7fd40c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5926b13c11e25163 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd3f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd3f7fff700 1 -- 172.21.15.49:0/3681458710 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09aeb0 con 0x55d5e2d6e010 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd3f7fff700 1 -- 172.21.15.49:0/3681458710 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd40c09bc50 con 0x55d5e2d6e010 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd40c09b550 con 0x55d5e2d6e010 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd3f7fff700 1 -- 172.21.15.49:0/3681458710 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd40c09ac50 con 0x55d5e2d6e010 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd42b32bb80 10 client.15597 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d5e2aea260 con 0x55d5e2d6e010 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd3f7fff700 1 -- 172.21.15.49:0/3681458710 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd40c09b550 con 0x55d5e2d6e010 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd3f7fff700 1 -- 172.21.15.49:0/3681458710 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd40c09f160 con 0x55d5e2d6e010 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.267+0000 7fd42b32bb80 10 client.15597 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 10 client.15597 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-08T09:14:53.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 20 client.15597 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2ddba60 0x55d5e2dd8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 4 client.15597 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 --> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d5e2cb9ae0 con 0x55d5e2ddba60 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd417fff700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2ddba60 0x55d5e2dd8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd3f7fff700 10 client.15597 ms_handle_connect on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd417fff700 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2ddba60 0x55d5e2dd8290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=893df2935f0bf175 server_cookie=d1c24d570fc147cc in_seq=0 out_seq=0 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd3f7fff700 1 -- 172.21.15.49:0/3681458710 <== mds.1 v2:172.21.15.184:6838/2795080121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d5e2cb9ae0 con 0x55d5e2ddba60 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd3f7fff700 10 client.15597 handle_command_reply: tid=0 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 1 client.15597 shutdown 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 msgr2=0x55d5e2d7ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.268+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d7ee90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0a3320 tx=0x7fd40c005c00 comp rx=0 tx=0).stop 2023-10-08T09:14:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 shutdown_connections 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x55d5e2ddba60 0x55d5e2dd8290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d5e2d6f7b0 0x55d5e2d7bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd3f7fff700 0 client.15597 ms_handle_reset on v2:172.21.15.184:6838/2795080121 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d5e2d6e010 0x55d5e2d7ee90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 --2- 172.21.15.49:0/3681458710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d5e2d6e970 0x55d5e2d7ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 >> 172.21.15.49:0/3681458710 conn(0x55d5e2ce2210 msgr2=0x55d5e2d82180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 shutdown_connections 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 1 -- 172.21.15.49:0/3681458710 wait complete. 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd42b32bb80 20 client.15597 trim_cache size 0 max 16384 2023-10-08T09:14:53.277 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24616, 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 83.820615566000001, 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 90, 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 124.03284263099999 2023-10-08T09:14:53.279 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4080a1480 msgr2=0x7fd4080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4080a1480 0x7fd4080a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd404005ea0 tx=0x7fd404005e30 comp rx=0 tx=0).stop 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 msgr2=0x7fd418074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418074550 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009f50 tx=0x7fd40c003970 comp rx=0 tx=0).stop 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 shutdown_connections 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4080a1480 0x7fd4080a3940 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd418102350 0x7fd418073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd4181007c0 0x7fd418074550 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 --2- 172.21.15.49:0/2286810830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4181019b0 0x7fd418072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 >> 172.21.15.49:0/2286810830 conn(0x7fd4180fbd30 msgr2=0x7fd418104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.269+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 shutdown_connections 2023-10-08T09:14:53.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.270+0000 7fd41f142700 1 -- 172.21.15.49:0/2286810830 wait complete. 2023-10-08T09:14:53.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T09:14:53.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 -- 172.21.15.49:0/2737069264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8101230 msgr2=0x7fdaa8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/2737069264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8101230 0x7fdaa8107ae0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b0d0 tx=0x7fdaa000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 -- 172.21.15.49:0/2737069264 shutdown_connections 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/2737069264 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8101230 0x7fdaa8107ae0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/2737069264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa80ff6a0 0x7fdaa80ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/2737069264 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaa8100890 0x7fdaa8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 -- 172.21.15.49:0/2737069264 >> 172.21.15.49:0/2737069264 conn(0x7fdaa80fac10 msgr2=0x7fdaa80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 -- 172.21.15.49:0/2737069264 shutdown_connections 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 -- 172.21.15.49:0/2737069264 wait complete. 2023-10-08T09:14:53.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 Processor -- start 2023-10-08T09:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.408+0000 7fdaad5c1700 1 -- start start 2023-10-08T09:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaad5c1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaa80ff6a0 0x7fdaa8068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaad5c1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8100890 0x7fdaa819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaad5c1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa8101230 0x7fdaa819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaad5c1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa819e2a0 con 0x7fdaa8100890 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaad5c1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa819e410 con 0x7fdaa8101230 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaad5c1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa819e5d0 con 0x7fdaa80ff6a0 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8100890 0x7fdaa819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8100890 0x7fdaa819b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49762/0 (socket says 172.21.15.49:49762) 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 -- 172.21.15.49:0/1253339793 learned_addr learned my addr 172.21.15.49:0/1253339793 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa77fe700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa8101230 0x7fdaa819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaa80ff6a0 0x7fdaa8068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 -- 172.21.15.49:0/1253339793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaa80ff6a0 msgr2=0x7fdaa8068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaa80ff6a0 0x7fdaa8068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 -- 172.21.15.49:0/1253339793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa8101230 msgr2=0x7fdaa819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa8101230 0x7fdaa819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 -- 172.21.15.49:0/1253339793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0009d00 con 0x7fdaa8100890 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaa80ff6a0 0x7fdaa8068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8100890 0x7fdaa819b490 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c9b0 tx=0x7fda9800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fda8ffff700 1 -- 172.21.15.49:0/1253339793 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98007b00 con 0x7fdaa8100890 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.409+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa819e850 con 0x7fdaa8100890 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.410+0000 7fda8ffff700 1 -- 172.21.15.49:0/1253339793 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda98007c60 con 0x7fdaa8100890 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.410+0000 7fda8ffff700 1 -- 172.21.15.49:0/1253339793 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98018660 con 0x7fdaa8100890 2023-10-08T09:14:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.410+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa819edb0 con 0x7fdaa8100890 2023-10-08T09:14:53.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.412+0000 7fda8ffff700 1 -- 172.21.15.49:0/1253339793 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fda9800f450 con 0x7fdaa8100890 2023-10-08T09:14:53.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.412+0000 7fda8ffff700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fda900a1480 0x7fda900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.413+0000 7fda8ffff700 1 -- 172.21.15.49:0/1253339793 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fda980cd040 con 0x7fdaa8100890 2023-10-08T09:14:53.420 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.413+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fda900a1480 0x7fda900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.413+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fda900a1480 0x7fda900a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdaa81006f0 tx=0x7fda9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.413+0000 7fdab97aab80 1 Processor -- start 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.413+0000 7fdab97aab80 1 -- start start 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.413+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.413+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bd1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd17b0 0x559481bd1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdab97aab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481b38410 con 0x559481bd0010 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdab97aab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481b29bd0 con 0x559481bd17b0 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdab97aab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481b274f0 con 0x559481bd0970 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49768/0 (socket says 172.21.15.49:49768) 2023-10-08T09:14:53.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/1443224114 learned_addr learned my addr 172.21.15.49:0/1443224114 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bd1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa77fe700 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd17b0 0x559481bd1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/1443224114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 msgr2=0x559481bd1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bd1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/1443224114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd17b0 msgr2=0x559481bd1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd17b0 0x559481bd1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/1443224114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55948194dd90 con 0x559481bd0010 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd0430 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fda9c011ed0 tx=0x7fda9c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11a6a22b5f61eeb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fda8d7fa700 1 -- 172.21.15.49:0/1443224114 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c01f620 con 0x559481bd0010 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fda8d7fa700 1 -- 172.21.15.49:0/1443224114 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda9c016070 con 0x559481bd0010 2023-10-08T09:14:53.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.414+0000 7fda8d7fa700 1 -- 172.21.15.49:0/1443224114 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c01d9e0 con 0x559481bd0010 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 -- 172.21.15.49:0/1443224114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 msgr2=0x559481bd0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd0430 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fda9c011ed0 tx=0x7fda9c014000 comp rx=0 tx=0).stop 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 -- 172.21.15.49:0/1443224114 shutdown_connections 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd17b0 0x559481bd1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd0430 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 --2- 172.21.15.49:0/1443224114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bd1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 -- 172.21.15.49:0/1443224114 >> 172.21.15.49:0/1443224114 conn(0x559481b44210 msgr2=0x559481bcadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 -- 172.21.15.49:0/1443224114 shutdown_connections 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 -- 172.21.15.49:0/1443224114 wait complete. 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 -- start start 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.415+0000 7fdab97aab80 1 Processor -- start 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bda2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd17b0 0x559481be1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdab97aab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481b22de0 con 0x559481bd17b0 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdab97aab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481abf6c0 con 0x559481bd0010 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdab97aab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481abf360 con 0x559481bd0970 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdab97aab80 10 client.0 _ll_register_callbacks cb 0x55948191c7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd17b0 0x559481be1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd17b0 0x559481be1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49778/0 (socket says 172.21.15.49:49778) 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 -- 172.21.15.49:0/2367725595 learned_addr learned my addr 172.21.15.49:0/2367725595 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bda2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 -- 172.21.15.49:0/2367725595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 msgr2=0x559481bda2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.424 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bda2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 -- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 msgr2=0x559481bd9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 -- 172.21.15.49:0/2367725595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c011c60 con 0x559481bd17b0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bda2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fdaa77fe700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd17b0 0x559481be1300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0015000 tx=0x7fdaa0007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917c09e086eb1d2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.416+0000 7fda8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fda8aff5700 1 -- 172.21.15.49:0/2367725595 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000e040 con 0x559481bd17b0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fda8aff5700 1 -- 172.21.15.49:0/2367725595 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdaa0008e90 con 0x559481bd17b0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559481aeae60 con 0x559481bd17b0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fda8aff5700 1 -- 172.21.15.49:0/2367725595 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa0003a00 con 0x559481bd17b0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 10 client.15609 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559481936c10 con 0x559481bd17b0 2023-10-08T09:14:53.425 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fda8aff5700 1 -- 172.21.15.49:0/2367725595 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559481aeae60 con 0x559481bd17b0 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fda8aff5700 1 -- 172.21.15.49:0/2367725595 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdaa000b100 con 0x559481bd17b0 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 10 client.15609 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 10 client.15609 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 20 client.15609 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481be5a20 0x559481be44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 4 client.15609 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.417+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559481b1bdc0 con 0x559481be5a20 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.418+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481be5a20 0x559481be44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.418+0000 7fda8aff5700 10 client.15609 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.418+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481be5a20 0x559481be44f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc3afce93882e68 server_cookie=ede526be07143314 in_seq=0 out_seq=0 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fda8aff5700 1 -- 172.21.15.49:0/2367725595 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559481b1bdc0 con 0x559481be5a20 2023-10-08T09:14:53.426 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fda8aff5700 10 client.15609 handle_command_reply: tid=0 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 client.15609 shutdown 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd17b0 msgr2=0x559481be1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd17b0 0x559481be1300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0015000 tx=0x7fdaa0007710 comp rx=0 tx=0).stop 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 shutdown_connections 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481be5a20 0x559481be44f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd17b0 0x559481be1300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2367725595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd0970 0x559481bda2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fda8aff5700 0 client.15609 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 >> 172.21.15.49:0/2367725595 conn(0x559481b44210 msgr2=0x559481bca480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 shutdown_connections 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 1 -- 172.21.15.49:0/2367725595 wait complete. 2023-10-08T09:14:53.427 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.419+0000 7fdab97aab80 20 client.15609 trim_cache size 0 max 16384 2023-10-08T09:14:53.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.436+0000 7fdab97aab80 1 Processor -- start 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.436+0000 7fdab97aab80 1 -- start start 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.436+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.436+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481bd4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdab97aab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594819a38d0 con 0x559481bd0970 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdab97aab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481bcbcd0 con 0x559481bd0010 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdab97aab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481b1bdc0 con 0x559481bd17b0 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49784/0 (socket says 172.21.15.49:49784) 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 -- 172.21.15.49:0/556984642 learned_addr learned my addr 172.21.15.49:0/556984642 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa77fe700 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481bd4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 -- 172.21.15.49:0/556984642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 msgr2=0x559481bd4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481bd4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 -- 172.21.15.49:0/556984642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 msgr2=0x559481bd7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 -- 172.21.15.49:0/556984642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559481974200 con 0x559481bd0970 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0970 0x559481be1220 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fda980969f0 tx=0x7fda9800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f51d2404c4026b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fda8dffb700 1 -- 172.21.15.49:0/556984642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98007dd0 con 0x559481bd0970 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fda8dffb700 1 -- 172.21.15.49:0/556984642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda980b8270 con 0x559481bd0970 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.437+0000 7fda8dffb700 1 -- 172.21.15.49:0/556984642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98098490 con 0x559481bd0970 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 -- 172.21.15.49:0/556984642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0970 msgr2=0x559481be1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0970 0x559481be1220 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fda980969f0 tx=0x7fda9800fc30 comp rx=0 tx=0).stop 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 -- 172.21.15.49:0/556984642 shutdown_connections 2023-10-08T09:14:53.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481bd4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0010 0x559481bd7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 --2- 172.21.15.49:0/556984642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 -- 172.21.15.49:0/556984642 >> 172.21.15.49:0/556984642 conn(0x559481b44210 msgr2=0x559481bd9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 -- 172.21.15.49:0/556984642 shutdown_connections 2023-10-08T09:14:53.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 -- 172.21.15.49:0/556984642 wait complete. 2023-10-08T09:14:53.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 -- start start 2023-10-08T09:14:53.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.438+0000 7fdab97aab80 1 Processor -- start 2023-10-08T09:14:53.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481c39410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481c39950 con 0x559481bd0010 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481c39a90 con 0x559481bd0970 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559481c39bd0 con 0x559481bd17b0 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 10 client.0 _ll_register_callbacks cb 0x7fdaa0008c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49788/0 (socket says 172.21.15.49:49788) 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/2358084431 learned_addr learned my addr 172.21.15.49:0/2358084431 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa77fe700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481c39410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/2358084431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 msgr2=0x559481c39410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481c39410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0970 msgr2=0x559481be1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 -- 172.21.15.49:0/2358084431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa819edb0 con 0x559481bd0010 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdaa6ffd700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd6e80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fda9c022040 tx=0x7fda9c0365a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291367804d93186e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fda8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fda8aff5700 1 -- 172.21.15.49:0/2358084431 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9c02e040 con 0x559481bd0010 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fda8aff5700 1 -- 172.21.15.49:0/2358084431 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fda9c018070 con 0x559481bd0010 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda9c016070 con 0x559481bd0010 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.439+0000 7fda8aff5700 1 -- 172.21.15.49:0/2358084431 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda9c0051c0 con 0x559481bd0010 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 10 client.15615 fetch_fsmap learned FSMap version 28 2023-10-08T09:14:53.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55948194c260 con 0x559481bd0010 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fda8aff5700 1 -- 172.21.15.49:0/2358084431 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda9c016070 con 0x559481bd0010 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fda8aff5700 1 -- 172.21.15.49:0/2358084431 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda9c004d50 con 0x559481bd0010 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 10 client.15615 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 10 client.15615 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 20 client.15615 populate_metadata read hostname 'smithi049' 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481c3ef00 0x559481c3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 4 client.15615 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559481b1bae0 con 0x559481c3ef00 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.440+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481c3ef00 0x559481c3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdaa67fc700 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481c3ef00 0x559481c3b320 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7087f1d8180f2c1c server_cookie=96caf1a37052e9fb in_seq=0 out_seq=0 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fda8aff5700 10 client.15615 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fda8aff5700 1 -- 172.21.15.49:0/2358084431 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x559481b1bae0 con 0x559481c3ef00 2023-10-08T09:14:53.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fda8aff5700 10 client.15615 handle_command_reply: tid=0 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 client.15615 shutdown 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 msgr2=0x559481bd6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd6e80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fda9c022040 tx=0x7fda9c0365a0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 shutdown_connections 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559481c3ef00 0x559481c3b320 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559481bd17b0 0x559481c39410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559481bd0010 0x559481bd6e80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 --2- 172.21.15.49:0/2358084431 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559481bd0970 0x559481be1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fda8aff5700 0 client.15615 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 >> 172.21.15.49:0/2358084431 conn(0x559481b44210 msgr2=0x559481bd7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 shutdown_connections 2023-10-08T09:14:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 1 -- 172.21.15.49:0/2358084431 wait complete. 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.441+0000 7fdab97aab80 20 client.15615 trim_cache size 0 max 16384 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "b8694e94-65b9-11ee-8db5-212e2dc638e7", 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 24638, 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2023-10-08T09:14:53.450 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 109.722836089, 2023-10-08T09:14:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 28, 2023-10-08T09:14:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 90, 2023-10-08T09:14:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 86, 2023-10-08T09:14:53.451 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 117.228237171 2023-10-08T09:14:53.451 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fda900a1480 msgr2=0x7fda900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fda900a1480 0x7fda900a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdaa81006f0 tx=0x7fda9c009260 comp rx=0 tx=0).stop 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8100890 msgr2=0x7fdaa819b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8100890 0x7fdaa819b490 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c9b0 tx=0x7fda9800ccc0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 shutdown_connections 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fda900a1480 0x7fda900a3940 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdaa80ff6a0 0x7fdaa8068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdaa8101230 0x7fdaa819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 --2- 172.21.15.49:0/1253339793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdaa8100890 0x7fdaa819b490 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 >> 172.21.15.49:0/1253339793 conn(0x7fdaa80fac10 msgr2=0x7fdaa8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 shutdown_connections 2023-10-08T09:14:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:53.442+0000 7fdaad5c1700 1 -- 172.21.15.49:0/1253339793 wait complete. 2023-10-08T09:14:53.455 INFO:teuthology.run_tasks:Running task kclient... 2023-10-08T09:14:53.468 INFO:tasks.kclient:Mounting kernel clients... 2023-10-08T09:14:53.468 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-08T09:14:53.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:53.468 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-10-08T09:14:53.486 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-08T09:14:53.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:53.487 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T09:14:53.550 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:53 smithi049 ceph-mon[99485]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T09:14:53.550 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:53 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2568830679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:53.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:53.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-10-08T09:14:53.669 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-10-08T09:14:53.669 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T09:14:53.669 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-10-08T09:14:53.669 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-10-08T09:14:53.669 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph fs ls 2023-10-08T09:14:53.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:53 smithi184 ceph-mon[103675]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T09:14:53.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:53 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2568830679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:53.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:53 smithi132 ceph-mon[103389]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2023-10-08T09:14:53.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:53 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2568830679' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:54.235 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:14:55.581 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.579+0000 7f9602c58700 1 -- 172.21.15.49:0/841431510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc101260 msgr2=0x7f95fc10d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.579+0000 7f9602c58700 1 --2- 172.21.15.49:0/841431510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc101260 0x7f95fc10d650 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f95f400b0a0 tx=0x7f95f400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 -- 172.21.15.49:0/841431510 shutdown_connections 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 --2- 172.21.15.49:0/841431510 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc1057c0 0x7f95fc105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 --2- 172.21.15.49:0/841431510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc101260 0x7f95fc10d650 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 --2- 172.21.15.49:0/841431510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc1008a0 0x7f95fc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 -- 172.21.15.49:0/841431510 >> 172.21.15.49:0/841431510 conn(0x7f95fc0779e0 msgr2=0x7f95fc077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 -- 172.21.15.49:0/841431510 shutdown_connections 2023-10-08T09:14:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 -- 172.21.15.49:0/841431510 wait complete. 2023-10-08T09:14:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.580+0000 7f9602c58700 1 Processor -- start 2023-10-08T09:14:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f9602c58700 1 -- start start 2023-10-08T09:14:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f9602c58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc1008a0 0x7f95fc072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f9602c58700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc101260 0x7f95fc073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f9602c58700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc1057c0 0x7f95fc073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f9602c58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc1a5140 con 0x7f95fc1008a0 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f9602c58700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc1a52b0 con 0x7f95fc101260 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f9602c58700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc1a5470 con 0x7f95fc1057c0 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc101260 0x7f95fc073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc101260 0x7f95fc073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:39416/0 (socket says 172.21.15.49:39416) 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 -- 172.21.15.49:0/1374666159 learned_addr learned my addr 172.21.15.49:0/1374666159 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f96009f4700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc1008a0 0x7f95fc072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f96011f5700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc1057c0 0x7f95fc073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 -- 172.21.15.49:0/1374666159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc1057c0 msgr2=0x7f95fc073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc1057c0 0x7f95fc073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 -- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc1008a0 msgr2=0x7f95fc072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc1008a0 0x7f95fc072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f95fbfff700 1 -- 172.21.15.49:0/1374666159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009d00 con 0x7f95fc101260 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.581+0000 7f96009f4700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc1008a0 0x7f95fc072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.582+0000 7f95fbfff700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc101260 0x7f95fc073030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95e800d950 tx=0x7f95e800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.582+0000 7f95f9ffb700 1 -- 172.21.15.49:0/1374666159 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8009940 con 0x7f95fc101260 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.582+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc1a5750 con 0x7f95fc101260 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.582+0000 7f95f9ffb700 1 -- 172.21.15.49:0/1374666159 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95e8010460 con 0x7f95fc101260 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.582+0000 7f95f9ffb700 1 -- 172.21.15.49:0/1374666159 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e800f620 con 0x7f95fc101260 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.582+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc1a5c50 con 0x7f95fc101260 2023-10-08T09:14:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.583+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc108d80 con 0x7f95fc101260 2023-10-08T09:14:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.585+0000 7f95f9ffb700 1 -- 172.21.15.49:0/1374666159 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f95e80108d0 con 0x7f95fc101260 2023-10-08T09:14:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.585+0000 7f95f9ffb700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f95ec0a1480 0x7f95ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.585+0000 7f96009f4700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f95ec0a1480 0x7f95ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.585+0000 7f95f9ffb700 1 -- 172.21.15.49:0/1374666159 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f95e80cd120 con 0x7f95fc101260 2023-10-08T09:14:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.586+0000 7f96009f4700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f95ec0a1480 0x7f95ec0a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95fc101930 tx=0x7f95f0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.589+0000 7f95f9ffb700 1 -- 172.21.15.49:0/1374666159 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f95e8096450 con 0x7f95fc101260 2023-10-08T09:14:55.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:55 smithi184 ceph-mon[103675]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:55.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:55 smithi132 ceph-mon[103389]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:55.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.837+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f95fc073fd0 con 0x7f95fc101260 2023-10-08T09:14:55.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.837+0000 7f95f9ffb700 1 -- 172.21.15.49:0/1374666159 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f95e8016070 con 0x7f95fc101260 2023-10-08T09:14:55.839 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f95ec0a1480 msgr2=0x7f95ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f95ec0a1480 0x7f95ec0a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f95fc101930 tx=0x7f95f0009190 comp rx=0 tx=0).stop 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc101260 msgr2=0x7f95fc073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc101260 0x7f95fc073030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f95e800d950 tx=0x7f95e800dc60 comp rx=0 tx=0).stop 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 shutdown_connections 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f95ec0a1480 0x7f95ec0a3940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f95fc1057c0 0x7f95fc073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95fc1008a0 0x7f95fc072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 --2- 172.21.15.49:0/1374666159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f95fc101260 0x7f95fc073030 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:55.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 >> 172.21.15.49:0/1374666159 conn(0x7f95fc0779e0 msgr2=0x7f95fc0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:55.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 shutdown_connections 2023-10-08T09:14:55.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:55.839+0000 7f9602c58700 1 -- 172.21.15.49:0/1374666159 wait complete. 2023-10-08T09:14:55.868 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:55 smithi049 ceph-mon[99485]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T09:14:56.285 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T09:14:56.286 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-10-08T09:14:56.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.286 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2023-10-08T09:14:56.305 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T09:14:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T09:14:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T09:14:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:14:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2023-10-08T09:14:56.305 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 41765sec preferred_lft 41765sec 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2023-10-08T09:14:56.306 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T09:14:56.307 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2023-10-08T09:14:56.307 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T09:14:56.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.307 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:14:56.307 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:14:56.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2023-10-08T09:14:56.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2023-10-08T09:14:56.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2023-10-08T09:14:56.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T09:14:56.308 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:14:56.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.429 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T09:14:56.457 INFO:teuthology.orchestra.run.smithi049.stdout:1 2023-10-08T09:14:56.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.458 DEBUG:teuthology.orchestra.run.smithi049:> route 2023-10-08T09:14:56.519 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2023-10-08T09:14:56.519 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T09:14:56.519 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-08T09:14:56.519 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-08T09:14:56.519 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T09:14:56.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.520 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:14:56.520 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:14:56.520 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-08T09:14:56.520 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-08T09:14:56.520 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-08T09:14:56.520 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:14:56.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.629 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-10-08T09:14:56.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.646 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2023-10-08T09:14:56.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.704 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:14:56.704 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:14:56.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--tmp-cephfs 2023-10-08T09:14:56.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-10-08T09:14:56.704 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:14:56.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:56 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1374666159' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:14:56.783 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:56 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1374666159' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:14:56.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:56 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1374666159' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-08T09:14:56.800 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-10-08T09:14:56.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-10-08T09:14:56.801 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:14:57.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:57.034 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec bash -c ' 2023-10-08T09:14:57.034 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2023-10-08T09:14:57.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2023-10-08T09:14:57.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T09:14:57.034 DEBUG:teuthology.orchestra.run.smithi049:> ') 2023-10-08T09:14:57.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:14:57.141 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /tmp/cephfs 2023-10-08T09:14:57.166 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/tmp/cephfs' 2023-10-08T09:14:57.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T09:14:57.167 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /tmp/cephfs 2023-10-08T09:14:57.223 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-08T09:14:57.223 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-08T09:14:57.223 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/485a435c-3760-47ed-ad82-bcf59bb1ae2b 2023-10-08T09:14:57.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:57.224 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/485a435c-3760-47ed-ad82-bcf59bb1ae2b /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-10-08T09:14:57.309 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-10-08T09:14:57.310 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-08T09:14:57.490 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-10-08T09:14:57.490 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-08T09:14:57.490 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.132:3300,172.21.15.184:3300" 2023-10-08T09:14:57.491 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 0@b8694e94-65b9-11ee-8db5-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/485a435c-3760-47ed-ad82-bcf59bb1ae2b 2023-10-08T09:14:57.491 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.49:3300/172.21.15.132:3300/172.21.15.184:3300" will pass to kernel 2023-10-08T09:14:57.491 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T09:14:57.491 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T09:14:57.491 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.49:3300,172.21.15.132:3300,172.21.15.184:3300:/volumes/_nogroup/sv_1/485a435c-3760-47ed-ad82-bcf59bb1ae2b 2023-10-08T09:14:57.491 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mds_namespace=cephfs,fsid=b8694e94-65b9-11ee-8db5-212e2dc638e7" will pass to kernel 2023-10-08T09:14:57.494 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T09:14:57.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:57.494 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /tmp/cephfs 2023-10-08T09:14:57.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:57.528 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T09:14:57.602 INFO:teuthology.orchestra.run.smithi049.stdout:client15624 2023-10-08T09:14:57.602 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:14:57.603 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:14:57.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:57 smithi049 ceph-mon[99485]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:57.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:57.650 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T09:14:57.724 INFO:teuthology.orchestra.run.smithi049.stdout:client15624 2023-10-08T09:14:57.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:57.724 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T09:14:57.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:57 smithi184 ceph-mon[103675]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:57.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:57 smithi132 ceph-mon[103389]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-08T09:14:57.797 INFO:teuthology.orchestra.run.smithi049.stdout:client15624 2023-10-08T09:14:57.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:14:57.798 DEBUG:teuthology.orchestra.run.smithi049:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7.client15624/status') 2023-10-08T09:14:57.881 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.15624 (3)192.168.0.1:0/3811639912 2023-10-08T09:14:57.881 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2023-10-08T09:14:57.882 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2023-10-08T09:14:57.882 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2023-10-08T09:14:57.882 INFO:teuthology.orchestra.run.smithi049.stderr:70 bytes copied, 2.4553e-05 s, 2.9 MB/s 2023-10-08T09:14:57.883 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-08T09:14:57.895 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-08T09:14:57.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 -- 172.21.15.49:0/566184688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d8102250 msgr2=0x7ff6d8108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 --2- 172.21.15.49:0/566184688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d8102250 0x7ff6d8108b00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b0d0 tx=0x7ff6d000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 -- 172.21.15.49:0/566184688 shutdown_connections 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 --2- 172.21.15.49:0/566184688 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d8102250 0x7ff6d8108b00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 --2- 172.21.15.49:0/566184688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6d81006c0 0x7ff6d8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 --2- 172.21.15.49:0/566184688 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d81018b0 0x7ff6d8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 -- 172.21.15.49:0/566184688 >> 172.21.15.49:0/566184688 conn(0x7ff6d80fbc50 msgr2=0x7ff6d80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:58.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 -- 172.21.15.49:0/566184688 shutdown_connections 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.047+0000 7ff6dead9700 1 -- 172.21.15.49:0/566184688 wait complete. 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 Processor -- start 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 -- start start 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d81006c0 0x7ff6d8196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6d81018b0 0x7ff6d8196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d8102250 0x7ff6d819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819e280 con 0x7ff6d81006c0 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819e3c0 con 0x7ff6d81018b0 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dead9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d819e550 con 0x7ff6d8102250 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d81006c0 0x7ff6d8196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d81006c0 0x7ff6d8196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49820/0 (socket says 172.21.15.49:49820) 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 -- 172.21.15.49:0/1664869285 learned_addr learned my addr 172.21.15.49:0/1664869285 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6d7fff700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6d81018b0 0x7ff6d8196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 -- 172.21.15.49:0/1664869285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d8102250 msgr2=0x7ff6d819dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d8102250 0x7ff6d819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 -- 172.21.15.49:0/1664869285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6d81018b0 msgr2=0x7ff6d8196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6d81018b0 0x7ff6d8196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6dc875700 1 -- 172.21.15.49:0/1664869285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d0009d00 con 0x7ff6d81006c0 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.048+0000 7ff6d7fff700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6d81018b0 0x7ff6d8196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.049+0000 7ff6dc875700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d81006c0 0x7ff6d8196750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00d980 tx=0x7ff6cc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.049+0000 7ff6d5ffb700 1 -- 172.21.15.49:0/1664869285 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00de60 con 0x7ff6d81006c0 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.049+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d819e830 con 0x7ff6d81006c0 2023-10-08T09:14:58.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.049+0000 7ff6d5ffb700 1 -- 172.21.15.49:0/1664869285 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6cc0044e0 con 0x7ff6d81006c0 2023-10-08T09:14:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.049+0000 7ff6d5ffb700 1 -- 172.21.15.49:0/1664869285 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc003be0 con 0x7ff6d81006c0 2023-10-08T09:14:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.049+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d819ed40 con 0x7ff6d81006c0 2023-10-08T09:14:58.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.050+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d8107a70 con 0x7ff6d81006c0 2023-10-08T09:14:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.051+0000 7ff6d5ffb700 1 -- 172.21.15.49:0/1664869285 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff6cc009bb0 con 0x7ff6d81006c0 2023-10-08T09:14:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.051+0000 7ff6d5ffb700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6c80a1480 0x7ff6c80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.052+0000 7ff6d5ffb700 1 -- 172.21.15.49:0/1664869285 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff6cc0cce20 con 0x7ff6d81006c0 2023-10-08T09:14:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.052+0000 7ff6d7fff700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6c80a1480 0x7ff6c80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.052+0000 7ff6d7fff700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6c80a1480 0x7ff6c80a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4005950 tx=0x7ff6c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:58.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.056+0000 7ff6d5ffb700 1 -- 172.21.15.49:0/1664869285 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff6cc0960f0 con 0x7ff6d81006c0 2023-10-08T09:14:58.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.260+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6d8066bc0 con 0x7ff6d81006c0 2023-10-08T09:14:58.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.262+0000 7ff6d5ffb700 1 -- 172.21.15.49:0/1664869285 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7ff6cc009940 con 0x7ff6d81006c0 2023-10-08T09:14:58.264 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:14:58.264 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:14:58.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6c80a1480 msgr2=0x7ff6c80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6c80a1480 0x7ff6c80a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4005950 tx=0x7ff6c40058e0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d81006c0 msgr2=0x7ff6d8196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d81006c0 0x7ff6d8196750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00d980 tx=0x7ff6cc00dd40 comp rx=0 tx=0).stop 2023-10-08T09:14:58.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 shutdown_connections 2023-10-08T09:14:58.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6c80a1480 0x7ff6c80a3940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6d8102250 0x7ff6d819dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6d81006c0 0x7ff6d8196750 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 --2- 172.21.15.49:0/1664869285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6d81018b0 0x7ff6d8196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 >> 172.21.15.49:0/1664869285 conn(0x7ff6d80fbc50 msgr2=0x7ff6d8104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:58.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 shutdown_connections 2023-10-08T09:14:58.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.264+0000 7ff6dead9700 1 -- 172.21.15.49:0/1664869285 wait complete. 2023-10-08T09:14:58.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:14:58.279 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/2988339742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8103b50 msgr2=0x7fa5a8105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/2988339742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8103b50 0x7fa5a8105f40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009990 tx=0x7fa59c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/2988339742 shutdown_connections 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/2988339742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a8106480 0x7fa5a81088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/2988339742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5a8101220 0x7fa5a8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/2988339742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8103b50 0x7fa5a8105f40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/2988339742 >> 172.21.15.49:0/2988339742 conn(0x7fa5a80fac10 msgr2=0x7fa5a80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:58.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/2988339742 shutdown_connections 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.407+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/2988339742 wait complete. 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 Processor -- start 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 -- start start 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8101220 0x7fa5a8196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a8103b50 0x7fa5a8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5a8106480 0x7fa5a819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a819e240 con 0x7fa5a8101220 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a819e380 con 0x7fa5a8106480 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.408+0000 7fa5ae0bd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a819e510 con 0x7fa5a8103b50 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8101220 0x7fa5a8196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8101220 0x7fa5a8196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49834/0 (socket says 172.21.15.49:49834) 2023-10-08T09:14:58.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 -- 172.21.15.49:0/420977251 learned_addr learned my addr 172.21.15.49:0/420977251 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a6ffd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a8103b50 0x7fa5a8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a7fff700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5a8106480 0x7fa5a819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 -- 172.21.15.49:0/420977251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a8103b50 msgr2=0x7fa5a8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a8103b50 0x7fa5a8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 -- 172.21.15.49:0/420977251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5a8106480 msgr2=0x7fa5a819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5a8106480 0x7fa5a819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 -- 172.21.15.49:0/420977251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c0095f0 con 0x7fa5a8101220 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a77fe700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8101220 0x7fa5a8196710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa59800d980 tx=0x7fa59800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a4ff9700 1 -- 172.21.15.49:0/420977251 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598004220 con 0x7fa5a8101220 2023-10-08T09:14:58.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a4ff9700 1 -- 172.21.15.49:0/420977251 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa598007640 con 0x7fa5a8101220 2023-10-08T09:14:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a819e7f0 con 0x7fa5a8101220 2023-10-08T09:14:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5a4ff9700 1 -- 172.21.15.49:0/420977251 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598005610 con 0x7fa5a8101220 2023-10-08T09:14:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.409+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a819ed00 con 0x7fa5a8101220 2023-10-08T09:14:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.410+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a80fc790 con 0x7fa5a8101220 2023-10-08T09:14:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.411+0000 7fa5a4ff9700 1 -- 172.21.15.49:0/420977251 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa598004750 con 0x7fa5a8101220 2023-10-08T09:14:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.412+0000 7fa5a4ff9700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa5900a1480 0x7fa5900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.412+0000 7fa5a6ffd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa5900a1480 0x7fa5900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.412+0000 7fa5a4ff9700 1 -- 172.21.15.49:0/420977251 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa5980cd320 con 0x7fa5a8101220 2023-10-08T09:14:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.412+0000 7fa5a6ffd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa5900a1480 0x7fa5900a3940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b5c0 tx=0x7fa59c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:58.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.415+0000 7fa5a4ff9700 1 -- 172.21.15.49:0/420977251 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa5980966f0 con 0x7fa5a8101220 2023-10-08T09:14:58.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.619+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5a8067500 con 0x7fa5a8101220 2023-10-08T09:14:58.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.620+0000 7fa5a4ff9700 1 -- 172.21.15.49:0/420977251 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa598016020 con 0x7fa5a8101220 2023-10-08T09:14:58.621 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:14:58.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa5900a1480 msgr2=0x7fa5900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa5900a1480 0x7fa5900a3940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00b5c0 tx=0x7fa59c005fd0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8101220 msgr2=0x7fa5a8196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8101220 0x7fa5a8196710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa59800d980 tx=0x7fa59800dc90 comp rx=0 tx=0).stop 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 shutdown_connections 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa5900a1480 0x7fa5900a3940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5a8106480 0x7fa5a819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa5a8101220 0x7fa5a8196710 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 --2- 172.21.15.49:0/420977251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa5a8103b50 0x7fa5a8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 >> 172.21.15.49:0/420977251 conn(0x7fa5a80fac10 msgr2=0x7fa5a81030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 shutdown_connections 2023-10-08T09:14:58.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.622+0000 7fa5ae0bd700 1 -- 172.21.15.49:0/420977251 wait complete. 2023-10-08T09:14:58.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:58 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1664869285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:58.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:14:58.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:58 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1664869285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:58.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 -- 172.21.15.49:0/2529000155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f10102350 msgr2=0x7f3f10108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 --2- 172.21.15.49:0/2529000155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f10102350 0x7f3f10108c00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800b0d0 tx=0x7f3f0800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 -- 172.21.15.49:0/2529000155 shutdown_connections 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 --2- 172.21.15.49:0/2529000155 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f10102350 0x7f3f10108c00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 --2- 172.21.15.49:0/2529000155 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f101007c0 0x7f3f10100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 --2- 172.21.15.49:0/2529000155 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f101019b0 0x7f3f10101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 -- 172.21.15.49:0/2529000155 >> 172.21.15.49:0/2529000155 conn(0x7f3f100fbd30 msgr2=0x7f3f100fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 -- 172.21.15.49:0/2529000155 shutdown_connections 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.764+0000 7f3f173a7700 1 -- 172.21.15.49:0/2529000155 wait complete. 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 Processor -- start 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 -- start start 2023-10-08T09:14:58.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f101007c0 0x7f3f10196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f101019b0 0x7f3f10196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f10102350 0x7f3f1019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1019e2b0 con 0x7f3f101019b0 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1019e3f0 con 0x7f3f101007c0 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f173a7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1019e580 con 0x7f3f10102350 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f101019b0 0x7f3f10196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f101019b0 0x7f3f10196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49850/0 (socket says 172.21.15.49:49850) 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 -- 172.21.15.49:0/3470299519 learned_addr learned my addr 172.21.15.49:0/3470299519 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f15143700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f101007c0 0x7f3f10196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f15944700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f10102350 0x7f3f1019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 -- 172.21.15.49:0/3470299519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f10102350 msgr2=0x7f3f1019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f10102350 0x7f3f1019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 -- 172.21.15.49:0/3470299519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f101007c0 msgr2=0x7f3f10196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f101007c0 0x7f3f10196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 -- 172.21.15.49:0/3470299519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f08009d00 con 0x7f3f101019b0 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f15944700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f10102350 0x7f3f1019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.765+0000 7f3f14942700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f101019b0 0x7f3f10196cc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00c9b0 tx=0x7f3f0c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.766+0000 7f3f067fc700 1 -- 172.21.15.49:0/3470299519 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c007b00 con 0x7f3f101019b0 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.766+0000 7f3f067fc700 1 -- 172.21.15.49:0/3470299519 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f0c007c60 con 0x7f3f101019b0 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.766+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1019e860 con 0x7f3f101019b0 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.766+0000 7f3f067fc700 1 -- 172.21.15.49:0/3470299519 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c018660 con 0x7f3f101019b0 2023-10-08T09:14:58.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.766+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1019ed70 con 0x7f3f101019b0 2023-10-08T09:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.767+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10107b70 con 0x7f3f101019b0 2023-10-08T09:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.768+0000 7f3f067fc700 1 -- 172.21.15.49:0/3470299519 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3f0c00f450 con 0x7f3f101019b0 2023-10-08T09:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.768+0000 7f3f067fc700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3efc0a1480 0x7f3efc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.769+0000 7f3f15143700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3efc0a1480 0x7f3efc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:58.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.769+0000 7f3f067fc700 1 -- 172.21.15.49:0/3470299519 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3f0c0cd330 con 0x7f3f101019b0 2023-10-08T09:14:58.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.769+0000 7f3f15143700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3efc0a1480 0x7f3efc0a3940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3f10101810 tx=0x7f3f0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:58.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.772+0000 7f3f067fc700 1 -- 172.21.15.49:0/3470299519 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f0c096690 con 0x7f3f101019b0 2023-10-08T09:14:58.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:58 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1664869285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:14:58.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.976+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f1010bfe0 con 0x7f3f101019b0 2023-10-08T09:14:58.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.976+0000 7f3f067fc700 1 -- 172.21.15.49:0/3470299519 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f0c099cb0 con 0x7f3f101019b0 2023-10-08T09:14:58.978 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3efc0a1480 msgr2=0x7f3efc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.980 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3efc0a1480 0x7f3efc0a3940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3f10101810 tx=0x7f3f0000b410 comp rx=0 tx=0).stop 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f101019b0 msgr2=0x7f3f10196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f101019b0 0x7f3f10196cc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00c9b0 tx=0x7f3f0c00ccc0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 shutdown_connections 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3efc0a1480 0x7f3efc0a3940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f10102350 0x7f3f1019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f101007c0 0x7f3f10196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 --2- 172.21.15.49:0/3470299519 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f101019b0 0x7f3f10196cc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 >> 172.21.15.49:0/3470299519 conn(0x7f3f100fbd30 msgr2=0x7f3f10104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 shutdown_connections 2023-10-08T09:14:58.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:58.979+0000 7f3f173a7700 1 -- 172.21.15.49:0/3470299519 wait complete. 2023-10-08T09:14:58.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:14:59.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 -- 172.21.15.49:0/2788154258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c80ff6a0 msgr2=0x7fd3c80ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 --2- 172.21.15.49:0/2788154258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c80ff6a0 0x7fd3c80ffac0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80099c0 tx=0x7fd3b8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 -- 172.21.15.49:0/2788154258 shutdown_connections 2023-10-08T09:14:59.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 --2- 172.21.15.49:0/2788154258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c8101230 0x7fd3c8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 --2- 172.21.15.49:0/2788154258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c80ff6a0 0x7fd3c80ffac0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 --2- 172.21.15.49:0/2788154258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3c8100890 0x7fd3c8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 -- 172.21.15.49:0/2788154258 >> 172.21.15.49:0/2788154258 conn(0x7fd3c80fac10 msgr2=0x7fd3c80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:59.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 -- 172.21.15.49:0/2788154258 shutdown_connections 2023-10-08T09:14:59.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.120+0000 7fd3d050c700 1 -- 172.21.15.49:0/2788154258 wait complete. 2023-10-08T09:14:59.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 Processor -- start 2023-10-08T09:14:59.123 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 -- start start 2023-10-08T09:14:59.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3c80ff6a0 0x7fd3c8195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c8100890 0x7fd3c8195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c8101230 0x7fd3c819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c819d1a0 con 0x7fd3c8100890 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c819d2e0 con 0x7fd3c80ff6a0 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3d050c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c819d470 con 0x7fd3c8101230 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c8100890 0x7fd3c8195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c8100890 0x7fd3c8195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49864/0 (socket says 172.21.15.49:49864) 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 -- 172.21.15.49:0/1862017132 learned_addr learned my addr 172.21.15.49:0/1862017132 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3ce2a8700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3c80ff6a0 0x7fd3c8195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 -- 172.21.15.49:0/1862017132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c8101230 msgr2=0x7fd3c819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3ceaa9700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c8101230 0x7fd3c819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c8101230 0x7fd3c819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 -- 172.21.15.49:0/1862017132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3c80ff6a0 msgr2=0x7fd3c8195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3c80ff6a0 0x7fd3c8195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.121+0000 7fd3cdaa7700 1 -- 172.21.15.49:0/1862017132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b80095f0 con 0x7fd3c8100890 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.122+0000 7fd3ce2a8700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3c80ff6a0 0x7fd3c8195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.122+0000 7fd3cdaa7700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c8100890 0x7fd3c8195bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400c920 tx=0x7fd3c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.122+0000 7fd3bf7fe700 1 -- 172.21.15.49:0/1862017132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4007b00 con 0x7fd3c8100890 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.122+0000 7fd3bf7fe700 1 -- 172.21.15.49:0/1862017132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3c4004cf0 con 0x7fd3c8100890 2023-10-08T09:14:59.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.122+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c819d750 con 0x7fd3c8100890 2023-10-08T09:14:59.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.122+0000 7fd3bf7fe700 1 -- 172.21.15.49:0/1862017132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4005610 con 0x7fd3c8100890 2023-10-08T09:14:59.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.122+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c819dc60 con 0x7fd3c8100890 2023-10-08T09:14:59.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.123+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c8106b70 con 0x7fd3c8100890 2023-10-08T09:14:59.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.124+0000 7fd3bf7fe700 1 -- 172.21.15.49:0/1862017132 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd3c4004750 con 0x7fd3c8100890 2023-10-08T09:14:59.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.125+0000 7fd3bf7fe700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd3b40a1480 0x7fd3b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.125+0000 7fd3ce2a8700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd3b40a1480 0x7fd3b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.125+0000 7fd3bf7fe700 1 -- 172.21.15.49:0/1862017132 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd3c40cd090 con 0x7fd3c8100890 2023-10-08T09:14:59.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.125+0000 7fd3ce2a8700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd3b40a1480 0x7fd3b40a3940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b580 tx=0x7fd3b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:59.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.128+0000 7fd3bf7fe700 1 -- 172.21.15.49:0/1862017132 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd3c4096360 con 0x7fd3c8100890 2023-10-08T09:14:59.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.332+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3c8106550 con 0x7fd3c8100890 2023-10-08T09:14:59.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.332+0000 7fd3bf7fe700 1 -- 172.21.15.49:0/1862017132 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3c401d020 con 0x7fd3c8100890 2023-10-08T09:14:59.333 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd3b40a1480 msgr2=0x7fd3b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd3b40a1480 0x7fd3b40a3940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b580 tx=0x7fd3b8005fb0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c8100890 msgr2=0x7fd3c8195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c8100890 0x7fd3c8195bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400c920 tx=0x7fd3c400cce0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 shutdown_connections 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd3b40a1480 0x7fd3b40a3940 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd3c8101230 0x7fd3c819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd3c80ff6a0 0x7fd3c8195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 --2- 172.21.15.49:0/1862017132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd3c8100890 0x7fd3c8195bb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 >> 172.21.15.49:0/1862017132 conn(0x7fd3c80fac10 msgr2=0x7fd3c8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:59.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 shutdown_connections 2023-10-08T09:14:59.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.334+0000 7fd3d050c700 1 -- 172.21.15.49:0/1862017132 wait complete. 2023-10-08T09:14:59.347 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:14:59.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 -- 172.21.15.49:0/3946663694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce240ff6b0 msgr2=0x7fce240ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 --2- 172.21.15.49:0/3946663694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce240ff6b0 0x7fce240ffad0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fce140099c0 tx=0x7fce14009cd0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 -- 172.21.15.49:0/3946663694 shutdown_connections 2023-10-08T09:14:59.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 --2- 172.21.15.49:0/3946663694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce24101240 0x7fce24107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 --2- 172.21.15.49:0/3946663694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce240ff6b0 0x7fce240ffad0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 --2- 172.21.15.49:0/3946663694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce241008a0 0x7fce24100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 -- 172.21.15.49:0/3946663694 >> 172.21.15.49:0/3946663694 conn(0x7fce240fac40 msgr2=0x7fce240fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:59.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 -- 172.21.15.49:0/3946663694 shutdown_connections 2023-10-08T09:14:59.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.475+0000 7fce28e15700 1 -- 172.21.15.49:0/3946663694 wait complete. 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 Processor -- start 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 -- start start 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce240ff6b0 0x7fce24196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce241008a0 0x7fce24196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce24101240 0x7fce2419dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e2b0 con 0x7fce241008a0 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e3f0 con 0x7fce240ff6b0 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.476+0000 7fce28e15700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419e580 con 0x7fce24101240 2023-10-08T09:14:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce241008a0 0x7fce24196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce241008a0 0x7fce24196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49876/0 (socket says 172.21.15.49:49876) 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 -- 172.21.15.49:0/2170300395 learned_addr learned my addr 172.21.15.49:0/2170300395 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce2259c700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce240ff6b0 0x7fce24196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce22d9d700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce24101240 0x7fce2419dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 -- 172.21.15.49:0/2170300395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce24101240 msgr2=0x7fce2419dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce24101240 0x7fce2419dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 -- 172.21.15.49:0/2170300395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce240ff6b0 msgr2=0x7fce24196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce240ff6b0 0x7fce24196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 -- 172.21.15.49:0/2170300395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140095f0 con 0x7fce241008a0 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.477+0000 7fce21d9b700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce241008a0 0x7fce24196cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00d980 tx=0x7fce0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.478+0000 7fce1b7fe700 1 -- 172.21.15.49:0/2170300395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c004220 con 0x7fce241008a0 2023-10-08T09:14:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.478+0000 7fce1b7fe700 1 -- 172.21.15.49:0/2170300395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce0c007640 con 0x7fce241008a0 2023-10-08T09:14:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.478+0000 7fce1b7fe700 1 -- 172.21.15.49:0/2170300395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c005690 con 0x7fce241008a0 2023-10-08T09:14:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.478+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2419e860 con 0x7fce241008a0 2023-10-08T09:14:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.478+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2419ed70 con 0x7fce241008a0 2023-10-08T09:14:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.479+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce24067980 con 0x7fce241008a0 2023-10-08T09:14:59.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.480+0000 7fce1b7fe700 1 -- 172.21.15.49:0/2170300395 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fce0c004750 con 0x7fce241008a0 2023-10-08T09:14:59.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.481+0000 7fce1b7fe700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fce100a1500 0x7fce100a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.481+0000 7fce1b7fe700 1 -- 172.21.15.49:0/2170300395 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fce0c0cd350 con 0x7fce241008a0 2023-10-08T09:14:59.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.481+0000 7fce2259c700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fce100a1500 0x7fce100a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.481+0000 7fce2259c700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fce100a1500 0x7fce100a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fce14003730 tx=0x7fce14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:59.484 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.484+0000 7fce1b7fe700 1 -- 172.21.15.49:0/2170300395 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fce0c096620 con 0x7fce241008a0 2023-10-08T09:14:59.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:59 smithi049 ceph-mon[99485]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-08T09:14:59.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:59 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/420977251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:59 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3470299519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:14:59 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1862017132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.686+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fce2410b060 con 0x7fce241008a0 2023-10-08T09:14:59.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.687+0000 7fce1b7fe700 1 -- 172.21.15.49:0/2170300395 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fce0c016020 con 0x7fce241008a0 2023-10-08T09:14:59.687 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:14:59.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fce100a1500 msgr2=0x7fce100a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fce100a1500 0x7fce100a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fce14003730 tx=0x7fce14000bc0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce241008a0 msgr2=0x7fce24196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce241008a0 0x7fce24196cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00d980 tx=0x7fce0c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 shutdown_connections 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fce100a1500 0x7fce100a39c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce24101240 0x7fce2419dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce240ff6b0 0x7fce24196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 --2- 172.21.15.49:0/2170300395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fce241008a0 0x7fce24196cc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 >> 172.21.15.49:0/2170300395 conn(0x7fce240fac40 msgr2=0x7fce24068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 shutdown_connections 2023-10-08T09:14:59.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.689+0000 7fce28e15700 1 -- 172.21.15.49:0/2170300395 wait complete. 2023-10-08T09:14:59.702 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:14:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:59 smithi184 ceph-mon[103675]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-08T09:14:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:59 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/420977251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:59 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3470299519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:14:59 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1862017132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:59 smithi132 ceph-mon[103389]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-10-08T09:14:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:59 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/420977251' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:59 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3470299519' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:14:59 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1862017132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 -- 172.21.15.49:0/2008159407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb841023a0 msgr2=0x7fbb84108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 --2- 172.21.15.49:0/2008159407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb841023a0 0x7fbb84108c50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00b0a0 tx=0x7fbb7c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 -- 172.21.15.49:0/2008159407 shutdown_connections 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 --2- 172.21.15.49:0/2008159407 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb841023a0 0x7fbb84108c50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 --2- 172.21.15.49:0/2008159407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb840fe990 0x7fbb840fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 --2- 172.21.15.49:0/2008159407 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb84101a00 0x7fbb84101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 -- 172.21.15.49:0/2008159407 >> 172.21.15.49:0/2008159407 conn(0x7fbb840fa2c0 msgr2=0x7fbb840fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:14:59.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 -- 172.21.15.49:0/2008159407 shutdown_connections 2023-10-08T09:14:59.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.827+0000 7fbb89029700 1 -- 172.21.15.49:0/2008159407 wait complete. 2023-10-08T09:14:59.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 Processor -- start 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 -- start start 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb840fe990 0x7fbb84196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb84101a00 0x7fbb84196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb841023a0 0x7fbb8419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8419e240 con 0x7fbb84101a00 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8419e380 con 0x7fbb840fe990 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb89029700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8419e510 con 0x7fbb841023a0 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb84101a00 0x7fbb84196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb84101a00 0x7fbb84196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49890/0 (socket says 172.21.15.49:49890) 2023-10-08T09:14:59.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 -- 172.21.15.49:0/2650018930 learned_addr learned my addr 172.21.15.49:0/2650018930 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb82d9d700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb840fe990 0x7fbb84196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 -- 172.21.15.49:0/2650018930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb841023a0 msgr2=0x7fbb8419dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8359e700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb841023a0 0x7fbb8419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb841023a0 0x7fbb8419dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 -- 172.21.15.49:0/2650018930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb840fe990 msgr2=0x7fbb84196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb840fe990 0x7fbb84196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb8259c700 1 -- 172.21.15.49:0/2650018930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c009d00 con 0x7fbb84101a00 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.828+0000 7fbb82d9d700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb840fe990 0x7fbb84196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.829+0000 7fbb8259c700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb84101a00 0x7fbb84196c50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800e9a0 tx=0x7fbb7800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.829+0000 7fbb6bfff700 1 -- 172.21.15.49:0/2650018930 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78009790 con 0x7fbb84101a00 2023-10-08T09:14:59.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.829+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8419e7f0 con 0x7fbb84101a00 2023-10-08T09:14:59.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.829+0000 7fbb6bfff700 1 -- 172.21.15.49:0/2650018930 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7800cbd0 con 0x7fbb84101a00 2023-10-08T09:14:59.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.829+0000 7fbb6bfff700 1 -- 172.21.15.49:0/2650018930 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78018680 con 0x7fbb84101a00 2023-10-08T09:14:59.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.829+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8419ed50 con 0x7fbb84101a00 2023-10-08T09:14:59.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.830+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb84107bc0 con 0x7fbb84101a00 2023-10-08T09:14:59.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.831+0000 7fbb6bfff700 1 -- 172.21.15.49:0/2650018930 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbb7800cd40 con 0x7fbb84101a00 2023-10-08T09:14:59.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.831+0000 7fbb6bfff700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb6c0a1480 0x7fbb6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:14:59.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.832+0000 7fbb82d9d700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb6c0a1480 0x7fbb6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:14:59.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.832+0000 7fbb6bfff700 1 -- 172.21.15.49:0/2650018930 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbb78014070 con 0x7fbb84101a00 2023-10-08T09:14:59.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.832+0000 7fbb82d9d700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb6c0a1480 0x7fbb6c0a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb84067570 tx=0x7fbb74009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:14:59.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:14:59.836+0000 7fbb6bfff700 1 -- 172.21.15.49:0/2650018930 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb78096640 con 0x7fbb84101a00 2023-10-08T09:15:00.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.042+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb84066bc0 con 0x7fbb84101a00 2023-10-08T09:15:00.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.044+0000 7fbb6bfff700 1 -- 172.21.15.49:0/2650018930 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7fbb78099c60 con 0x7fbb84101a00 2023-10-08T09:15:00.046 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:00.046 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:15:00.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.046+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb6c0a1480 msgr2=0x7fbb6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.046+0000 7fbb89029700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb6c0a1480 0x7fbb6c0a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb84067570 tx=0x7fbb74009670 comp rx=0 tx=0).stop 2023-10-08T09:15:00.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.046+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb84101a00 msgr2=0x7fbb84196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.046+0000 7fbb89029700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb84101a00 0x7fbb84196c50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800e9a0 tx=0x7fbb7800ed60 comp rx=0 tx=0).stop 2023-10-08T09:15:00.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.046+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 shutdown_connections 2023-10-08T09:15:00.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.046+0000 7fbb89029700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbb6c0a1480 0x7fbb6c0a3940 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.047+0000 7fbb89029700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb841023a0 0x7fbb8419dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.047+0000 7fbb89029700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb840fe990 0x7fbb84196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.047+0000 7fbb89029700 1 --2- 172.21.15.49:0/2650018930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb84101a00 0x7fbb84196c50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.047+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 >> 172.21.15.49:0/2650018930 conn(0x7fbb840fa2c0 msgr2=0x7fbb84104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:00.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.047+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 shutdown_connections 2023-10-08T09:15:00.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.047+0000 7fbb89029700 1 -- 172.21.15.49:0/2650018930 wait complete. 2023-10-08T09:15:00.060 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:15:00.060 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:15:00.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 -- 172.21.15.49:0/3008173860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 msgr2=0x7fda100ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3008173860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda100ffac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fda000099c0 tx=0x7fda00009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 -- 172.21.15.49:0/3008173860 shutdown_connections 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3008173860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda10101230 0x7fda10107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3008173860 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda100ffac0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3008173860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda10100890 0x7fda10100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 -- 172.21.15.49:0/3008173860 >> 172.21.15.49:0/3008173860 conn(0x7fda100fac10 msgr2=0x7fda100fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 -- 172.21.15.49:0/3008173860 shutdown_connections 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.189+0000 7fda17b9c700 1 -- 172.21.15.49:0/3008173860 wait complete. 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 Processor -- start 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 -- start start 2023-10-08T09:15:00.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda101955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda10100890 0x7fda10195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda10101230 0x7fda1019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1019d0e0 con 0x7fda100ff6a0 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1019d220 con 0x7fda10100890 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda17b9c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1019d3b0 con 0x7fda10101230 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda101955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda101955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49902/0 (socket says 172.21.15.49:49902) 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 -- 172.21.15.49:0/3152251791 learned_addr learned my addr 172.21.15.49:0/3152251791 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15137700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda10100890 0x7fda10195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda16139700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda10101230 0x7fda1019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 -- 172.21.15.49:0/3152251791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda10101230 msgr2=0x7fda1019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda10101230 0x7fda1019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 -- 172.21.15.49:0/3152251791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda10100890 msgr2=0x7fda10195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda10100890 0x7fda10195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 -- 172.21.15.49:0/3152251791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda000095f0 con 0x7fda100ff6a0 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15137700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda10100890 0x7fda10195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.190+0000 7fda15938700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda101955b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fda00006010 tx=0x7fda00005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.191+0000 7fda06ffd700 1 -- 172.21.15.49:0/3152251791 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0001d070 con 0x7fda100ff6a0 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.191+0000 7fda06ffd700 1 -- 172.21.15.49:0/3152251791 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda00004150 con 0x7fda100ff6a0 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.191+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1019d630 con 0x7fda100ff6a0 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.191+0000 7fda06ffd700 1 -- 172.21.15.49:0/3152251791 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00021410 con 0x7fda100ff6a0 2023-10-08T09:15:00.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.191+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1019dab0 con 0x7fda100ff6a0 2023-10-08T09:15:00.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.192+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda10106ad0 con 0x7fda100ff6a0 2023-10-08T09:15:00.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.193+0000 7fda06ffd700 1 -- 172.21.15.49:0/3152251791 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fda0000f6f0 con 0x7fda100ff6a0 2023-10-08T09:15:00.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.193+0000 7fda06ffd700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9fc0a1500 0x7fd9fc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.193+0000 7fda15137700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9fc0a1500 0x7fd9fc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.193+0000 7fda06ffd700 1 -- 172.21.15.49:0/3152251791 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fda000ce2f0 con 0x7fda100ff6a0 2023-10-08T09:15:00.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.194+0000 7fda15137700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9fc0a1500 0x7fd9fc0a39c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005fd0 tx=0x7fda0c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:00.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.197+0000 7fda06ffd700 1 -- 172.21.15.49:0/3152251791 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fda000976b0 con 0x7fda100ff6a0 2023-10-08T09:15:00.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.397+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda1010b1b0 con 0x7fda100ff6a0 2023-10-08T09:15:00.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.398+0000 7fda06ffd700 1 -- 172.21.15.49:0/3152251791 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda0009acd0 con 0x7fda100ff6a0 2023-10-08T09:15:00.398 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:15:00.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9fc0a1500 msgr2=0x7fd9fc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9fc0a1500 0x7fd9fc0a39c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda0c005fd0 tx=0x7fda0c005e20 comp rx=0 tx=0).stop 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 msgr2=0x7fda101955b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda101955b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fda00006010 tx=0x7fda00005af0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 shutdown_connections 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9fc0a1500 0x7fd9fc0a39c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fda10101230 0x7fda1019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fda100ff6a0 0x7fda101955b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 --2- 172.21.15.49:0/3152251791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda10100890 0x7fda10195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 >> 172.21.15.49:0/3152251791 conn(0x7fda100fac10 msgr2=0x7fda10103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:00.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 shutdown_connections 2023-10-08T09:15:00.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.400+0000 7fda17b9c700 1 -- 172.21.15.49:0/3152251791 wait complete. 2023-10-08T09:15:00.413 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:15:00.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.542+0000 7f0f10c6c700 1 -- 172.21.15.49:0/593274917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1007c0 msgr2=0x7f0f0c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.542+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/593274917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1007c0 0x7f0f0c100be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0099c0 tx=0x7f0efc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 -- 172.21.15.49:0/593274917 shutdown_connections 2023-10-08T09:15:00.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/593274917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0c102350 0x7f0f0c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/593274917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1007c0 0x7f0f0c100be0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/593274917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f0c1019b0 0x7f0f0c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 -- 172.21.15.49:0/593274917 >> 172.21.15.49:0/593274917 conn(0x7f0f0c0fbd30 msgr2=0x7f0f0c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:00.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 -- 172.21.15.49:0/593274917 shutdown_connections 2023-10-08T09:15:00.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 -- 172.21.15.49:0/593274917 wait complete. 2023-10-08T09:15:00.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 Processor -- start 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.543+0000 7f0f10c6c700 1 -- start start 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f10c6c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0c1007c0 0x7f0f0c196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f10c6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1019b0 0x7f0f0c196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f10c6c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f0c102350 0x7f0f0c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f10c6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c19e2c0 con 0x7f0f0c1019b0 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f10c6c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c19e400 con 0x7f0f0c102350 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f10c6c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c19e590 con 0x7f0f0c1007c0 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1019b0 0x7f0f0c196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1019b0 0x7f0f0c196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49918/0 (socket says 172.21.15.49:49918) 2023-10-08T09:15:00.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 -- 172.21.15.49:0/3265848770 learned_addr learned my addr 172.21.15.49:0/3265848770 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f0a59c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0c1007c0 0x7f0f0c196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f0ad9d700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f0c102350 0x7f0f0c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 -- 172.21.15.49:0/3265848770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0c1007c0 msgr2=0x7f0f0c196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0c1007c0 0x7f0f0c196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 -- 172.21.15.49:0/3265848770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f0c102350 msgr2=0x7f0f0c19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f0c102350 0x7f0f0c19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 -- 172.21.15.49:0/3265848770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc0095f0 con 0x7f0f0c1019b0 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f09d9b700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1019b0 0x7f0f0c196cd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000d950 tx=0x7f0f0000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0efb7fe700 1 -- 172.21.15.49:0/3265848770 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00009940 con 0x7f0f0c1019b0 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c19e870 con 0x7f0f0c1019b0 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0efb7fe700 1 -- 172.21.15.49:0/3265848770 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f00010460 con 0x7f0f0c1019b0 2023-10-08T09:15:00.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.544+0000 7f0efb7fe700 1 -- 172.21.15.49:0/3265848770 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000f620 con 0x7f0f0c1019b0 2023-10-08T09:15:00.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.545+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c19ed80 con 0x7f0f0c1019b0 2023-10-08T09:15:00.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.545+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c107c40 con 0x7f0f0c1019b0 2023-10-08T09:15:00.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.546+0000 7f0efb7fe700 1 -- 172.21.15.49:0/3265848770 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0f000108d0 con 0x7f0f0c1019b0 2023-10-08T09:15:00.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.547+0000 7f0efb7fe700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0ef40a1480 0x7f0ef40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.547+0000 7f0f0a59c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0ef40a1480 0x7f0ef40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.547+0000 7f0efb7fe700 1 -- 172.21.15.49:0/3265848770 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0f000ccea0 con 0x7f0f0c1019b0 2023-10-08T09:15:00.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.548+0000 7f0f0a59c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0ef40a1480 0x7f0ef40a3940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009f50 tx=0x7f0efc005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:00.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.552+0000 7f0efb7fe700 1 -- 172.21.15.49:0/3265848770 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0f000961d0 con 0x7f0f0c1019b0 2023-10-08T09:15:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:00 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2170300395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:00 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2650018930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:00 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3152251791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:15:00.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.754+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f0c107620 con 0x7f0f0c1019b0 2023-10-08T09:15:00.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.754+0000 7f0efb7fe700 1 -- 172.21.15.49:0/3265848770 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f00016020 con 0x7f0f0c1019b0 2023-10-08T09:15:00.755 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:15:00.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0ef40a1480 msgr2=0x7f0ef40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0ef40a1480 0x7f0ef40a3940 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009f50 tx=0x7f0efc005ea0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1019b0 msgr2=0x7f0f0c196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1019b0 0x7f0f0c196cd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000d950 tx=0x7f0f0000dc60 comp rx=0 tx=0).stop 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 shutdown_connections 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0ef40a1480 0x7f0ef40a3940 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f0c1007c0 0x7f0f0c196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f0c102350 0x7f0f0c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 --2- 172.21.15.49:0/3265848770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f0c1019b0 0x7f0f0c196cd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 >> 172.21.15.49:0/3265848770 conn(0x7f0f0c0fbd30 msgr2=0x7f0f0c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 shutdown_connections 2023-10-08T09:15:00.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.757+0000 7f0f10c6c700 1 -- 172.21.15.49:0/3265848770 wait complete. 2023-10-08T09:15:00.768 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:00 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2170300395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:00.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:00 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2650018930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:00.769 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:00 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3152251791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:15:00.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:15:00.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:00 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2170300395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:00 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2650018930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:00.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:00 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3152251791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 -- 172.21.15.49:0/853286017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1019b0 msgr2=0x7f5d4c101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/853286017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1019b0 0x7f5d4c101e10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5d380099c0 tx=0x7f5d38009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 -- 172.21.15.49:0/853286017 shutdown_connections 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/853286017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c1007c0 0x7f5d4c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/853286017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d4c102350 0x7f5d4c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/853286017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1019b0 0x7f5d4c101e10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 -- 172.21.15.49:0/853286017 >> 172.21.15.49:0/853286017 conn(0x7f5d4c0fbd30 msgr2=0x7f5d4c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:00.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 -- 172.21.15.49:0/853286017 shutdown_connections 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.898+0000 7f5d52cd4700 1 -- 172.21.15.49:0/853286017 wait complete. 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 Processor -- start 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 -- start start 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1007c0 0x7f5d4c196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c1019b0 0x7f5d4c196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d4c102350 0x7f5d4c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c19e280 con 0x7f5d4c1007c0 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c19e3c0 con 0x7f5d4c102350 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d52cd4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c19e550 con 0x7f5d4c1019b0 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d50a70700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1007c0 0x7f5d4c196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d50a70700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1007c0 0x7f5d4c196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49930/0 (socket says 172.21.15.49:49930) 2023-10-08T09:15:00.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.899+0000 7f5d50a70700 1 -- 172.21.15.49:0/475296313 learned_addr learned my addr 172.21.15.49:0/475296313 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d4bfff700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c1019b0 0x7f5d4c196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d51271700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d4c102350 0x7f5d4c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d50a70700 1 -- 172.21.15.49:0/475296313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c1019b0 msgr2=0x7f5d4c196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d50a70700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c1019b0 0x7f5d4c196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d50a70700 1 -- 172.21.15.49:0/475296313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d4c102350 msgr2=0x7f5d4c19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d50a70700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d4c102350 0x7f5d4c19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d50a70700 1 -- 172.21.15.49:0/475296313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d380095f0 con 0x7f5d4c1007c0 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d4bfff700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c1019b0 0x7f5d4c196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d50a70700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1007c0 0x7f5d4c196750 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000d980 tx=0x7f5d4000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d49ffb700 1 -- 172.21.15.49:0/475296313 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000de60 con 0x7f5d4c1007c0 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d49ffb700 1 -- 172.21.15.49:0/475296313 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d400044e0 con 0x7f5d4c1007c0 2023-10-08T09:15:00.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c19e830 con 0x7f5d4c1007c0 2023-10-08T09:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d49ffb700 1 -- 172.21.15.49:0/475296313 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40003be0 con 0x7f5d4c1007c0 2023-10-08T09:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.900+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c19ed90 con 0x7f5d4c1007c0 2023-10-08T09:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.901+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c107c40 con 0x7f5d4c1007c0 2023-10-08T09:15:00.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.902+0000 7f5d49ffb700 1 -- 172.21.15.49:0/475296313 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5d40009bb0 con 0x7f5d4c1007c0 2023-10-08T09:15:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.903+0000 7f5d49ffb700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5d3c0a1500 0x7f5d3c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.903+0000 7f5d49ffb700 1 -- 172.21.15.49:0/475296313 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5d400ccd70 con 0x7f5d4c1007c0 2023-10-08T09:15:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.903+0000 7f5d4bfff700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5d3c0a1500 0x7f5d3c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:00.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.903+0000 7f5d4bfff700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5d3c0a1500 0x7f5d3c0a39c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000c00 tx=0x7f5d38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:00.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:00.907+0000 7f5d49ffb700 1 -- 172.21.15.49:0/475296313 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5d400960a0 con 0x7f5d4c1007c0 2023-10-08T09:15:01.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.108+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d4c107620 con 0x7f5d4c1007c0 2023-10-08T09:15:01.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.108+0000 7f5d49ffb700 1 -- 172.21.15.49:0/475296313 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d40009940 con 0x7f5d4c1007c0 2023-10-08T09:15:01.109 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:15:01.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5d3c0a1500 msgr2=0x7f5d3c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5d3c0a1500 0x7f5d3c0a39c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d38000c00 tx=0x7f5d38005fd0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1007c0 msgr2=0x7f5d4c196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1007c0 0x7f5d4c196750 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000d980 tx=0x7f5d4000dd40 comp rx=0 tx=0).stop 2023-10-08T09:15:01.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 shutdown_connections 2023-10-08T09:15:01.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5d3c0a1500 0x7f5d3c0a39c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d4c102350 0x7f5d4c19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d4c1007c0 0x7f5d4c196750 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 --2- 172.21.15.49:0/475296313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5d4c1019b0 0x7f5d4c196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 >> 172.21.15.49:0/475296313 conn(0x7f5d4c0fbd30 msgr2=0x7f5d4c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:01.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 shutdown_connections 2023-10-08T09:15:01.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.110+0000 7f5d52cd4700 1 -- 172.21.15.49:0/475296313 wait complete. 2023-10-08T09:15:01.123 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:15:01.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 -- 172.21.15.49:0/2984173507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a80ff6a0 msgr2=0x7fd2a80ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/2984173507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a80ff6a0 0x7fd2a80ffac0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd2980099c0 tx=0x7fd298009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 -- 172.21.15.49:0/2984173507 shutdown_connections 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/2984173507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a8101230 0x7fd2a8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/2984173507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a80ff6a0 0x7fd2a80ffac0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/2984173507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2a8100890 0x7fd2a8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 -- 172.21.15.49:0/2984173507 >> 172.21.15.49:0/2984173507 conn(0x7fd2a80fac10 msgr2=0x7fd2a80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 -- 172.21.15.49:0/2984173507 shutdown_connections 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.253+0000 7fd2aff5a700 1 -- 172.21.15.49:0/2984173507 wait complete. 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 Processor -- start 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 -- start start 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a80ff6a0 0x7fd2a8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2a8100890 0x7fd2a8198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a8101230 0x7fd2a819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81a0430 con 0x7fd2a8101230 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81a0570 con 0x7fd2a80ff6a0 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2aff5a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81a0700 con 0x7fd2a8100890 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a8101230 0x7fd2a819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a8101230 0x7fd2a819fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49948/0 (socket says 172.21.15.49:49948) 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 -- 172.21.15.49:0/3670018011 learned_addr learned my addr 172.21.15.49:0/3670018011 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ad4f5700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2a8100890 0x7fd2a8198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2adcf6700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a80ff6a0 0x7fd2a8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 -- 172.21.15.49:0/3670018011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2a8100890 msgr2=0x7fd2a8198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2a8100890 0x7fd2a8198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 -- 172.21.15.49:0/3670018011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a80ff6a0 msgr2=0x7fd2a8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a80ff6a0 0x7fd2a8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 -- 172.21.15.49:0/3670018011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980095f0 con 0x7fd2a8101230 2023-10-08T09:15:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2adcf6700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a80ff6a0 0x7fd2a8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.254+0000 7fd2ae4f7700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a8101230 0x7fd2a819fe60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007d70 tx=0x7fd2a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.255+0000 7fd29effd700 1 -- 172.21.15.49:0/3670018011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000ad40 con 0x7fd2a8101230 2023-10-08T09:15:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.255+0000 7fd29effd700 1 -- 172.21.15.49:0/3670018011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2a00128e0 con 0x7fd2a8101230 2023-10-08T09:15:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.255+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a81a09e0 con 0x7fd2a8101230 2023-10-08T09:15:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.255+0000 7fd29effd700 1 -- 172.21.15.49:0/3670018011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00116b0 con 0x7fd2a8101230 2023-10-08T09:15:01.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.255+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a81a0ef0 con 0x7fd2a8101230 2023-10-08T09:15:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.256+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8067530 con 0x7fd2a8101230 2023-10-08T09:15:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.257+0000 7fd29effd700 1 -- 172.21.15.49:0/3670018011 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd2a0012420 con 0x7fd2a8101230 2023-10-08T09:15:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.257+0000 7fd29effd700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd2940a1500 0x7fd2940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.258+0000 7fd29effd700 1 -- 172.21.15.49:0/3670018011 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd2a00cd620 con 0x7fd2a8101230 2023-10-08T09:15:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.258+0000 7fd2adcf6700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd2940a1500 0x7fd2940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.258+0000 7fd2adcf6700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd2940a1500 0x7fd2940a39c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd29800b580 tx=0x7fd298005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:01.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.261+0000 7fd29effd700 1 -- 172.21.15.49:0/3670018011 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd2a0096950 con 0x7fd2a8101230 2023-10-08T09:15:01.461 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.461+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd2a81a1140 con 0x7fd2a8101230 2023-10-08T09:15:01.462 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.461+0000 7fd29effd700 1 -- 172.21.15.49:0/3670018011 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd2a0099f70 con 0x7fd2a8101230 2023-10-08T09:15:01.462 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:15:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd2940a1500 msgr2=0x7fd2940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd2940a1500 0x7fd2940a39c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd29800b580 tx=0x7fd298005fb0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a8101230 msgr2=0x7fd2a819fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a8101230 0x7fd2a819fe60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0007d70 tx=0x7fd2a000a340 comp rx=0 tx=0).stop 2023-10-08T09:15:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 shutdown_connections 2023-10-08T09:15:01.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd2940a1500 0x7fd2940a39c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2a8101230 0x7fd2a819fe60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2a80ff6a0 0x7fd2a8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 --2- 172.21.15.49:0/3670018011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2a8100890 0x7fd2a8198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 >> 172.21.15.49:0/3670018011 conn(0x7fd2a80fac10 msgr2=0x7fd2a80fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 shutdown_connections 2023-10-08T09:15:01.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.463+0000 7fd2aff5a700 1 -- 172.21.15.49:0/3670018011 wait complete. 2023-10-08T09:15:01.476 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-08T09:15:01.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:15:01.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 -- 172.21.15.49:0/1809693976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 msgr2=0x7f0ccc0ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/1809693976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc0ff9f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80099c0 tx=0x7f0cc8009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 -- 172.21.15.49:0/1809693976 shutdown_connections 2023-10-08T09:15:01.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/1809693976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ccc101160 0x7f0ccc107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/1809693976 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc0ff9f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/1809693976 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc1007c0 0x7f0ccc100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 -- 172.21.15.49:0/1809693976 >> 172.21.15.49:0/1809693976 conn(0x7f0ccc0fab40 msgr2=0x7f0ccc0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 -- 172.21.15.49:0/1809693976 shutdown_connections 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 -- 172.21.15.49:0/1809693976 wait complete. 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 Processor -- start 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.602+0000 7f0cd3a27700 1 -- start start 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ccc1007c0 0x7f0ccc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc101160 0x7f0ccc19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc19e130 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc19e2a0 con 0x7f0ccc1007c0 2023-10-08T09:15:01.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc19e460 con 0x7f0ccc101160 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc069570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49958/0 (socket says 172.21.15.49:49958) 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 -- 172.21.15.49:0/641410598 learned_addr learned my addr 172.21.15.49:0/641410598 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd0fc2700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ccc1007c0 0x7f0ccc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd1fc4700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc101160 0x7f0ccc19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 -- 172.21.15.49:0/641410598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc101160 msgr2=0x7f0ccc19d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc101160 0x7f0ccc19d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 -- 172.21.15.49:0/641410598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ccc1007c0 msgr2=0x7f0ccc19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ccc1007c0 0x7f0ccc19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 -- 172.21.15.49:0/641410598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80095f0 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd1fc4700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc101160 0x7f0ccc19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd0fc2700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ccc1007c0 0x7f0ccc19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd17c3700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc069570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8006010 tx=0x7f0cc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cc27fc700 1 -- 172.21.15.49:0/641410598 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc801d070 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ccc19e680 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cc27fc700 1 -- 172.21.15.49:0/641410598 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cc8005940 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cc27fc700 1 -- 172.21.15.49:0/641410598 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc8022da0 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.603+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ccc19eb80 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.604+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ccc106a50 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.605+0000 7f0cc27fc700 1 -- 172.21.15.49:0/641410598 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0cc8022470 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.606+0000 7f0cc27fc700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0cb80a1480 0x7f0cb80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:01.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.606+0000 7f0cd0fc2700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0cb80a1480 0x7f0cb80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:01.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.606+0000 7f0cc27fc700 1 -- 172.21.15.49:0/641410598 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0cc80ce240 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.606+0000 7f0cd0fc2700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0cb80a1480 0x7f0cb80a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005950 tx=0x7f0cbc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:01.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.610+0000 7f0cc27fc700 1 -- 172.21.15.49:0/641410598 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0cc805d100 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:01 smithi184 ceph-mon[103675]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-10-08T09:15:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:01 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3265848770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:01 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/475296313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:01 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3670018011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:01 smithi132 ceph-mon[103389]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-10-08T09:15:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:01 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3265848770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:01 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/475296313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:01 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3670018011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.858+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ccc066bc0 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.859+0000 7f0cc27fc700 1 -- 172.21.15.49:0/641410598 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f0cc80975a0 con 0x7f0ccc0ff5d0 2023-10-08T09:15:01.859 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:01.859 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:15:01.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0cb80a1480 msgr2=0x7f0cb80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0cb80a1480 0x7f0cb80a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc005950 tx=0x7f0cbc0058e0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 msgr2=0x7f0ccc069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:01.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc069570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8006010 tx=0x7f0cc80057f0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 shutdown_connections 2023-10-08T09:15:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0cb80a1480 0x7f0cb80a3940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ccc101160 0x7f0ccc19d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ccc0ff5d0 0x7f0ccc069570 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 --2- 172.21.15.49:0/641410598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0ccc1007c0 0x7f0ccc19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 >> 172.21.15.49:0/641410598 conn(0x7f0ccc0fab40 msgr2=0x7f0ccc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 shutdown_connections 2023-10-08T09:15:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:01.861+0000 7f0cd3a27700 1 -- 172.21.15.49:0/641410598 wait complete. 2023-10-08T09:15:01.865 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:15:01.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:01 smithi049 ceph-mon[99485]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2023-10-08T09:15:01.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:01 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3265848770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:01 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/475296313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.873 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:01 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3670018011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:01.874 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-08T09:15:01.874 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-08T09:15:01.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:15:02.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 -- 172.21.15.49:0/2018606289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf20102250 msgr2=0x7fdf20108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 --2- 172.21.15.49:0/2018606289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf20102250 0x7fdf20108b00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800b0d0 tx=0x7fdf1800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 -- 172.21.15.49:0/2018606289 shutdown_connections 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 --2- 172.21.15.49:0/2018606289 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf20102250 0x7fdf20108b00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 --2- 172.21.15.49:0/2018606289 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf201006c0 0x7fdf20100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 --2- 172.21.15.49:0/2018606289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf201018b0 0x7fdf20101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 -- 172.21.15.49:0/2018606289 >> 172.21.15.49:0/2018606289 conn(0x7fdf200fbc30 msgr2=0x7fdf200fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 -- 172.21.15.49:0/2018606289 shutdown_connections 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.000+0000 7fdf27d79700 1 -- 172.21.15.49:0/2018606289 wait complete. 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 Processor -- start 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 -- start start 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf201006c0 0x7fdf20074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf201018b0 0x7fdf20072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf20102250 0x7fdf20073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf200738c0 con 0x7fdf201018b0 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf20073a30 con 0x7fdf20102250 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf27d79700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf201a0770 con 0x7fdf201006c0 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf201018b0 0x7fdf20072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf201018b0 0x7fdf20072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49974/0 (socket says 172.21.15.49:49974) 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 -- 172.21.15.49:0/3927129417 learned_addr learned my addr 172.21.15.49:0/3927129417 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25b15700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf201006c0 0x7fdf20074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf26316700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf20102250 0x7fdf20073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 -- 172.21.15.49:0/3927129417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf201006c0 msgr2=0x7fdf20074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf201006c0 0x7fdf20074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 -- 172.21.15.49:0/3927129417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf20102250 msgr2=0x7fdf20073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf20102250 0x7fdf20073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.001+0000 7fdf25314700 1 -- 172.21.15.49:0/3927129417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf18009d00 con 0x7fdf201018b0 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.002+0000 7fdf25314700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf201018b0 0x7fdf20072be0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d950 tx=0x7fdf1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.002+0000 7fdf12ffd700 1 -- 172.21.15.49:0/3927129417 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400f890 con 0x7fdf201018b0 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.002+0000 7fdf12ffd700 1 -- 172.21.15.49:0/3927129417 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf14010460 con 0x7fdf201018b0 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.002+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf201a0a00 con 0x7fdf201018b0 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.002+0000 7fdf12ffd700 1 -- 172.21.15.49:0/3927129417 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140195c0 con 0x7fdf201018b0 2023-10-08T09:15:02.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.002+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf201a0f10 con 0x7fdf201018b0 2023-10-08T09:15:02.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.003+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf20107a70 con 0x7fdf201018b0 2023-10-08T09:15:02.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.004+0000 7fdf12ffd700 1 -- 172.21.15.49:0/3927129417 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdf140108d0 con 0x7fdf201018b0 2023-10-08T09:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.005+0000 7fdf12ffd700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdf0c0a1480 0x7fdf0c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.005+0000 7fdf25b15700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdf0c0a1480 0x7fdf0c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.005+0000 7fdf12ffd700 1 -- 172.21.15.49:0/3927129417 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdf140ccdf0 con 0x7fdf201018b0 2023-10-08T09:15:02.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.006+0000 7fdf25b15700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdf0c0a1480 0x7fdf0c0a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdf20101710 tx=0x7fdf1c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:02.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.008+0000 7fdf12ffd700 1 -- 172.21.15.49:0/3927129417 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdf140960c0 con 0x7fdf201018b0 2023-10-08T09:15:02.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.205+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf2002cfb0 con 0x7fdf0c0a1480 2023-10-08T09:15:02.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.220+0000 7fdf12ffd700 1 -- 172.21.15.49:0/3927129417 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427132 (secure 0 0 0) 0x7fdf2002cfb0 con 0x7fdf0c0a1480 2023-10-08T09:15:02.227 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdf0c0a1480 msgr2=0x7fdf0c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdf0c0a1480 0x7fdf0c0a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdf20101710 tx=0x7fdf1c00a380 comp rx=0 tx=0).stop 2023-10-08T09:15:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf201018b0 msgr2=0x7fdf20072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf201018b0 0x7fdf20072be0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d950 tx=0x7fdf1400dc60 comp rx=0 tx=0).stop 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 shutdown_connections 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdf0c0a1480 0x7fdf0c0a3940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf201006c0 0x7fdf20074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf20102250 0x7fdf20073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 --2- 172.21.15.49:0/3927129417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdf201018b0 0x7fdf20072be0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 >> 172.21.15.49:0/3927129417 conn(0x7fdf200fbc30 msgr2=0x7fdf20104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 shutdown_connections 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.224+0000 7fdf27d79700 1 -- 172.21.15.49:0/3927129417 wait complete. 2023-10-08T09:15:02.232 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-10-08T09:15:02.237 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2023-10-08T09:15:01.884548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657474,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1045,"num_read_kb":642,"num_write":430,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180859,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336424,"kb_used_data":12188,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605400,"statfs":{"total":1151940427776,"available":1151595929600,"internally_reserved":0,"allocated":12480512,"data_stored":8025723,"data_compressed":391095,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003618"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429290+0000","last_change":"2023-10-08T09:13:02.831436+0000","last_active":"2023-10-08T09:14:37.429290+0000","last_peered":"2023-10-08T09:14:37.429290+0000","last_clean":"2023-10-08T09:14:37.429290+0000","last_became_active":"2023-10-08T09:13:02.831123+0000","last_became_peered":"2023-10-08T09:13:02.831123+0000","last_unstale":"2023-10-08T09:14:37.429290+0000","last_undegraded":"2023-10-08T09:14:37.429290+0000","last_fullsized":"2023-10-08T09:14:37.429290+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:33.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737779+0000","last_change":"2023-10-08T09:13:02.832286+0000","last_active":"2023-10-08T09:14:37.737779+0000","last_peered":"2023-10-08T09:14:37.737779+0000","last_clean":"2023-10-08T09:14:37.737779+0000","last_became_active":"2023-10-08T09:13:02.831881+0000","last_became_peered":"2023-10-08T09:13:02.831881+0000","last_unstale":"2023-10-08T09:14:37.737779+0000","last_undegraded":"2023-10-08T09:14:37.737779+0000","last_fullsized":"2023-10-08T09:14:37.737779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:31.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737613+0000","last_change":"2023-10-08T09:13:02.831454+0000","last_active":"2023-10-08T09:14:37.737613+0000","last_peered":"2023-10-08T09:14:37.737613+0000","last_clean":"2023-10-08T09:14:37.737613+0000","last_became_active":"2023-10-08T09:13:02.831040+0000","last_became_peered":"2023-10-08T09:13:02.831040+0000","last_unstale":"2023-10-08T09:14:37.737613+0000","last_undegraded":"2023-10-08T09:14:37.737613+0000","last_fullsized":"2023-10-08T09:14:37.737613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:16:02.788279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514426+0000","last_change":"2023-10-08T09:13:02.832699+0000","last_active":"2023-10-08T09:14:37.514426+0000","last_peered":"2023-10-08T09:14:37.514426+0000","last_clean":"2023-10-08T09:14:37.514426+0000","last_became_active":"2023-10-08T09:13:02.832345+0000","last_became_peered":"2023-10-08T09:13:02.832345+0000","last_unstale":"2023-10-08T09:14:37.514426+0000","last_undegraded":"2023-10-08T09:14:37.514426+0000","last_fullsized":"2023-10-08T09:14:37.514426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:01.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534951+0000","last_change":"2023-10-08T09:13:02.827767+0000","last_active":"2023-10-08T09:14:37.534951+0000","last_peered":"2023-10-08T09:14:37.534951+0000","last_clean":"2023-10-08T09:14:37.534951+0000","last_became_active":"2023-10-08T09:13:02.827454+0000","last_became_peered":"2023-10-08T09:13:02.827454+0000","last_unstale":"2023-10-08T09:14:37.534951+0000","last_undegraded":"2023-10-08T09:14:37.534951+0000","last_fullsized":"2023-10-08T09:14:37.534951+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:43.122347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512454+0000","last_change":"2023-10-08T09:13:37.452701+0000","last_active":"2023-10-08T09:14:37.512454+0000","last_peered":"2023-10-08T09:14:37.512454+0000","last_clean":"2023-10-08T09:14:37.512454+0000","last_became_active":"2023-10-08T09:13:37.452261+0000","last_became_peered":"2023-10-08T09:13:37.452261+0000","last_unstale":"2023-10-08T09:14:37.512454+0000","last_undegraded":"2023-10-08T09:14:37.512454+0000","last_fullsized":"2023-10-08T09:14:37.512454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:56.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426872+0000","last_change":"2023-10-08T09:13:02.831765+0000","last_active":"2023-10-08T09:14:37.426872+0000","last_peered":"2023-10-08T09:14:37.426872+0000","last_clean":"2023-10-08T09:14:37.426872+0000","last_became_active":"2023-10-08T09:13:02.831446+0000","last_became_peered":"2023-10-08T09:13:02.831446+0000","last_unstale":"2023-10-08T09:14:37.426872+0000","last_undegraded":"2023-10-08T09:14:37.426872+0000","last_fullsized":"2023-10-08T09:14:37.426872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:41.463161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428811+0000","last_change":"2023-10-08T09:13:02.817586+0000","last_active":"2023-10-08T09:14:37.428811+0000","last_peered":"2023-10-08T09:14:37.428811+0000","last_clean":"2023-10-08T09:14:37.428811+0000","last_became_active":"2023-10-08T09:13:02.817231+0000","last_became_peered":"2023-10-08T09:13:02.817231+0000","last_unstale":"2023-10-08T09:14:37.428811+0000","last_undegraded":"2023-10-08T09:14:37.428811+0000","last_fullsized":"2023-10-08T09:14:37.428811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:23.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.738138+0000","last_change":"2023-10-08T09:13:02.831925+0000","last_active":"2023-10-08T09:14:37.738138+0000","last_peered":"2023-10-08T09:14:37.738138+0000","last_clean":"2023-10-08T09:14:37.738138+0000","last_became_active":"2023-10-08T09:13:02.831625+0000","last_became_peered":"2023-10-08T09:13:02.831625+0000","last_unstale":"2023-10-08T09:14:37.738138+0000","last_undegraded":"2023-10-08T09:14:37.738138+0000","last_fullsized":"2023-10-08T09:14:37.738138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:39.115740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512758+0000","last_change":"2023-10-08T09:13:02.831752+0000","last_active":"2023-10-08T09:14:37.512758+0000","last_peered":"2023-10-08T09:14:37.512758+0000","last_clean":"2023-10-08T09:14:37.512758+0000","last_became_active":"2023-10-08T09:13:02.831468+0000","last_became_peered":"2023-10-08T09:13:02.831468+0000","last_unstale":"2023-10-08T09:14:37.512758+0000","last_undegraded":"2023-10-08T09:14:37.512758+0000","last_fullsized":"2023-10-08T09:14:37.512758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:05.758017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427202+0000","last_change":"2023-10-08T09:13:02.828838+0000","last_active":"2023-10-08T09:14:37.427202+0000","last_peered":"2023-10-08T09:14:37.427202+0000","last_clean":"2023-10-08T09:14:37.427202+0000","last_became_active":"2023-10-08T09:13:02.828598+0000","last_became_peered":"2023-10-08T09:13:02.828598+0000","last_unstale":"2023-10-08T09:14:37.427202+0000","last_undegraded":"2023-10-08T09:14:37.427202+0000","last_fullsized":"2023-10-08T09:14:37.427202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:04.317649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425720+0000","last_change":"2023-10-08T09:13:37.450251+0000","last_active":"2023-10-08T09:14:37.425720+0000","last_peered":"2023-10-08T09:14:37.425720+0000","last_clean":"2023-10-08T09:14:37.425720+0000","last_became_active":"2023-10-08T09:13:37.449848+0000","last_became_peered":"2023-10-08T09:13:37.449848+0000","last_unstale":"2023-10-08T09:14:37.425720+0000","last_undegraded":"2023-10-08T09:14:37.425720+0000","last_fullsized":"2023-10-08T09:14:37.425720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:09:16.541785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511528+0000","last_change":"2023-10-08T09:13:37.450381+0000","last_active":"2023-10-08T09:14:37.511528+0000","last_peered":"2023-10-08T09:14:37.511528+0000","last_clean":"2023-10-08T09:14:37.511528+0000","last_became_active":"2023-10-08T09:13:37.450044+0000","last_became_peered":"2023-10-08T09:13:37.450044+0000","last_unstale":"2023-10-08T09:14:37.511528+0000","last_undegraded":"2023-10-08T09:14:37.511528+0000","last_fullsized":"2023-10-08T09:14:37.511528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:03.203894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737338+0000","last_change":"2023-10-08T09:13:02.829027+0000","last_active":"2023-10-08T09:14:37.737338+0000","last_peered":"2023-10-08T09:14:37.737338+0000","last_clean":"2023-10-08T09:14:37.737338+0000","last_became_active":"2023-10-08T09:13:02.828714+0000","last_became_peered":"2023-10-08T09:13:02.828714+0000","last_unstale":"2023-10-08T09:14:37.737338+0000","last_undegraded":"2023-10-08T09:14:37.737338+0000","last_fullsized":"2023-10-08T09:14:37.737338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:13.121274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737347+0000","last_change":"2023-10-08T09:13:02.838640+0000","last_active":"2023-10-08T09:14:37.737347+0000","last_peered":"2023-10-08T09:14:37.737347+0000","last_clean":"2023-10-08T09:14:37.737347+0000","last_became_active":"2023-10-08T09:13:02.838302+0000","last_became_peered":"2023-10-08T09:13:02.838302+0000","last_unstale":"2023-10-08T09:14:37.737347+0000","last_undegraded":"2023-10-08T09:14:37.737347+0000","last_fullsized":"2023-10-08T09:14:37.737347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:54.268315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737546+0000","last_change":"2023-10-08T09:13:02.834027+0000","last_active":"2023-10-08T09:14:37.737546+0000","last_peered":"2023-10-08T09:14:37.737546+0000","last_clean":"2023-10-08T09:14:37.737546+0000","last_became_active":"2023-10-08T09:13:02.833733+0000","last_became_peered":"2023-10-08T09:13:02.833733+0000","last_unstale":"2023-10-08T09:14:37.737546+0000","last_undegraded":"2023-10-08T09:14:37.737546+0000","last_fullsized":"2023-10-08T09:14:37.737546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:03.644640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429275+0000","last_change":"2023-10-08T09:13:02.832040+0000","last_active":"2023-10-08T09:14:37.429275+0000","last_peered":"2023-10-08T09:14:37.429275+0000","last_clean":"2023-10-08T09:14:37.429275+0000","last_became_active":"2023-10-08T09:13:02.831752+0000","last_became_peered":"2023-10-08T09:13:02.831752+0000","last_unstale":"2023-10-08T09:14:37.429275+0000","last_undegraded":"2023-10-08T09:14:37.429275+0000","last_fullsized":"2023-10-08T09:14:37.429275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:38.938703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426182+0000","last_change":"2023-10-08T09:13:02.830535+0000","last_active":"2023-10-08T09:14:37.426182+0000","last_peered":"2023-10-08T09:14:37.426182+0000","last_clean":"2023-10-08T09:14:37.426182+0000","last_became_active":"2023-10-08T09:13:02.830144+0000","last_became_peered":"2023-10-08T09:13:02.830144+0000","last_unstale":"2023-10-08T09:14:37.426182+0000","last_undegraded":"2023-10-08T09:14:37.426182+0000","last_fullsized":"2023-10-08T09:14:37.426182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:33.850815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426536+0000","last_change":"2023-10-08T09:13:02.830938+0000","last_active":"2023-10-08T09:14:37.426536+0000","last_peered":"2023-10-08T09:14:37.426536+0000","last_clean":"2023-10-08T09:14:37.426536+0000","last_became_active":"2023-10-08T09:13:02.830551+0000","last_became_peered":"2023-10-08T09:13:02.830551+0000","last_unstale":"2023-10-08T09:14:37.426536+0000","last_undegraded":"2023-10-08T09:14:37.426536+0000","last_fullsized":"2023-10-08T09:14:37.426536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:26.894614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425970+0000","last_change":"2023-10-08T09:13:07.871322+0000","last_active":"2023-10-08T09:14:37.425970+0000","last_peered":"2023-10-08T09:14:37.425970+0000","last_clean":"2023-10-08T09:14:37.425970+0000","last_became_active":"2023-10-08T09:13:07.870828+0000","last_became_peered":"2023-10-08T09:13:07.870828+0000","last_unstale":"2023-10-08T09:14:37.425970+0000","last_undegraded":"2023-10-08T09:14:37.425970+0000","last_fullsized":"2023-10-08T09:14:37.425970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:37.145705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.558590+0000","last_change":"2023-10-08T09:13:02.373510+0000","last_active":"2023-10-08T09:14:57.558590+0000","last_peered":"2023-10-08T09:14:57.558590+0000","last_clean":"2023-10-08T09:14:57.558590+0000","last_became_active":"2023-10-08T09:13:02.373174+0000","last_became_peered":"2023-10-08T09:13:02.373174+0000","last_unstale":"2023-10-08T09:14:57.558590+0000","last_undegraded":"2023-10-08T09:14:57.558590+0000","last_fullsized":"2023-10-08T09:14:57.558590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429497+0000","last_change":"2023-10-08T09:13:02.835195+0000","last_active":"2023-10-08T09:14:37.429497+0000","last_peered":"2023-10-08T09:14:37.429497+0000","last_clean":"2023-10-08T09:14:37.429497+0000","last_became_active":"2023-10-08T09:13:02.834893+0000","last_became_peered":"2023-10-08T09:13:02.834893+0000","last_unstale":"2023-10-08T09:14:37.429497+0000","last_undegraded":"2023-10-08T09:14:37.429497+0000","last_fullsized":"2023-10-08T09:14:37.429497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:35.835017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513867+0000","last_change":"2023-10-08T09:13:07.871418+0000","last_active":"2023-10-08T09:14:37.513867+0000","last_peered":"2023-10-08T09:14:37.513867+0000","last_clean":"2023-10-08T09:14:37.513867+0000","last_became_active":"2023-10-08T09:13:07.870961+0000","last_became_peered":"2023-10-08T09:13:07.870961+0000","last_unstale":"2023-10-08T09:14:37.513867+0000","last_undegraded":"2023-10-08T09:14:37.513867+0000","last_fullsized":"2023-10-08T09:14:37.513867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:31.723498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.546688+0000","last_change":"2023-10-08T09:13:02.370728+0000","last_active":"2023-10-08T09:14:38.546688+0000","last_peered":"2023-10-08T09:14:38.546688+0000","last_clean":"2023-10-08T09:14:38.546688+0000","last_became_active":"2023-10-08T09:13:02.370396+0000","last_became_peered":"2023-10-08T09:13:02.370396+0000","last_unstale":"2023-10-08T09:14:38.546688+0000","last_undegraded":"2023-10-08T09:14:38.546688+0000","last_fullsized":"2023-10-08T09:14:38.546688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:32.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737684+0000","last_change":"2023-10-08T09:13:02.827823+0000","last_active":"2023-10-08T09:14:37.737684+0000","last_peered":"2023-10-08T09:14:37.737684+0000","last_clean":"2023-10-08T09:14:37.737684+0000","last_became_active":"2023-10-08T09:13:02.827417+0000","last_became_peered":"2023-10-08T09:13:02.827417+0000","last_unstale":"2023-10-08T09:14:37.737684+0000","last_undegraded":"2023-10-08T09:14:37.737684+0000","last_fullsized":"2023-10-08T09:14:37.737684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:38.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513945+0000","last_change":"2023-10-08T09:13:37.455325+0000","last_active":"2023-10-08T09:14:37.513945+0000","last_peered":"2023-10-08T09:14:37.513945+0000","last_clean":"2023-10-08T09:14:37.513945+0000","last_became_active":"2023-10-08T09:13:37.454985+0000","last_became_peered":"2023-10-08T09:13:37.454985+0000","last_unstale":"2023-10-08T09:14:37.513945+0000","last_undegraded":"2023-10-08T09:14:37.513945+0000","last_fullsized":"2023-10-08T09:14:37.513945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:13.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535311+0000","last_change":"2023-10-08T09:13:02.375244+0000","last_active":"2023-10-08T09:14:37.535311+0000","last_peered":"2023-10-08T09:14:37.535311+0000","last_clean":"2023-10-08T09:14:37.535311+0000","last_became_active":"2023-10-08T09:13:02.374526+0000","last_became_peered":"2023-10-08T09:13:02.374526+0000","last_unstale":"2023-10-08T09:14:37.535311+0000","last_undegraded":"2023-10-08T09:14:37.535311+0000","last_fullsized":"2023-10-08T09:14:37.535311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:36.380491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737863+0000","last_change":"2023-10-08T09:13:02.838373+0000","last_active":"2023-10-08T09:14:37.737863+0000","last_peered":"2023-10-08T09:14:37.737863+0000","last_clean":"2023-10-08T09:14:37.737863+0000","last_became_active":"2023-10-08T09:13:02.838100+0000","last_became_peered":"2023-10-08T09:13:02.838100+0000","last_unstale":"2023-10-08T09:14:37.737863+0000","last_undegraded":"2023-10-08T09:14:37.737863+0000","last_fullsized":"2023-10-08T09:14:37.737863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:59.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513906+0000","last_change":"2023-10-08T09:13:07.868929+0000","last_active":"2023-10-08T09:14:37.513906+0000","last_peered":"2023-10-08T09:14:37.513906+0000","last_clean":"2023-10-08T09:14:37.513906+0000","last_became_active":"2023-10-08T09:13:07.868451+0000","last_became_peered":"2023-10-08T09:13:07.868451+0000","last_unstale":"2023-10-08T09:14:37.513906+0000","last_undegraded":"2023-10-08T09:14:37.513906+0000","last_fullsized":"2023-10-08T09:14:37.513906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:36.540319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535135+0000","last_change":"2023-10-08T09:13:02.366404+0000","last_active":"2023-10-08T09:14:37.535135+0000","last_peered":"2023-10-08T09:14:37.535135+0000","last_clean":"2023-10-08T09:14:37.535135+0000","last_became_active":"2023-10-08T09:13:02.366025+0000","last_became_peered":"2023-10-08T09:13:02.366025+0000","last_unstale":"2023-10-08T09:14:37.535135+0000","last_undegraded":"2023-10-08T09:14:37.535135+0000","last_fullsized":"2023-10-08T09:14:37.535135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:51.916565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515611+0000","last_change":"2023-10-08T09:13:02.832852+0000","last_active":"2023-10-08T09:14:37.515611+0000","last_peered":"2023-10-08T09:14:37.515611+0000","last_clean":"2023-10-08T09:14:37.515611+0000","last_became_active":"2023-10-08T09:13:02.832530+0000","last_became_peered":"2023-10-08T09:13:02.832530+0000","last_unstale":"2023-10-08T09:14:37.515611+0000","last_undegraded":"2023-10-08T09:14:37.515611+0000","last_fullsized":"2023-10-08T09:14:37.515611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:20.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426351+0000","last_change":"2023-10-08T09:13:07.863864+0000","last_active":"2023-10-08T09:14:37.426351+0000","last_peered":"2023-10-08T09:14:37.426351+0000","last_clean":"2023-10-08T09:14:37.426351+0000","last_became_active":"2023-10-08T09:13:07.863509+0000","last_became_peered":"2023-10-08T09:13:07.863509+0000","last_unstale":"2023-10-08T09:14:37.426351+0000","last_undegraded":"2023-10-08T09:14:37.426351+0000","last_fullsized":"2023-10-08T09:14:37.426351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:22.933327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":117,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.548141+0000","last_change":"2023-10-08T09:13:02.374215+0000","last_active":"2023-10-08T09:14:57.548141+0000","last_peered":"2023-10-08T09:14:57.548141+0000","last_clean":"2023-10-08T09:14:57.548141+0000","last_became_active":"2023-10-08T09:13:02.373912+0000","last_became_peered":"2023-10-08T09:13:02.373912+0000","last_unstale":"2023-10-08T09:14:57.548141+0000","last_undegraded":"2023-10-08T09:14:57.548141+0000","last_fullsized":"2023-10-08T09:14:57.548141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:18.315244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535090+0000","last_change":"2023-10-08T09:13:02.830651+0000","last_active":"2023-10-08T09:14:37.535090+0000","last_peered":"2023-10-08T09:14:37.535090+0000","last_clean":"2023-10-08T09:14:37.535090+0000","last_became_active":"2023-10-08T09:13:02.830324+0000","last_became_peered":"2023-10-08T09:13:02.830324+0000","last_unstale":"2023-10-08T09:14:37.535090+0000","last_undegraded":"2023-10-08T09:14:37.535090+0000","last_fullsized":"2023-10-08T09:14:37.535090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:02.143794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426203+0000","last_change":"2023-10-08T09:13:37.454054+0000","last_active":"2023-10-08T09:14:37.426203+0000","last_peered":"2023-10-08T09:14:37.426203+0000","last_clean":"2023-10-08T09:14:37.426203+0000","last_became_active":"2023-10-08T09:13:37.453482+0000","last_became_peered":"2023-10-08T09:13:37.453482+0000","last_unstale":"2023-10-08T09:14:37.426203+0000","last_undegraded":"2023-10-08T09:14:37.426203+0000","last_fullsized":"2023-10-08T09:14:37.426203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:31.550837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512273+0000","last_change":"2023-10-08T09:13:02.370334+0000","last_active":"2023-10-08T09:14:37.512273+0000","last_peered":"2023-10-08T09:14:37.512273+0000","last_clean":"2023-10-08T09:14:37.512273+0000","last_became_active":"2023-10-08T09:13:02.370068+0000","last_became_peered":"2023-10-08T09:13:02.370068+0000","last_unstale":"2023-10-08T09:14:37.512273+0000","last_undegraded":"2023-10-08T09:14:37.512273+0000","last_fullsized":"2023-10-08T09:14:37.512273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.962779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515743+0000","last_change":"2023-10-08T09:13:02.832346+0000","last_active":"2023-10-08T09:14:37.515743+0000","last_peered":"2023-10-08T09:14:37.515743+0000","last_clean":"2023-10-08T09:14:37.515743+0000","last_became_active":"2023-10-08T09:13:02.832059+0000","last_became_peered":"2023-10-08T09:13:02.832059+0000","last_unstale":"2023-10-08T09:14:37.515743+0000","last_undegraded":"2023-10-08T09:14:37.515743+0000","last_fullsized":"2023-10-08T09:14:37.515743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:30.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429192+0000","last_change":"2023-10-08T09:13:07.863466+0000","last_active":"2023-10-08T09:14:37.429192+0000","last_peered":"2023-10-08T09:14:37.429192+0000","last_clean":"2023-10-08T09:14:37.429192+0000","last_became_active":"2023-10-08T09:13:07.863015+0000","last_became_peered":"2023-10-08T09:13:07.863015+0000","last_unstale":"2023-10-08T09:14:37.429192+0000","last_undegraded":"2023-10-08T09:14:37.429192+0000","last_fullsized":"2023-10-08T09:14:37.429192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:30.316625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511291+0000","last_change":"2023-10-08T09:13:02.369444+0000","last_active":"2023-10-08T09:14:37.511291+0000","last_peered":"2023-10-08T09:14:37.511291+0000","last_clean":"2023-10-08T09:14:37.511291+0000","last_became_active":"2023-10-08T09:13:02.369142+0000","last_became_peered":"2023-10-08T09:13:02.369142+0000","last_unstale":"2023-10-08T09:14:37.511291+0000","last_undegraded":"2023-10-08T09:14:37.511291+0000","last_fullsized":"2023-10-08T09:14:37.511291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:28.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428912+0000","last_change":"2023-10-08T09:13:02.836125+0000","last_active":"2023-10-08T09:14:37.428912+0000","last_peered":"2023-10-08T09:14:37.428912+0000","last_clean":"2023-10-08T09:14:37.428912+0000","last_became_active":"2023-10-08T09:13:02.835809+0000","last_became_peered":"2023-10-08T09:13:02.835809+0000","last_unstale":"2023-10-08T09:14:37.428912+0000","last_undegraded":"2023-10-08T09:14:37.428912+0000","last_fullsized":"2023-10-08T09:14:37.428912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:15.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737758+0000","last_change":"2023-10-08T09:13:07.861633+0000","last_active":"2023-10-08T09:14:37.737758+0000","last_peered":"2023-10-08T09:14:37.737758+0000","last_clean":"2023-10-08T09:14:37.737758+0000","last_became_active":"2023-10-08T09:13:07.861168+0000","last_became_peered":"2023-10-08T09:13:07.861168+0000","last_unstale":"2023-10-08T09:14:37.737758+0000","last_undegraded":"2023-10-08T09:14:37.737758+0000","last_fullsized":"2023-10-08T09:14:37.737758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:47.283892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515340+0000","last_change":"2023-10-08T09:13:02.365703+0000","last_active":"2023-10-08T09:14:37.515340+0000","last_peered":"2023-10-08T09:14:37.515340+0000","last_clean":"2023-10-08T09:14:37.515340+0000","last_became_active":"2023-10-08T09:13:02.365421+0000","last_became_peered":"2023-10-08T09:13:02.365421+0000","last_unstale":"2023-10-08T09:14:37.515340+0000","last_undegraded":"2023-10-08T09:14:37.515340+0000","last_fullsized":"2023-10-08T09:14:37.515340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:43.124511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425689+0000","last_change":"2023-10-08T09:13:02.840941+0000","last_active":"2023-10-08T09:14:37.425689+0000","last_peered":"2023-10-08T09:14:37.425689+0000","last_clean":"2023-10-08T09:14:37.425689+0000","last_became_active":"2023-10-08T09:13:02.840605+0000","last_became_peered":"2023-10-08T09:13:02.840605+0000","last_unstale":"2023-10-08T09:14:37.425689+0000","last_undegraded":"2023-10-08T09:14:37.425689+0000","last_fullsized":"2023-10-08T09:14:37.425689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:24:13.417028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429080+0000","last_change":"2023-10-08T09:13:07.868950+0000","last_active":"2023-10-08T09:14:37.429080+0000","last_peered":"2023-10-08T09:14:37.429080+0000","last_clean":"2023-10-08T09:14:37.429080+0000","last_became_active":"2023-10-08T09:13:07.868542+0000","last_became_peered":"2023-10-08T09:13:07.868542+0000","last_unstale":"2023-10-08T09:14:37.429080+0000","last_undegraded":"2023-10-08T09:14:37.429080+0000","last_fullsized":"2023-10-08T09:14:37.429080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:11.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":201,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549263+0000","last_change":"2023-10-08T09:13:02.376188+0000","last_active":"2023-10-08T09:14:58.549263+0000","last_peered":"2023-10-08T09:14:58.549263+0000","last_clean":"2023-10-08T09:14:58.549263+0000","last_became_active":"2023-10-08T09:13:02.375914+0000","last_became_peered":"2023-10-08T09:13:02.375914+0000","last_unstale":"2023-10-08T09:14:58.549263+0000","last_undegraded":"2023-10-08T09:14:58.549263+0000","last_fullsized":"2023-10-08T09:14:58.549263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:10.744085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.738033+0000","last_change":"2023-10-08T09:13:02.833429+0000","last_active":"2023-10-08T09:14:37.738033+0000","last_peered":"2023-10-08T09:14:37.738033+0000","last_clean":"2023-10-08T09:14:37.738033+0000","last_became_active":"2023-10-08T09:13:02.832792+0000","last_became_peered":"2023-10-08T09:13:02.832792+0000","last_unstale":"2023-10-08T09:14:37.738033+0000","last_undegraded":"2023-10-08T09:14:37.738033+0000","last_fullsized":"2023-10-08T09:14:37.738033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:24.563447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425404+0000","last_change":"2023-10-08T09:13:07.862688+0000","last_active":"2023-10-08T09:14:37.425404+0000","last_peered":"2023-10-08T09:14:37.425404+0000","last_clean":"2023-10-08T09:14:37.425404+0000","last_became_active":"2023-10-08T09:13:07.862197+0000","last_became_peered":"2023-10-08T09:13:07.862197+0000","last_unstale":"2023-10-08T09:14:37.425404+0000","last_undegraded":"2023-10-08T09:14:37.425404+0000","last_fullsized":"2023-10-08T09:14:37.425404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:56.792045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427056+0000","last_change":"2023-10-08T09:13:02.380283+0000","last_active":"2023-10-08T09:14:37.427056+0000","last_peered":"2023-10-08T09:14:37.427056+0000","last_clean":"2023-10-08T09:14:37.427056+0000","last_became_active":"2023-10-08T09:13:02.379844+0000","last_became_peered":"2023-10-08T09:13:02.379844+0000","last_unstale":"2023-10-08T09:14:37.427056+0000","last_undegraded":"2023-10-08T09:14:37.427056+0000","last_fullsized":"2023-10-08T09:14:37.427056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:18.162437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737756+0000","last_change":"2023-10-08T09:13:02.839211+0000","last_active":"2023-10-08T09:14:37.737756+0000","last_peered":"2023-10-08T09:14:37.737756+0000","last_clean":"2023-10-08T09:14:37.737756+0000","last_became_active":"2023-10-08T09:13:02.838968+0000","last_became_peered":"2023-10-08T09:13:02.838968+0000","last_unstale":"2023-10-08T09:14:37.737756+0000","last_undegraded":"2023-10-08T09:14:37.737756+0000","last_fullsized":"2023-10-08T09:14:37.737756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:24:02.541749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"86'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514099+0000","last_change":"2023-10-08T09:13:37.457701+0000","last_active":"2023-10-08T09:14:37.514099+0000","last_peered":"2023-10-08T09:14:37.514099+0000","last_clean":"2023-10-08T09:14:37.514099+0000","last_became_active":"2023-10-08T09:13:37.452309+0000","last_became_peered":"2023-10-08T09:13:37.452309+0000","last_unstale":"2023-10-08T09:14:37.514099+0000","last_undegraded":"2023-10-08T09:14:37.514099+0000","last_fullsized":"2023-10-08T09:14:37.514099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:14.898542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515550+0000","last_change":"2023-10-08T09:13:02.377027+0000","last_active":"2023-10-08T09:14:37.515550+0000","last_peered":"2023-10-08T09:14:37.515550+0000","last_clean":"2023-10-08T09:14:37.515550+0000","last_became_active":"2023-10-08T09:13:02.376746+0000","last_became_peered":"2023-10-08T09:13:02.376746+0000","last_unstale":"2023-10-08T09:14:37.515550+0000","last_undegraded":"2023-10-08T09:14:37.515550+0000","last_fullsized":"2023-10-08T09:14:37.515550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:19.062296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737868+0000","last_change":"2023-10-08T09:13:02.837279+0000","last_active":"2023-10-08T09:14:37.737868+0000","last_peered":"2023-10-08T09:14:37.737868+0000","last_clean":"2023-10-08T09:14:37.737868+0000","last_became_active":"2023-10-08T09:13:02.836966+0000","last_became_peered":"2023-10-08T09:13:02.836966+0000","last_unstale":"2023-10-08T09:14:37.737868+0000","last_undegraded":"2023-10-08T09:14:37.737868+0000","last_fullsized":"2023-10-08T09:14:37.737868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:25.462835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425457+0000","last_change":"2023-10-08T09:13:07.865538+0000","last_active":"2023-10-08T09:14:37.425457+0000","last_peered":"2023-10-08T09:14:37.425457+0000","last_clean":"2023-10-08T09:14:37.425457+0000","last_became_active":"2023-10-08T09:13:07.865118+0000","last_became_peered":"2023-10-08T09:13:07.865118+0000","last_unstale":"2023-10-08T09:14:37.425457+0000","last_undegraded":"2023-10-08T09:14:37.425457+0000","last_fullsized":"2023-10-08T09:14:37.425457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:32.167439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":246,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549867+0000","last_change":"2023-10-08T09:13:02.378173+0000","last_active":"2023-10-08T09:14:58.549867+0000","last_peered":"2023-10-08T09:14:58.549867+0000","last_clean":"2023-10-08T09:14:58.549867+0000","last_became_active":"2023-10-08T09:13:02.377880+0000","last_became_peered":"2023-10-08T09:13:02.377880+0000","last_unstale":"2023-10-08T09:14:58.549867+0000","last_undegraded":"2023-10-08T09:14:58.549867+0000","last_fullsized":"2023-10-08T09:14:58.549867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59058,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":151,"num_write":40,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535159+0000","last_change":"2023-10-08T09:13:02.829874+0000","last_active":"2023-10-08T09:14:37.535159+0000","last_peered":"2023-10-08T09:14:37.535159+0000","last_clean":"2023-10-08T09:14:37.535159+0000","last_became_active":"2023-10-08T09:13:02.829552+0000","last_became_peered":"2023-10-08T09:13:02.829552+0000","last_unstale":"2023-10-08T09:14:37.535159+0000","last_undegraded":"2023-10-08T09:14:37.535159+0000","last_fullsized":"2023-10-08T09:14:37.535159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:29.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515922+0000","last_change":"2023-10-08T09:13:07.869661+0000","last_active":"2023-10-08T09:14:37.515922+0000","last_peered":"2023-10-08T09:14:37.515922+0000","last_clean":"2023-10-08T09:14:37.515922+0000","last_became_active":"2023-10-08T09:13:07.869236+0000","last_became_peered":"2023-10-08T09:13:07.869236+0000","last_unstale":"2023-10-08T09:14:37.515922+0000","last_undegraded":"2023-10-08T09:14:37.515922+0000","last_fullsized":"2023-10-08T09:14:37.515922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:26.354461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426116+0000","last_change":"2023-10-08T09:13:02.376941+0000","last_active":"2023-10-08T09:14:37.426116+0000","last_peered":"2023-10-08T09:14:37.426116+0000","last_clean":"2023-10-08T09:14:37.426116+0000","last_became_active":"2023-10-08T09:13:02.375810+0000","last_became_peered":"2023-10-08T09:13:02.375810+0000","last_unstale":"2023-10-08T09:14:37.426116+0000","last_undegraded":"2023-10-08T09:14:37.426116+0000","last_fullsized":"2023-10-08T09:14:37.426116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:17.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511365+0000","last_change":"2023-10-08T09:13:02.836781+0000","last_active":"2023-10-08T09:14:37.511365+0000","last_peered":"2023-10-08T09:14:37.511365+0000","last_clean":"2023-10-08T09:14:37.511365+0000","last_became_active":"2023-10-08T09:13:02.836493+0000","last_became_peered":"2023-10-08T09:13:02.836493+0000","last_unstale":"2023-10-08T09:14:37.511365+0000","last_undegraded":"2023-10-08T09:14:37.511365+0000","last_fullsized":"2023-10-08T09:14:37.511365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:03.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428895+0000","last_change":"2023-10-08T09:13:07.859104+0000","last_active":"2023-10-08T09:14:37.428895+0000","last_peered":"2023-10-08T09:14:37.428895+0000","last_clean":"2023-10-08T09:14:37.428895+0000","last_became_active":"2023-10-08T09:13:07.858691+0000","last_became_peered":"2023-10-08T09:13:07.858691+0000","last_unstale":"2023-10-08T09:14:37.428895+0000","last_undegraded":"2023-10-08T09:14:37.428895+0000","last_fullsized":"2023-10-08T09:14:37.428895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:33.256044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515740+0000","last_change":"2023-10-08T09:13:02.364472+0000","last_active":"2023-10-08T09:14:37.515740+0000","last_peered":"2023-10-08T09:14:37.515740+0000","last_clean":"2023-10-08T09:14:37.515740+0000","last_became_active":"2023-10-08T09:13:02.364130+0000","last_became_peered":"2023-10-08T09:13:02.364130+0000","last_unstale":"2023-10-08T09:14:37.515740+0000","last_undegraded":"2023-10-08T09:14:37.515740+0000","last_fullsized":"2023-10-08T09:14:37.515740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:18.156186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427207+0000","last_change":"2023-10-08T09:13:02.836119+0000","last_active":"2023-10-08T09:14:37.427207+0000","last_peered":"2023-10-08T09:14:37.427207+0000","last_clean":"2023-10-08T09:14:37.427207+0000","last_became_active":"2023-10-08T09:13:02.835548+0000","last_became_peered":"2023-10-08T09:13:02.835548+0000","last_unstale":"2023-10-08T09:14:37.427207+0000","last_undegraded":"2023-10-08T09:14:37.427207+0000","last_fullsized":"2023-10-08T09:14:37.427207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:34.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426091+0000","last_change":"2023-10-08T09:13:07.871447+0000","last_active":"2023-10-08T09:14:37.426091+0000","last_peered":"2023-10-08T09:14:37.426091+0000","last_clean":"2023-10-08T09:14:37.426091+0000","last_became_active":"2023-10-08T09:13:07.871018+0000","last_became_peered":"2023-10-08T09:13:07.871018+0000","last_unstale":"2023-10-08T09:14:37.426091+0000","last_undegraded":"2023-10-08T09:14:37.426091+0000","last_fullsized":"2023-10-08T09:14:37.426091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:34.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.547617+0000","last_change":"2023-10-08T09:13:02.378954+0000","last_active":"2023-10-08T09:14:57.547617+0000","last_peered":"2023-10-08T09:14:57.547617+0000","last_clean":"2023-10-08T09:14:57.547617+0000","last_became_active":"2023-10-08T09:13:02.378635+0000","last_became_peered":"2023-10-08T09:13:02.378635+0000","last_unstale":"2023-10-08T09:14:57.547617+0000","last_undegraded":"2023-10-08T09:14:57.547617+0000","last_fullsized":"2023-10-08T09:14:57.547617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:09.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":906,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426816+0000","last_change":"2023-10-08T09:13:02.828732+0000","last_active":"2023-10-08T09:14:37.426816+0000","last_peered":"2023-10-08T09:14:37.426816+0000","last_clean":"2023-10-08T09:14:37.426816+0000","last_became_active":"2023-10-08T09:13:02.828449+0000","last_became_peered":"2023-10-08T09:13:02.828449+0000","last_unstale":"2023-10-08T09:14:37.426816+0000","last_undegraded":"2023-10-08T09:14:37.426816+0000","last_fullsized":"2023-10-08T09:14:37.426816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:09.883908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511958+0000","last_change":"2023-10-08T09:13:07.869610+0000","last_active":"2023-10-08T09:14:37.511958+0000","last_peered":"2023-10-08T09:14:37.511958+0000","last_clean":"2023-10-08T09:14:37.511958+0000","last_became_active":"2023-10-08T09:13:07.869225+0000","last_became_peered":"2023-10-08T09:13:07.869225+0000","last_unstale":"2023-10-08T09:14:37.511958+0000","last_undegraded":"2023-10-08T09:14:37.511958+0000","last_fullsized":"2023-10-08T09:14:37.511958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:35.225452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534680+0000","last_change":"2023-10-08T09:13:02.375658+0000","last_active":"2023-10-08T09:14:37.534680+0000","last_peered":"2023-10-08T09:14:37.534680+0000","last_clean":"2023-10-08T09:14:37.534680+0000","last_became_active":"2023-10-08T09:13:02.375383+0000","last_became_peered":"2023-10-08T09:13:02.375383+0000","last_unstale":"2023-10-08T09:14:37.534680+0000","last_undegraded":"2023-10-08T09:14:37.534680+0000","last_fullsized":"2023-10-08T09:14:37.534680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:52.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515917+0000","last_change":"2023-10-08T09:13:37.455645+0000","last_active":"2023-10-08T09:14:37.515917+0000","last_peered":"2023-10-08T09:14:37.515917+0000","last_clean":"2023-10-08T09:14:37.515917+0000","last_became_active":"2023-10-08T09:13:37.455207+0000","last_became_peered":"2023-10-08T09:13:37.455207+0000","last_unstale":"2023-10-08T09:14:37.515917+0000","last_undegraded":"2023-10-08T09:14:37.515917+0000","last_fullsized":"2023-10-08T09:14:37.515917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:18.021608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515352+0000","last_change":"2023-10-08T09:13:07.871441+0000","last_active":"2023-10-08T09:14:37.515352+0000","last_peered":"2023-10-08T09:14:37.515352+0000","last_clean":"2023-10-08T09:14:37.515352+0000","last_became_active":"2023-10-08T09:13:07.870968+0000","last_became_peered":"2023-10-08T09:13:07.870968+0000","last_unstale":"2023-10-08T09:14:37.515352+0000","last_undegraded":"2023-10-08T09:14:37.515352+0000","last_fullsized":"2023-10-08T09:14:37.515352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:48.746147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535222+0000","last_change":"2023-10-08T09:13:02.372514+0000","last_active":"2023-10-08T09:14:37.535222+0000","last_peered":"2023-10-08T09:14:37.535222+0000","last_clean":"2023-10-08T09:14:37.535222+0000","last_became_active":"2023-10-08T09:13:02.372181+0000","last_became_peered":"2023-10-08T09:13:02.372181+0000","last_unstale":"2023-10-08T09:14:37.535222+0000","last_undegraded":"2023-10-08T09:14:37.535222+0000","last_fullsized":"2023-10-08T09:14:37.535222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:28.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429168+0000","last_change":"2023-10-08T09:13:02.835220+0000","last_active":"2023-10-08T09:14:37.429168+0000","last_peered":"2023-10-08T09:14:37.429168+0000","last_clean":"2023-10-08T09:14:37.429168+0000","last_became_active":"2023-10-08T09:13:02.834914+0000","last_became_peered":"2023-10-08T09:13:02.834914+0000","last_unstale":"2023-10-08T09:14:37.429168+0000","last_undegraded":"2023-10-08T09:14:37.429168+0000","last_fullsized":"2023-10-08T09:14:37.429168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:37.447418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535013+0000","last_change":"2023-10-08T09:13:08.096146+0000","last_active":"2023-10-08T09:14:37.535013+0000","last_peered":"2023-10-08T09:14:37.535013+0000","last_clean":"2023-10-08T09:14:37.535013+0000","last_became_active":"2023-10-08T09:13:08.095703+0000","last_became_peered":"2023-10-08T09:13:08.095703+0000","last_unstale":"2023-10-08T09:14:37.535013+0000","last_undegraded":"2023-10-08T09:14:37.535013+0000","last_fullsized":"2023-10-08T09:14:37.535013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:37.511094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425782+0000","last_change":"2023-10-08T09:13:02.372836+0000","last_active":"2023-10-08T09:14:37.425782+0000","last_peered":"2023-10-08T09:14:37.425782+0000","last_clean":"2023-10-08T09:14:37.425782+0000","last_became_active":"2023-10-08T09:13:02.372488+0000","last_became_peered":"2023-10-08T09:13:02.372488+0000","last_unstale":"2023-10-08T09:14:37.425782+0000","last_undegraded":"2023-10-08T09:14:37.425782+0000","last_fullsized":"2023-10-08T09:14:37.425782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:19.269539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512564+0000","last_change":"2023-10-08T09:13:02.827306+0000","last_active":"2023-10-08T09:14:37.512564+0000","last_peered":"2023-10-08T09:14:37.512564+0000","last_clean":"2023-10-08T09:14:37.512564+0000","last_became_active":"2023-10-08T09:13:02.826965+0000","last_became_peered":"2023-10-08T09:13:02.826965+0000","last_unstale":"2023-10-08T09:14:37.512564+0000","last_undegraded":"2023-10-08T09:14:37.512564+0000","last_fullsized":"2023-10-08T09:14:37.512564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:27.513860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511590+0000","last_change":"2023-10-08T09:13:07.862637+0000","last_active":"2023-10-08T09:14:37.511590+0000","last_peered":"2023-10-08T09:14:37.511590+0000","last_clean":"2023-10-08T09:14:37.511590+0000","last_became_active":"2023-10-08T09:13:07.862117+0000","last_became_peered":"2023-10-08T09:13:07.862117+0000","last_unstale":"2023-10-08T09:14:37.511590+0000","last_undegraded":"2023-10-08T09:14:37.511590+0000","last_fullsized":"2023-10-08T09:14:37.511590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:14.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:59.561747+0000","last_change":"2023-10-08T09:13:02.379585+0000","last_active":"2023-10-08T09:14:59.561747+0000","last_peered":"2023-10-08T09:14:59.561747+0000","last_clean":"2023-10-08T09:14:59.561747+0000","last_became_active":"2023-10-08T09:13:02.379324+0000","last_became_peered":"2023-10-08T09:13:02.379324+0000","last_unstale":"2023-10-08T09:14:59.561747+0000","last_undegraded":"2023-10-08T09:14:59.561747+0000","last_fullsized":"2023-10-08T09:14:59.561747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:31.609558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512613+0000","last_change":"2023-10-08T09:13:02.833416+0000","last_active":"2023-10-08T09:14:37.512613+0000","last_peered":"2023-10-08T09:14:37.512613+0000","last_clean":"2023-10-08T09:14:37.512613+0000","last_became_active":"2023-10-08T09:13:02.833040+0000","last_became_peered":"2023-10-08T09:13:02.833040+0000","last_unstale":"2023-10-08T09:14:37.512613+0000","last_undegraded":"2023-10-08T09:14:37.512613+0000","last_fullsized":"2023-10-08T09:14:37.512613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:44.480445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511906+0000","last_change":"2023-10-08T09:13:08.095723+0000","last_active":"2023-10-08T09:14:37.511906+0000","last_peered":"2023-10-08T09:14:37.511906+0000","last_clean":"2023-10-08T09:14:37.511906+0000","last_became_active":"2023-10-08T09:13:08.095271+0000","last_became_peered":"2023-10-08T09:13:08.095271+0000","last_unstale":"2023-10-08T09:14:37.511906+0000","last_undegraded":"2023-10-08T09:14:37.511906+0000","last_fullsized":"2023-10-08T09:14:37.511906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:04.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515464+0000","last_change":"2023-10-08T09:13:02.829663+0000","last_active":"2023-10-08T09:14:37.515464+0000","last_peered":"2023-10-08T09:14:37.515464+0000","last_clean":"2023-10-08T09:14:37.515464+0000","last_became_active":"2023-10-08T09:13:02.829354+0000","last_became_peered":"2023-10-08T09:13:02.829354+0000","last_unstale":"2023-10-08T09:14:37.515464+0000","last_undegraded":"2023-10-08T09:14:37.515464+0000","last_fullsized":"2023-10-08T09:14:37.515464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:55.332346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549710+0000","last_change":"2023-10-08T09:13:02.372114+0000","last_active":"2023-10-08T09:14:58.549710+0000","last_peered":"2023-10-08T09:14:58.549710+0000","last_clean":"2023-10-08T09:14:58.549710+0000","last_became_active":"2023-10-08T09:13:02.371603+0000","last_became_peered":"2023-10-08T09:13:02.371603+0000","last_unstale":"2023-10-08T09:14:58.549710+0000","last_undegraded":"2023-10-08T09:14:58.549710+0000","last_fullsized":"2023-10-08T09:14:58.549710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:39.078307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427099+0000","last_change":"2023-10-08T09:13:07.863492+0000","last_active":"2023-10-08T09:14:37.427099+0000","last_peered":"2023-10-08T09:14:37.427099+0000","last_clean":"2023-10-08T09:14:37.427099+0000","last_became_active":"2023-10-08T09:13:07.862803+0000","last_became_peered":"2023-10-08T09:13:07.862803+0000","last_unstale":"2023-10-08T09:14:37.427099+0000","last_undegraded":"2023-10-08T09:14:37.427099+0000","last_fullsized":"2023-10-08T09:14:37.427099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:39.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428936+0000","last_change":"2023-10-08T09:13:02.836596+0000","last_active":"2023-10-08T09:14:37.428936+0000","last_peered":"2023-10-08T09:14:37.428936+0000","last_clean":"2023-10-08T09:14:37.428936+0000","last_became_active":"2023-10-08T09:13:02.836283+0000","last_became_peered":"2023-10-08T09:13:02.836283+0000","last_unstale":"2023-10-08T09:14:37.428936+0000","last_undegraded":"2023-10-08T09:14:37.428936+0000","last_fullsized":"2023-10-08T09:14:37.428936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:02.393575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514133+0000","last_change":"2023-10-08T09:13:02.379516+0000","last_active":"2023-10-08T09:14:37.514133+0000","last_peered":"2023-10-08T09:14:37.514133+0000","last_clean":"2023-10-08T09:14:37.514133+0000","last_became_active":"2023-10-08T09:13:02.379229+0000","last_became_peered":"2023-10-08T09:13:02.379229+0000","last_unstale":"2023-10-08T09:14:37.514133+0000","last_undegraded":"2023-10-08T09:14:37.514133+0000","last_fullsized":"2023-10-08T09:14:37.514133+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:55.294651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.533330+0000","last_change":"2023-10-08T09:12:02.620769+0000","last_active":"2023-10-08T09:14:38.533330+0000","last_peered":"2023-10-08T09:14:38.533330+0000","last_clean":"2023-10-08T09:14:38.533330+0000","last_became_active":"2023-10-08T09:12:02.620388+0000","last_became_peered":"2023-10-08T09:12:02.620388+0000","last_unstale":"2023-10-08T09:14:38.533330+0000","last_undegraded":"2023-10-08T09:14:38.533330+0000","last_fullsized":"2023-10-08T09:14:38.533330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_clean_scrub_stamp":"2023-10-08T09:09:10.746288+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:55:35.713060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515785+0000","last_change":"2023-10-08T09:13:08.096318+0000","last_active":"2023-10-08T09:14:37.515785+0000","last_peered":"2023-10-08T09:14:37.515785+0000","last_clean":"2023-10-08T09:14:37.515785+0000","last_became_active":"2023-10-08T09:13:08.095774+0000","last_became_peered":"2023-10-08T09:13:08.095774+0000","last_unstale":"2023-10-08T09:14:37.515785+0000","last_undegraded":"2023-10-08T09:14:37.515785+0000","last_fullsized":"2023-10-08T09:14:37.515785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:53.089468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511157+0000","last_change":"2023-10-08T09:13:02.835167+0000","last_active":"2023-10-08T09:14:37.511157+0000","last_peered":"2023-10-08T09:14:37.511157+0000","last_clean":"2023-10-08T09:14:37.511157+0000","last_became_active":"2023-10-08T09:13:02.834850+0000","last_became_peered":"2023-10-08T09:13:02.834850+0000","last_unstale":"2023-10-08T09:14:37.511157+0000","last_undegraded":"2023-10-08T09:14:37.511157+0000","last_fullsized":"2023-10-08T09:14:37.511157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:19.422372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426957+0000","last_change":"2023-10-08T09:13:02.377746+0000","last_active":"2023-10-08T09:14:37.426957+0000","last_peered":"2023-10-08T09:14:37.426957+0000","last_clean":"2023-10-08T09:14:37.426957+0000","last_became_active":"2023-10-08T09:13:02.377422+0000","last_became_peered":"2023-10-08T09:13:02.377422+0000","last_unstale":"2023-10-08T09:14:37.426957+0000","last_undegraded":"2023-10-08T09:14:37.426957+0000","last_fullsized":"2023-10-08T09:14:37.426957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:13.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428886+0000","last_change":"2023-10-08T09:13:07.864678+0000","last_active":"2023-10-08T09:14:37.428886+0000","last_peered":"2023-10-08T09:14:37.428886+0000","last_clean":"2023-10-08T09:14:37.428886+0000","last_became_active":"2023-10-08T09:13:07.864226+0000","last_became_peered":"2023-10-08T09:13:07.864226+0000","last_unstale":"2023-10-08T09:14:37.428886+0000","last_undegraded":"2023-10-08T09:14:37.428886+0000","last_fullsized":"2023-10-08T09:14:37.428886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:43:51.179853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535266+0000","last_change":"2023-10-08T09:13:02.370014+0000","last_active":"2023-10-08T09:14:37.535266+0000","last_peered":"2023-10-08T09:14:37.535266+0000","last_clean":"2023-10-08T09:14:37.535266+0000","last_became_active":"2023-10-08T09:13:02.369704+0000","last_became_peered":"2023-10-08T09:13:02.369704+0000","last_unstale":"2023-10-08T09:14:37.535266+0000","last_undegraded":"2023-10-08T09:14:37.535266+0000","last_fullsized":"2023-10-08T09:14:37.535266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:28.902274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511727+0000","last_change":"2023-10-08T09:13:02.833859+0000","last_active":"2023-10-08T09:14:37.511727+0000","last_peered":"2023-10-08T09:14:37.511727+0000","last_clean":"2023-10-08T09:14:37.511727+0000","last_became_active":"2023-10-08T09:13:02.833546+0000","last_became_peered":"2023-10-08T09:13:02.833546+0000","last_unstale":"2023-10-08T09:14:37.511727+0000","last_undegraded":"2023-10-08T09:14:37.511727+0000","last_fullsized":"2023-10-08T09:14:37.511727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:35.534588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428821+0000","last_change":"2023-10-08T09:13:08.095622+0000","last_active":"2023-10-08T09:14:37.428821+0000","last_peered":"2023-10-08T09:14:37.428821+0000","last_clean":"2023-10-08T09:14:37.428821+0000","last_became_active":"2023-10-08T09:13:08.095143+0000","last_became_peered":"2023-10-08T09:13:08.095143+0000","last_unstale":"2023-10-08T09:14:37.428821+0000","last_undegraded":"2023-10-08T09:14:37.428821+0000","last_fullsized":"2023-10-08T09:14:37.428821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:46.738102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426121+0000","last_change":"2023-10-08T09:13:37.459185+0000","last_active":"2023-10-08T09:14:37.426121+0000","last_peered":"2023-10-08T09:14:37.426121+0000","last_clean":"2023-10-08T09:14:37.426121+0000","last_became_active":"2023-10-08T09:13:37.451105+0000","last_became_peered":"2023-10-08T09:13:37.451105+0000","last_unstale":"2023-10-08T09:14:37.426121+0000","last_undegraded":"2023-10-08T09:14:37.426121+0000","last_fullsized":"2023-10-08T09:14:37.426121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.768269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429624+0000","last_change":"2023-10-08T09:13:02.834053+0000","last_active":"2023-10-08T09:14:37.429624+0000","last_peered":"2023-10-08T09:14:37.429624+0000","last_clean":"2023-10-08T09:14:37.429624+0000","last_became_active":"2023-10-08T09:13:02.833707+0000","last_became_peered":"2023-10-08T09:13:02.833707+0000","last_unstale":"2023-10-08T09:14:37.429624+0000","last_undegraded":"2023-10-08T09:14:37.429624+0000","last_fullsized":"2023-10-08T09:14:37.429624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:31.372003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425862+0000","last_change":"2023-10-08T09:13:07.855734+0000","last_active":"2023-10-08T09:14:37.425862+0000","last_peered":"2023-10-08T09:14:37.425862+0000","last_clean":"2023-10-08T09:14:37.425862+0000","last_became_active":"2023-10-08T09:13:07.855313+0000","last_became_peered":"2023-10-08T09:13:07.855313+0000","last_unstale":"2023-10-08T09:14:37.425862+0000","last_undegraded":"2023-10-08T09:14:37.425862+0000","last_fullsized":"2023-10-08T09:14:37.425862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:15.207337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514427+0000","last_change":"2023-10-08T09:13:02.371319+0000","last_active":"2023-10-08T09:14:37.514427+0000","last_peered":"2023-10-08T09:14:37.514427+0000","last_clean":"2023-10-08T09:14:37.514427+0000","last_became_active":"2023-10-08T09:13:02.371049+0000","last_became_peered":"2023-10-08T09:13:02.371049+0000","last_unstale":"2023-10-08T09:14:37.514427+0000","last_undegraded":"2023-10-08T09:14:37.514427+0000","last_fullsized":"2023-10-08T09:14:37.514427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:28.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737818+0000","last_change":"2023-10-08T09:13:02.838682+0000","last_active":"2023-10-08T09:14:37.737818+0000","last_peered":"2023-10-08T09:14:37.737818+0000","last_clean":"2023-10-08T09:14:37.737818+0000","last_became_active":"2023-10-08T09:13:02.838389+0000","last_became_peered":"2023-10-08T09:13:02.838389+0000","last_unstale":"2023-10-08T09:14:37.737818+0000","last_undegraded":"2023-10-08T09:14:37.737818+0000","last_fullsized":"2023-10-08T09:14:37.737818+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:52.937723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514169+0000","last_change":"2023-10-08T09:13:08.098131+0000","last_active":"2023-10-08T09:14:37.514169+0000","last_peered":"2023-10-08T09:14:37.514169+0000","last_clean":"2023-10-08T09:14:37.514169+0000","last_became_active":"2023-10-08T09:13:08.097352+0000","last_became_peered":"2023-10-08T09:13:08.097352+0000","last_unstale":"2023-10-08T09:14:37.514169+0000","last_undegraded":"2023-10-08T09:14:37.514169+0000","last_fullsized":"2023-10-08T09:14:37.514169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:30.388875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737576+0000","last_change":"2023-10-08T09:13:37.461067+0000","last_active":"2023-10-08T09:14:37.737576+0000","last_peered":"2023-10-08T09:14:37.737576+0000","last_clean":"2023-10-08T09:14:37.737576+0000","last_became_active":"2023-10-08T09:13:37.451739+0000","last_became_peered":"2023-10-08T09:13:37.451739+0000","last_unstale":"2023-10-08T09:14:37.737576+0000","last_undegraded":"2023-10-08T09:14:37.737576+0000","last_fullsized":"2023-10-08T09:14:37.737576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:26.603798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534742+0000","last_change":"2023-10-08T09:13:02.830017+0000","last_active":"2023-10-08T09:14:37.534742+0000","last_peered":"2023-10-08T09:14:37.534742+0000","last_clean":"2023-10-08T09:14:37.534742+0000","last_became_active":"2023-10-08T09:13:02.829700+0000","last_became_peered":"2023-10-08T09:13:02.829700+0000","last_unstale":"2023-10-08T09:14:37.534742+0000","last_undegraded":"2023-10-08T09:14:37.534742+0000","last_fullsized":"2023-10-08T09:14:37.534742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:31:31.387049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426196+0000","last_change":"2023-10-08T09:13:07.869097+0000","last_active":"2023-10-08T09:14:37.426196+0000","last_peered":"2023-10-08T09:14:37.426196+0000","last_clean":"2023-10-08T09:14:37.426196+0000","last_became_active":"2023-10-08T09:13:07.868710+0000","last_became_peered":"2023-10-08T09:13:07.868710+0000","last_unstale":"2023-10-08T09:14:37.426196+0000","last_undegraded":"2023-10-08T09:14:37.426196+0000","last_fullsized":"2023-10-08T09:14:37.426196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:37:27.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429207+0000","last_change":"2023-10-08T09:13:02.370694+0000","last_active":"2023-10-08T09:14:37.429207+0000","last_peered":"2023-10-08T09:14:37.429207+0000","last_clean":"2023-10-08T09:14:37.429207+0000","last_became_active":"2023-10-08T09:13:02.370161+0000","last_became_peered":"2023-10-08T09:13:02.370161+0000","last_unstale":"2023-10-08T09:14:37.429207+0000","last_undegraded":"2023-10-08T09:14:37.429207+0000","last_fullsized":"2023-10-08T09:14:37.429207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:04.809703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428791+0000","last_change":"2023-10-08T09:13:02.831026+0000","last_active":"2023-10-08T09:14:37.428791+0000","last_peered":"2023-10-08T09:14:37.428791+0000","last_clean":"2023-10-08T09:14:37.428791+0000","last_became_active":"2023-10-08T09:13:02.830715+0000","last_became_peered":"2023-10-08T09:13:02.830715+0000","last_unstale":"2023-10-08T09:14:37.428791+0000","last_undegraded":"2023-10-08T09:14:37.428791+0000","last_fullsized":"2023-10-08T09:14:37.428791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:47.709820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514173+0000","last_change":"2023-10-08T09:13:07.860464+0000","last_active":"2023-10-08T09:14:37.514173+0000","last_peered":"2023-10-08T09:14:37.514173+0000","last_clean":"2023-10-08T09:14:37.514173+0000","last_became_active":"2023-10-08T09:13:07.859962+0000","last_became_peered":"2023-10-08T09:13:07.859962+0000","last_unstale":"2023-10-08T09:14:37.514173+0000","last_undegraded":"2023-10-08T09:14:37.514173+0000","last_fullsized":"2023-10-08T09:14:37.514173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:04.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426052+0000","last_change":"2023-10-08T09:13:02.371308+0000","last_active":"2023-10-08T09:14:37.426052+0000","last_peered":"2023-10-08T09:14:37.426052+0000","last_clean":"2023-10-08T09:14:37.426052+0000","last_became_active":"2023-10-08T09:13:02.370968+0000","last_became_peered":"2023-10-08T09:13:02.370968+0000","last_unstale":"2023-10-08T09:14:37.426052+0000","last_undegraded":"2023-10-08T09:14:37.426052+0000","last_fullsized":"2023-10-08T09:14:37.426052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:58.491112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429125+0000","last_change":"2023-10-08T09:13:02.826511+0000","last_active":"2023-10-08T09:14:37.429125+0000","last_peered":"2023-10-08T09:14:37.429125+0000","last_clean":"2023-10-08T09:14:37.429125+0000","last_became_active":"2023-10-08T09:13:02.826205+0000","last_became_peered":"2023-10-08T09:13:02.826205+0000","last_unstale":"2023-10-08T09:14:37.429125+0000","last_undegraded":"2023-10-08T09:14:37.429125+0000","last_fullsized":"2023-10-08T09:14:37.429125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:12.270176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514117+0000","last_change":"2023-10-08T09:13:08.096360+0000","last_active":"2023-10-08T09:14:37.514117+0000","last_peered":"2023-10-08T09:14:37.514117+0000","last_clean":"2023-10-08T09:14:37.514117+0000","last_became_active":"2023-10-08T09:13:08.095915+0000","last_became_peered":"2023-10-08T09:13:08.095915+0000","last_unstale":"2023-10-08T09:14:37.514117+0000","last_undegraded":"2023-10-08T09:14:37.514117+0000","last_fullsized":"2023-10-08T09:14:37.514117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:59.412471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534927+0000","last_change":"2023-10-08T09:13:02.375287+0000","last_active":"2023-10-08T09:14:37.534927+0000","last_peered":"2023-10-08T09:14:37.534927+0000","last_clean":"2023-10-08T09:14:37.534927+0000","last_became_active":"2023-10-08T09:13:02.374668+0000","last_became_peered":"2023-10-08T09:13:02.374668+0000","last_unstale":"2023-10-08T09:14:37.534927+0000","last_undegraded":"2023-10-08T09:14:37.534927+0000","last_fullsized":"2023-10-08T09:14:37.534927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:22.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426986+0000","last_change":"2023-10-08T09:13:02.828240+0000","last_active":"2023-10-08T09:14:37.426986+0000","last_peered":"2023-10-08T09:14:37.426986+0000","last_clean":"2023-10-08T09:14:37.426986+0000","last_became_active":"2023-10-08T09:13:02.827951+0000","last_became_peered":"2023-10-08T09:13:02.827951+0000","last_unstale":"2023-10-08T09:14:37.426986+0000","last_undegraded":"2023-10-08T09:14:37.426986+0000","last_fullsized":"2023-10-08T09:14:37.426986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:37.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737603+0000","last_change":"2023-10-08T09:13:07.865846+0000","last_active":"2023-10-08T09:14:37.737603+0000","last_peered":"2023-10-08T09:14:37.737603+0000","last_clean":"2023-10-08T09:14:37.737603+0000","last_became_active":"2023-10-08T09:13:07.865529+0000","last_became_peered":"2023-10-08T09:13:07.865529+0000","last_unstale":"2023-10-08T09:14:37.737603+0000","last_undegraded":"2023-10-08T09:14:37.737603+0000","last_fullsized":"2023-10-08T09:14:37.737603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:34:03.408036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:59.571238+0000","last_change":"2023-10-08T09:13:02.380128+0000","last_active":"2023-10-08T09:14:59.571238+0000","last_peered":"2023-10-08T09:14:59.571238+0000","last_clean":"2023-10-08T09:14:59.571238+0000","last_became_active":"2023-10-08T09:13:02.379800+0000","last_became_peered":"2023-10-08T09:13:02.379800+0000","last_unstale":"2023-10-08T09:14:59.571238+0000","last_undegraded":"2023-10-08T09:14:59.571238+0000","last_fullsized":"2023-10-08T09:14:59.571238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:01.546289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427243+0000","last_change":"2023-10-08T09:13:02.829474+0000","last_active":"2023-10-08T09:14:37.427243+0000","last_peered":"2023-10-08T09:14:37.427243+0000","last_clean":"2023-10-08T09:14:37.427243+0000","last_became_active":"2023-10-08T09:13:02.829173+0000","last_became_peered":"2023-10-08T09:13:02.829173+0000","last_unstale":"2023-10-08T09:14:37.427243+0000","last_undegraded":"2023-10-08T09:14:37.427243+0000","last_fullsized":"2023-10-08T09:14:37.427243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:02.469158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737465+0000","last_change":"2023-10-08T09:13:37.453778+0000","last_active":"2023-10-08T09:14:37.737465+0000","last_peered":"2023-10-08T09:14:37.737465+0000","last_clean":"2023-10-08T09:14:37.737465+0000","last_became_active":"2023-10-08T09:13:37.453354+0000","last_became_peered":"2023-10-08T09:13:37.453354+0000","last_unstale":"2023-10-08T09:14:37.737465+0000","last_undegraded":"2023-10-08T09:14:37.737465+0000","last_fullsized":"2023-10-08T09:14:37.737465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:50.035588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515804+0000","last_change":"2023-10-08T09:13:02.376641+0000","last_active":"2023-10-08T09:14:37.515804+0000","last_peered":"2023-10-08T09:14:37.515804+0000","last_clean":"2023-10-08T09:14:37.515804+0000","last_became_active":"2023-10-08T09:13:02.376348+0000","last_became_peered":"2023-10-08T09:13:02.376348+0000","last_unstale":"2023-10-08T09:14:37.515804+0000","last_undegraded":"2023-10-08T09:14:37.515804+0000","last_fullsized":"2023-10-08T09:14:37.515804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:42.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427333+0000","last_change":"2023-10-08T09:13:02.836201+0000","last_active":"2023-10-08T09:14:37.427333+0000","last_peered":"2023-10-08T09:14:37.427333+0000","last_clean":"2023-10-08T09:14:37.427333+0000","last_became_active":"2023-10-08T09:13:02.835885+0000","last_became_peered":"2023-10-08T09:13:02.835885+0000","last_unstale":"2023-10-08T09:14:37.427333+0000","last_undegraded":"2023-10-08T09:14:37.427333+0000","last_fullsized":"2023-10-08T09:14:37.427333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:34.310854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515138+0000","last_change":"2023-10-08T09:13:07.863854+0000","last_active":"2023-10-08T09:14:37.515138+0000","last_peered":"2023-10-08T09:14:37.515138+0000","last_clean":"2023-10-08T09:14:37.515138+0000","last_became_active":"2023-10-08T09:13:07.863460+0000","last_became_peered":"2023-10-08T09:13:07.863460+0000","last_unstale":"2023-10-08T09:14:37.515138+0000","last_undegraded":"2023-10-08T09:14:37.515138+0000","last_fullsized":"2023-10-08T09:14:37.515138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:03.093858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511956+0000","last_change":"2023-10-08T09:13:02.372336+0000","last_active":"2023-10-08T09:14:37.511956+0000","last_peered":"2023-10-08T09:14:37.511956+0000","last_clean":"2023-10-08T09:14:37.511956+0000","last_became_active":"2023-10-08T09:13:02.372022+0000","last_became_peered":"2023-10-08T09:13:02.372022+0000","last_unstale":"2023-10-08T09:14:37.511956+0000","last_undegraded":"2023-10-08T09:14:37.511956+0000","last_fullsized":"2023-10-08T09:14:37.511956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:10.610332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427293+0000","last_change":"2023-10-08T09:13:02.832430+0000","last_active":"2023-10-08T09:14:37.427293+0000","last_peered":"2023-10-08T09:14:37.427293+0000","last_clean":"2023-10-08T09:14:37.427293+0000","last_became_active":"2023-10-08T09:13:02.832092+0000","last_became_peered":"2023-10-08T09:13:02.832092+0000","last_unstale":"2023-10-08T09:14:37.427293+0000","last_undegraded":"2023-10-08T09:14:37.427293+0000","last_fullsized":"2023-10-08T09:14:37.427293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:00.798900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428785+0000","last_change":"2023-10-08T09:13:02.833114+0000","last_active":"2023-10-08T09:14:37.428785+0000","last_peered":"2023-10-08T09:14:37.428785+0000","last_clean":"2023-10-08T09:14:37.428785+0000","last_became_active":"2023-10-08T09:13:02.830959+0000","last_became_peered":"2023-10-08T09:13:02.830959+0000","last_unstale":"2023-10-08T09:14:37.428785+0000","last_undegraded":"2023-10-08T09:14:37.428785+0000","last_fullsized":"2023-10-08T09:14:37.428785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:13.853585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426056+0000","last_change":"2023-10-08T09:13:02.836046+0000","last_active":"2023-10-08T09:14:37.426056+0000","last_peered":"2023-10-08T09:14:37.426056+0000","last_clean":"2023-10-08T09:14:37.426056+0000","last_became_active":"2023-10-08T09:13:02.835722+0000","last_became_peered":"2023-10-08T09:13:02.835722+0000","last_unstale":"2023-10-08T09:14:37.426056+0000","last_undegraded":"2023-10-08T09:14:37.426056+0000","last_fullsized":"2023-10-08T09:14:37.426056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:00.316259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425931+0000","last_change":"2023-10-08T09:13:02.830021+0000","last_active":"2023-10-08T09:14:37.425931+0000","last_peered":"2023-10-08T09:14:37.425931+0000","last_clean":"2023-10-08T09:14:37.425931+0000","last_became_active":"2023-10-08T09:13:02.829397+0000","last_became_peered":"2023-10-08T09:13:02.829397+0000","last_unstale":"2023-10-08T09:14:37.425931+0000","last_undegraded":"2023-10-08T09:14:37.425931+0000","last_fullsized":"2023-10-08T09:14:37.425931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:29.153457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737569+0000","last_change":"2023-10-08T09:13:02.832002+0000","last_active":"2023-10-08T09:14:37.737569+0000","last_peered":"2023-10-08T09:14:37.737569+0000","last_clean":"2023-10-08T09:14:37.737569+0000","last_became_active":"2023-10-08T09:13:02.831666+0000","last_became_peered":"2023-10-08T09:13:02.831666+0000","last_unstale":"2023-10-08T09:14:37.737569+0000","last_undegraded":"2023-10-08T09:14:37.737569+0000","last_fullsized":"2023-10-08T09:14:37.737569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:16.414127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511793+0000","last_change":"2023-10-08T09:13:02.835881+0000","last_active":"2023-10-08T09:14:37.511793+0000","last_peered":"2023-10-08T09:14:37.511793+0000","last_clean":"2023-10-08T09:14:37.511793+0000","last_became_active":"2023-10-08T09:13:02.835587+0000","last_became_peered":"2023-10-08T09:13:02.835587+0000","last_unstale":"2023-10-08T09:14:37.511793+0000","last_undegraded":"2023-10-08T09:14:37.511793+0000","last_fullsized":"2023-10-08T09:14:37.511793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:52.323234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429154+0000","last_change":"2023-10-08T09:13:02.826371+0000","last_active":"2023-10-08T09:14:37.429154+0000","last_peered":"2023-10-08T09:14:37.429154+0000","last_clean":"2023-10-08T09:14:37.429154+0000","last_became_active":"2023-10-08T09:13:02.826093+0000","last_became_peered":"2023-10-08T09:13:02.826093+0000","last_unstale":"2023-10-08T09:14:37.429154+0000","last_undegraded":"2023-10-08T09:14:37.429154+0000","last_fullsized":"2023-10-08T09:14:37.429154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:08.113739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426543+0000","last_change":"2023-10-08T09:13:02.829936+0000","last_active":"2023-10-08T09:14:37.426543+0000","last_peered":"2023-10-08T09:14:37.426543+0000","last_clean":"2023-10-08T09:14:37.426543+0000","last_became_active":"2023-10-08T09:13:02.829498+0000","last_became_peered":"2023-10-08T09:13:02.829498+0000","last_unstale":"2023-10-08T09:14:37.426543+0000","last_undegraded":"2023-10-08T09:14:37.426543+0000","last_fullsized":"2023-10-08T09:14:37.426543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:42.200791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429728+0000","last_change":"2023-10-08T09:13:02.840415+0000","last_active":"2023-10-08T09:14:37.429728+0000","last_peered":"2023-10-08T09:14:37.429728+0000","last_clean":"2023-10-08T09:14:37.429728+0000","last_became_active":"2023-10-08T09:13:02.840075+0000","last_became_peered":"2023-10-08T09:13:02.840075+0000","last_unstale":"2023-10-08T09:14:37.429728+0000","last_undegraded":"2023-10-08T09:14:37.429728+0000","last_fullsized":"2023-10-08T09:14:37.429728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:48.959372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515335+0000","last_change":"2023-10-08T09:13:02.828337+0000","last_active":"2023-10-08T09:14:37.515335+0000","last_peered":"2023-10-08T09:14:37.515335+0000","last_clean":"2023-10-08T09:14:37.515335+0000","last_became_active":"2023-10-08T09:13:02.827998+0000","last_became_peered":"2023-10-08T09:13:02.827998+0000","last_unstale":"2023-10-08T09:14:37.515335+0000","last_undegraded":"2023-10-08T09:14:37.515335+0000","last_fullsized":"2023-10-08T09:14:37.515335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:27.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.547574+0000","last_change":"2023-10-08T09:14:38.547245+0000","last_active":"2023-10-08T09:14:38.547574+0000","last_peered":"2023-10-08T09:14:38.547574+0000","last_clean":"2023-10-08T09:14:38.547574+0000","last_became_active":"2023-10-08T09:14:38.546964+0000","last_became_peered":"2023-10-08T09:14:38.546964+0000","last_unstale":"2023-10-08T09:14:38.547574+0000","last_undegraded":"2023-10-08T09:14:38.547574+0000","last_fullsized":"2023-10-08T09:14:38.547574+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:40.120946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737724+0000","last_change":"2023-10-08T09:13:02.840921+0000","last_active":"2023-10-08T09:14:37.737724+0000","last_peered":"2023-10-08T09:14:37.737724+0000","last_clean":"2023-10-08T09:14:37.737724+0000","last_became_active":"2023-10-08T09:13:02.840566+0000","last_became_peered":"2023-10-08T09:13:02.840566+0000","last_unstale":"2023-10-08T09:14:37.737724+0000","last_undegraded":"2023-10-08T09:14:37.737724+0000","last_fullsized":"2023-10-08T09:14:37.737724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:20.219581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511631+0000","last_change":"2023-10-08T09:13:02.834491+0000","last_active":"2023-10-08T09:14:37.511631+0000","last_peered":"2023-10-08T09:14:37.511631+0000","last_clean":"2023-10-08T09:14:37.511631+0000","last_became_active":"2023-10-08T09:13:02.834058+0000","last_became_peered":"2023-10-08T09:13:02.834058+0000","last_unstale":"2023-10-08T09:14:37.511631+0000","last_undegraded":"2023-10-08T09:14:37.511631+0000","last_fullsized":"2023-10-08T09:14:37.511631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:47.699423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428850+0000","last_change":"2023-10-08T09:13:02.836433+0000","last_active":"2023-10-08T09:14:37.428850+0000","last_peered":"2023-10-08T09:14:37.428850+0000","last_clean":"2023-10-08T09:14:37.428850+0000","last_became_active":"2023-10-08T09:13:02.835147+0000","last_became_peered":"2023-10-08T09:13:02.835147+0000","last_unstale":"2023-10-08T09:14:37.428850+0000","last_undegraded":"2023-10-08T09:14:37.428850+0000","last_fullsized":"2023-10-08T09:14:37.428850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:51.233662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247538,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66860,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":919,"num_read_kb":533,"num_write":247,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514532,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":507696,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69999999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59699999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90100000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.81799999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86699999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.77400000000000002,"5min":1.147,"15min":1.147},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.62,"15min":0.63},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83799999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.78700000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0389999999999999,"5min":2.222,"15min":2.222},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.77200000000000002,"15min":0.79100000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.85099999999999998,"15min":0.86199999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.98199999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0009999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.745},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.83499999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82899999999999996,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98099999999999998,"5min":1.679,"15min":1.679},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.008,"5min":1.752,"15min":1.752},"last":0.85999999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.331,"15min":1.331},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.86799999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.78000000000000003}]}]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1157453,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66400000000000003}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72299999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.752}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73499999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83299999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.505,"5min":0.434,"15min":0.434},"max":{"1min":0.96199999999999997,"5min":1.01,"15min":1.01},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80700000000000005,"5min":1.115,"15min":1.115},"last":0.80600000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.441},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74299999999999999,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89300000000000002,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.125,"5min":1.534,"15min":1.534},"last":0.91300000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.879,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0840000000000001,"5min":1.649,"15min":1.649},"last":0.93799999999999994}]},{"osd":11,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.85399999999999998,"5min":1.04,"15min":1.04},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.98199999999999998}]}]},{"osd":9,"up_from":64,"seq":274877906988,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":507095,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.42599999999999999,"15min":0.44},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.51700000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.45500000000000002,"15min":0.48099999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.45700000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.35599999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85399999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69299999999999995,"5min":1.012,"15min":1.012},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77300000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.56999999999999995,"15min":0.61399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.54200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80600000000000005,"5min":1.165,"15min":1.165},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.48299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.79400000000000004,"5min":1.216,"15min":1.216},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78200000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.79200000000000004,"15min":0.83599999999999997},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79000000000000004,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.77000000000000002,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66700000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.72499999999999998,"15min":0.76800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":1.337,"15min":1.337},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.81000000000000005,"15min":0.84799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88400000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.61799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.81200000000000006,"15min":0.82999999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96299999999999997,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79500000000000004,"5min":1.024,"15min":1.024},"last":0.63200000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64200000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":503046,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58399999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82299999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.48999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.876,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72599999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.441},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71499999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.496,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.505}]},{"osd":3,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.42499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.88100000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90200000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.50600000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.96999999999999997,"5min":0.999,"15min":0.999},"last":0.78900000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.58499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.0129999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.70799999999999996,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.026,"5min":1.149,"15min":1.149},"last":0.94899999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.77300000000000002,"15min":0.746},"min":{"1min":0.622,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0680000000000001,"5min":1.179,"15min":1.179},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.64200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.071,"15min":1.071},"last":0.878}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.754,"15min":0.72899999999999998},"min":{"1min":0.76200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.69499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91500000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":1},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.93999999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":1100332,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.41199999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54700000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.745},"last":0.42499999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83599999999999997,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.518,"15min":1.518},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81599999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.505,"15min":0.47699999999999998},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.587,"15min":1.587},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":1.496,"15min":1.496},"last":0.85299999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.71399999999999997,"15min":0.68100000000000005},"min":{"1min":0.54300000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.48799999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69199999999999995,"15min":0.66700000000000004},"min":{"1min":0.52100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91700000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47899999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95799999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76000000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.88500000000000001,"15min":0.81299999999999994},"min":{"1min":0.59299999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.707000000000001,"5min":11.707000000000001,"15min":11.707000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.079,"5min":0.82699999999999996,"15min":0.78300000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":6.5899999999999999,"5min":6.5899999999999999,"15min":6.5899999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91700000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78100000000000003}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83399999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561081,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":503130,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.54500000000000004,"15min":0.42299999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.849,"5min":16.849,"15min":16.849},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.38600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.67000000000000004,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":9.093,"5min":9.093,"15min":9.093},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.65400000000000003,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":5.9980000000000002,"5min":5.9980000000000002,"15min":5.9980000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.68500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":9.0050000000000008,"5min":9.0050000000000008,"15min":9.0050000000000008},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.58399999999999996,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0270000000000001,"5min":6.0270000000000001,"15min":6.0270000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.623},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.745,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.1500000000000004,"5min":9.1500000000000004,"15min":9.1500000000000004},"last":0.70099999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.70699999999999996,"15min":0.65700000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":5.8499999999999996,"5min":5.8499999999999996,"15min":5.8499999999999996},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.68999999999999995,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.84699999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":0.81299999999999994,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.1229999999999993,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.74299999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.83899999999999997,"15min":0.79200000000000004},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":9.0589999999999993,"5min":9.0589999999999993,"15min":9.0589999999999993},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.89200000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.83699999999999997,"15min":0.82299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.81299999999999994}]},{"osd":10,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.61299999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.25,"5min":0.96999999999999997,"15min":0.92400000000000004},"min":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":9.1769999999999996,"5min":9.1769999999999996,"15min":9.1769999999999996},"last":0.86099999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":5.9610000000000003,"5min":5.9610000000000003,"15min":5.9610000000000003},"last":0.95899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":567041,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91500000000000004,"5min":0.999,"15min":0.999},"last":0.44}]},{"osd":2,"last update":"Sun Oct 8 09:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.45400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60699999999999998,"15min":0.54200000000000004},"min":{"1min":0.49099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65900000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.50700000000000001},"min":{"1min":0.41299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81499999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61199999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":0.751,"15min":0.751},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.42099999999999999},"min":{"1min":0.41799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.60599999999999998},"min":{"1min":0.58499999999999996,"5min":0.377,"15min":0.377},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.59299999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80000000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64700000000000002,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.59299999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.53700000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.63900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.78000000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.879},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.86199999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.73599999999999999,"15min":0.71399999999999997},"min":{"1min":0.72899999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84199999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.752},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":504586,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1819999999999999,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.63800000000000001,"15min":0.54100000000000004},"min":{"1min":0.45000000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1739999999999999,"5min":3.468,"15min":3.468},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.56399999999999995},"min":{"1min":0.47899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2190000000000001,"5min":3.4409999999999998,"15min":3.4409999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Oct 8 09:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.622,"15min":0.59299999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60799999999999998,"15min":0.57399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0920000000000001,"5min":1.095,"15min":1.095},"last":0.61099999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81499999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53800000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.71499999999999997},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97399999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.621}]},{"osd":10,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.79400000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.4569999999999999,"5min":3.4569999999999999,"15min":3.4569999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.58499999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90700000000000003,"5min":1.008,"15min":1.008},"last":0.59399999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.60199999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1459999999999999,"5min":3.427,"15min":3.427},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.63800000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68000000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":567534,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72799999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52300000000000002,"15min":0.49099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69799999999999995,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.58699999999999997,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0049999999999999,"5min":1.071,"15min":1.071},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94499999999999995,"5min":1.758,"15min":1.758},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.622,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":1.772,"15min":1.772},"last":0.72199999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.48199999999999998,"15min":0.44700000000000001},"min":{"1min":0.314,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.65200000000000002,"15min":0.629},"min":{"1min":0.47499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.026,"5min":1.839,"15min":1.839},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68799999999999994,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":1.492,"15min":1.492},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.89100000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.63500000000000001,"15min":0.59899999999999998},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.113,"5min":1.127,"15min":1.127},"last":0.875},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.58999999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.748}]},{"osd":9,"last update":"Sun Oct 8 09:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.0760000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.95199999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.877,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.96999999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.92100000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":501976,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.379},"min":{"1min":0.314,"5min":0.217,"15min":0.217},"max":{"1min":0.55900000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.53600000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":1,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81499999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73699999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60699999999999998,"15min":0.58899999999999997},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82199999999999995,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.49399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72399999999999998,"5min":1.494,"15min":1.494},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61499999999999999,"15min":0.56100000000000005},"min":{"1min":0.48099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.87,"15min":0.87},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64100000000000001,"5min":1.01,"15min":1.01},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65900000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70899999999999996,"15min":0.72799999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80200000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.55300000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81599999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.022},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.73299999999999998},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64100000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92500000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.81799999999999995,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":1.125,"15min":1.125},"last":0.81899999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73899999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":506673,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.499,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":1.466,"15min":1.466},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.36799999999999999},"min":{"1min":0.30199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.29999999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.52700000000000002},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.89300000000000002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.52500000000000002},"min":{"1min":0.36899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77100000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":4.7850000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.505},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.56000000000000005},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":1.429,"15min":1.429},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63700000000000001,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86299999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78900000000000003,"5min":0.873,"15min":0.873},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.40999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.82299999999999995,"5min":1.353,"15min":1.353},"last":0.47699999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.68200000000000005},"min":{"1min":0.51800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97699999999999998,"5min":1.373,"15min":1.373},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.628}]},{"osd":9,"last update":"Sun Oct 8 09:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93600000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.91300000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":4.8259999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.48699999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.50900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.51600000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70699999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1099161,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.52600000000000002,"15min":0.53600000000000003},"min":{"1min":0.32300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.53300000000000003},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.998,"15min":0.998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.44700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.41599999999999998,"15min":0.40200000000000002},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.625},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.432,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79100000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77200000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70299999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.57099999999999995},"min":{"1min":0.51200000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.47799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.77900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.61199999999999999},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77900000000000003,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.748},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76100000000000001,"5min":1.175,"15min":1.175},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.87},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.52400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.016,"15min":1.016},"last":0.85499999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.83999999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.56399999999999995,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91300000000000003,"5min":1.071,"15min":1.071},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93200000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.93600000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.006},"last":0.88300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:15:02.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:15:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 -- 172.21.15.49:0/3758762120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 msgr2=0x7f61f4105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 --2- 172.21.15.49:0/3758762120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4105e70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009990 tx=0x7f61e4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 -- 172.21.15.49:0/3758762120 shutdown_connections 2023-10-08T09:15:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 --2- 172.21.15.49:0/3758762120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61f4101150 0x7f61f4103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 --2- 172.21.15.49:0/3758762120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61f41063b0 0x7f61f4108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 --2- 172.21.15.49:0/3758762120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4105e70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 -- 172.21.15.49:0/3758762120 >> 172.21.15.49:0/3758762120 conn(0x7f61f40fab40 msgr2=0x7f61f40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 -- 172.21.15.49:0/3758762120 shutdown_connections 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.404+0000 7f61f8d58700 1 -- 172.21.15.49:0/3758762120 wait complete. 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 Processor -- start 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 -- start start 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61f4101150 0x7f61f4196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61f41063b0 0x7f61f419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f419e090 con 0x7f61f4103a80 2023-10-08T09:15:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f419e1d0 con 0x7f61f4101150 2023-10-08T09:15:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f8d58700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f419e360 con 0x7f61f41063b0 2023-10-08T09:15:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49988/0 (socket says 172.21.15.49:49988) 2023-10-08T09:15:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f1d9b700 1 -- 172.21.15.49:0/2584398247 learned_addr learned my addr 172.21.15.49:0/2584398247 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:02.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f259c700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61f4101150 0x7f61f4196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.405+0000 7f61f2d9d700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61f41063b0 0x7f61f419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f1d9b700 1 -- 172.21.15.49:0/2584398247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61f41063b0 msgr2=0x7f61f419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f1d9b700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61f41063b0 0x7f61f419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f1d9b700 1 -- 172.21.15.49:0/2584398247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61f4101150 msgr2=0x7f61f4196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f1d9b700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61f4101150 0x7f61f4196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.410 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f1d9b700 1 -- 172.21.15.49:0/2584398247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e40095f0 con 0x7f61f4103a80 2023-10-08T09:15:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f2d9d700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61f41063b0 0x7f61f419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f1d9b700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4196aa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61e4000c00 tx=0x7f61e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61e37fe700 1 -- 172.21.15.49:0/2584398247 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e401d070 con 0x7f61f4103a80 2023-10-08T09:15:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61e37fe700 1 -- 172.21.15.49:0/2584398247 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61e40054f0 con 0x7f61f4103a80 2023-10-08T09:15:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f419e5e0 con 0x7f61f4103a80 2023-10-08T09:15:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61e37fe700 1 -- 172.21.15.49:0/2584398247 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4004cd0 con 0x7f61f4103a80 2023-10-08T09:15:02.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.406+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f419e9a0 con 0x7f61f4103a80 2023-10-08T09:15:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.407+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f40fc6c0 con 0x7f61f4103a80 2023-10-08T09:15:02.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.408+0000 7f61e37fe700 1 -- 172.21.15.49:0/2584398247 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f61e400fc40 con 0x7f61f4103a80 2023-10-08T09:15:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.409+0000 7f61e37fe700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61dc0a1480 0x7f61dc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.409+0000 7f61e37fe700 1 -- 172.21.15.49:0/2584398247 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f61e40ce310 con 0x7f61f4103a80 2023-10-08T09:15:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.409+0000 7f61f259c700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61dc0a1480 0x7f61dc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.409+0000 7f61f259c700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61dc0a1480 0x7f61dc0a3940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009c60 tx=0x7f61e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:02.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.412+0000 7f61e37fe700 1 -- 172.21.15.49:0/2584398247 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f61e4097770 con 0x7f61f4103a80 2023-10-08T09:15:02.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.614+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61f402cf80 con 0x7f61dc0a1480 2023-10-08T09:15:02.630 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.629+0000 7f61e37fe700 1 -- 172.21.15.49:0/2584398247 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427132 (secure 0 0 0) 0x7f61f402cf80 con 0x7f61dc0a1480 2023-10-08T09:15:02.635 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:02.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.633+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61dc0a1480 msgr2=0x7f61dc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.633+0000 7f61f8d58700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61dc0a1480 0x7f61dc0a3940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009c60 tx=0x7f61e8009260 comp rx=0 tx=0).stop 2023-10-08T09:15:02.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.633+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 msgr2=0x7f61f4196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.633+0000 7f61f8d58700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4196aa0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f61e4000c00 tx=0x7f61e4004730 comp rx=0 tx=0).stop 2023-10-08T09:15:02.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 shutdown_connections 2023-10-08T09:15:02.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61dc0a1480 0x7f61dc0a3940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61f41063b0 0x7f61f419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61f4101150 0x7f61f4196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 --2- 172.21.15.49:0/2584398247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f61f4103a80 0x7f61f4196aa0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 >> 172.21.15.49:0/2584398247 conn(0x7f61f40fab40 msgr2=0x7f61f4101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:02.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 shutdown_connections 2023-10-08T09:15:02.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.634+0000 7f61f8d58700 1 -- 172.21.15.49:0/2584398247 wait complete. 2023-10-08T09:15:02.641 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-10-08T09:15:02.646 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2023-10-08T09:15:01.884548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657474,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1045,"num_read_kb":642,"num_write":430,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180859,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336424,"kb_used_data":12188,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605400,"statfs":{"total":1151940427776,"available":1151595929600,"internally_reserved":0,"allocated":12480512,"data_stored":8025723,"data_compressed":391095,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003618"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429290+0000","last_change":"2023-10-08T09:13:02.831436+0000","last_active":"2023-10-08T09:14:37.429290+0000","last_peered":"2023-10-08T09:14:37.429290+0000","last_clean":"2023-10-08T09:14:37.429290+0000","last_became_active":"2023-10-08T09:13:02.831123+0000","last_became_peered":"2023-10-08T09:13:02.831123+0000","last_unstale":"2023-10-08T09:14:37.429290+0000","last_undegraded":"2023-10-08T09:14:37.429290+0000","last_fullsized":"2023-10-08T09:14:37.429290+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:33.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737779+0000","last_change":"2023-10-08T09:13:02.832286+0000","last_active":"2023-10-08T09:14:37.737779+0000","last_peered":"2023-10-08T09:14:37.737779+0000","last_clean":"2023-10-08T09:14:37.737779+0000","last_became_active":"2023-10-08T09:13:02.831881+0000","last_became_peered":"2023-10-08T09:13:02.831881+0000","last_unstale":"2023-10-08T09:14:37.737779+0000","last_undegraded":"2023-10-08T09:14:37.737779+0000","last_fullsized":"2023-10-08T09:14:37.737779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:31.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737613+0000","last_change":"2023-10-08T09:13:02.831454+0000","last_active":"2023-10-08T09:14:37.737613+0000","last_peered":"2023-10-08T09:14:37.737613+0000","last_clean":"2023-10-08T09:14:37.737613+0000","last_became_active":"2023-10-08T09:13:02.831040+0000","last_became_peered":"2023-10-08T09:13:02.831040+0000","last_unstale":"2023-10-08T09:14:37.737613+0000","last_undegraded":"2023-10-08T09:14:37.737613+0000","last_fullsized":"2023-10-08T09:14:37.737613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:16:02.788279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514426+0000","last_change":"2023-10-08T09:13:02.832699+0000","last_active":"2023-10-08T09:14:37.514426+0000","last_peered":"2023-10-08T09:14:37.514426+0000","last_clean":"2023-10-08T09:14:37.514426+0000","last_became_active":"2023-10-08T09:13:02.832345+0000","last_became_peered":"2023-10-08T09:13:02.832345+0000","last_unstale":"2023-10-08T09:14:37.514426+0000","last_undegraded":"2023-10-08T09:14:37.514426+0000","last_fullsized":"2023-10-08T09:14:37.514426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:01.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534951+0000","last_change":"2023-10-08T09:13:02.827767+0000","last_active":"2023-10-08T09:14:37.534951+0000","last_peered":"2023-10-08T09:14:37.534951+0000","last_clean":"2023-10-08T09:14:37.534951+0000","last_became_active":"2023-10-08T09:13:02.827454+0000","last_became_peered":"2023-10-08T09:13:02.827454+0000","last_unstale":"2023-10-08T09:14:37.534951+0000","last_undegraded":"2023-10-08T09:14:37.534951+0000","last_fullsized":"2023-10-08T09:14:37.534951+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:43.122347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512454+0000","last_change":"2023-10-08T09:13:37.452701+0000","last_active":"2023-10-08T09:14:37.512454+0000","last_peered":"2023-10-08T09:14:37.512454+0000","last_clean":"2023-10-08T09:14:37.512454+0000","last_became_active":"2023-10-08T09:13:37.452261+0000","last_became_peered":"2023-10-08T09:13:37.452261+0000","last_unstale":"2023-10-08T09:14:37.512454+0000","last_undegraded":"2023-10-08T09:14:37.512454+0000","last_fullsized":"2023-10-08T09:14:37.512454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:56.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426872+0000","last_change":"2023-10-08T09:13:02.831765+0000","last_active":"2023-10-08T09:14:37.426872+0000","last_peered":"2023-10-08T09:14:37.426872+0000","last_clean":"2023-10-08T09:14:37.426872+0000","last_became_active":"2023-10-08T09:13:02.831446+0000","last_became_peered":"2023-10-08T09:13:02.831446+0000","last_unstale":"2023-10-08T09:14:37.426872+0000","last_undegraded":"2023-10-08T09:14:37.426872+0000","last_fullsized":"2023-10-08T09:14:37.426872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:41.463161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428811+0000","last_change":"2023-10-08T09:13:02.817586+0000","last_active":"2023-10-08T09:14:37.428811+0000","last_peered":"2023-10-08T09:14:37.428811+0000","last_clean":"2023-10-08T09:14:37.428811+0000","last_became_active":"2023-10-08T09:13:02.817231+0000","last_became_peered":"2023-10-08T09:13:02.817231+0000","last_unstale":"2023-10-08T09:14:37.428811+0000","last_undegraded":"2023-10-08T09:14:37.428811+0000","last_fullsized":"2023-10-08T09:14:37.428811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:23.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.738138+0000","last_change":"2023-10-08T09:13:02.831925+0000","last_active":"2023-10-08T09:14:37.738138+0000","last_peered":"2023-10-08T09:14:37.738138+0000","last_clean":"2023-10-08T09:14:37.738138+0000","last_became_active":"2023-10-08T09:13:02.831625+0000","last_became_peered":"2023-10-08T09:13:02.831625+0000","last_unstale":"2023-10-08T09:14:37.738138+0000","last_undegraded":"2023-10-08T09:14:37.738138+0000","last_fullsized":"2023-10-08T09:14:37.738138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:39.115740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512758+0000","last_change":"2023-10-08T09:13:02.831752+0000","last_active":"2023-10-08T09:14:37.512758+0000","last_peered":"2023-10-08T09:14:37.512758+0000","last_clean":"2023-10-08T09:14:37.512758+0000","last_became_active":"2023-10-08T09:13:02.831468+0000","last_became_peered":"2023-10-08T09:13:02.831468+0000","last_unstale":"2023-10-08T09:14:37.512758+0000","last_undegraded":"2023-10-08T09:14:37.512758+0000","last_fullsized":"2023-10-08T09:14:37.512758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:05.758017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427202+0000","last_change":"2023-10-08T09:13:02.828838+0000","last_active":"2023-10-08T09:14:37.427202+0000","last_peered":"2023-10-08T09:14:37.427202+0000","last_clean":"2023-10-08T09:14:37.427202+0000","last_became_active":"2023-10-08T09:13:02.828598+0000","last_became_peered":"2023-10-08T09:13:02.828598+0000","last_unstale":"2023-10-08T09:14:37.427202+0000","last_undegraded":"2023-10-08T09:14:37.427202+0000","last_fullsized":"2023-10-08T09:14:37.427202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:04.317649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425720+0000","last_change":"2023-10-08T09:13:37.450251+0000","last_active":"2023-10-08T09:14:37.425720+0000","last_peered":"2023-10-08T09:14:37.425720+0000","last_clean":"2023-10-08T09:14:37.425720+0000","last_became_active":"2023-10-08T09:13:37.449848+0000","last_became_peered":"2023-10-08T09:13:37.449848+0000","last_unstale":"2023-10-08T09:14:37.425720+0000","last_undegraded":"2023-10-08T09:14:37.425720+0000","last_fullsized":"2023-10-08T09:14:37.425720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:09:16.541785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511528+0000","last_change":"2023-10-08T09:13:37.450381+0000","last_active":"2023-10-08T09:14:37.511528+0000","last_peered":"2023-10-08T09:14:37.511528+0000","last_clean":"2023-10-08T09:14:37.511528+0000","last_became_active":"2023-10-08T09:13:37.450044+0000","last_became_peered":"2023-10-08T09:13:37.450044+0000","last_unstale":"2023-10-08T09:14:37.511528+0000","last_undegraded":"2023-10-08T09:14:37.511528+0000","last_fullsized":"2023-10-08T09:14:37.511528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:03.203894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737338+0000","last_change":"2023-10-08T09:13:02.829027+0000","last_active":"2023-10-08T09:14:37.737338+0000","last_peered":"2023-10-08T09:14:37.737338+0000","last_clean":"2023-10-08T09:14:37.737338+0000","last_became_active":"2023-10-08T09:13:02.828714+0000","last_became_peered":"2023-10-08T09:13:02.828714+0000","last_unstale":"2023-10-08T09:14:37.737338+0000","last_undegraded":"2023-10-08T09:14:37.737338+0000","last_fullsized":"2023-10-08T09:14:37.737338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:13.121274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737347+0000","last_change":"2023-10-08T09:13:02.838640+0000","last_active":"2023-10-08T09:14:37.737347+0000","last_peered":"2023-10-08T09:14:37.737347+0000","last_clean":"2023-10-08T09:14:37.737347+0000","last_became_active":"2023-10-08T09:13:02.838302+0000","last_became_peered":"2023-10-08T09:13:02.838302+0000","last_unstale":"2023-10-08T09:14:37.737347+0000","last_undegraded":"2023-10-08T09:14:37.737347+0000","last_fullsized":"2023-10-08T09:14:37.737347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:54.268315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737546+0000","last_change":"2023-10-08T09:13:02.834027+0000","last_active":"2023-10-08T09:14:37.737546+0000","last_peered":"2023-10-08T09:14:37.737546+0000","last_clean":"2023-10-08T09:14:37.737546+0000","last_became_active":"2023-10-08T09:13:02.833733+0000","last_became_peered":"2023-10-08T09:13:02.833733+0000","last_unstale":"2023-10-08T09:14:37.737546+0000","last_undegraded":"2023-10-08T09:14:37.737546+0000","last_fullsized":"2023-10-08T09:14:37.737546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:03.644640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429275+0000","last_change":"2023-10-08T09:13:02.832040+0000","last_active":"2023-10-08T09:14:37.429275+0000","last_peered":"2023-10-08T09:14:37.429275+0000","last_clean":"2023-10-08T09:14:37.429275+0000","last_became_active":"2023-10-08T09:13:02.831752+0000","last_became_peered":"2023-10-08T09:13:02.831752+0000","last_unstale":"2023-10-08T09:14:37.429275+0000","last_undegraded":"2023-10-08T09:14:37.429275+0000","last_fullsized":"2023-10-08T09:14:37.429275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:38.938703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426182+0000","last_change":"2023-10-08T09:13:02.830535+0000","last_active":"2023-10-08T09:14:37.426182+0000","last_peered":"2023-10-08T09:14:37.426182+0000","last_clean":"2023-10-08T09:14:37.426182+0000","last_became_active":"2023-10-08T09:13:02.830144+0000","last_became_peered":"2023-10-08T09:13:02.830144+0000","last_unstale":"2023-10-08T09:14:37.426182+0000","last_undegraded":"2023-10-08T09:14:37.426182+0000","last_fullsized":"2023-10-08T09:14:37.426182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:33.850815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426536+0000","last_change":"2023-10-08T09:13:02.830938+0000","last_active":"2023-10-08T09:14:37.426536+0000","last_peered":"2023-10-08T09:14:37.426536+0000","last_clean":"2023-10-08T09:14:37.426536+0000","last_became_active":"2023-10-08T09:13:02.830551+0000","last_became_peered":"2023-10-08T09:13:02.830551+0000","last_unstale":"2023-10-08T09:14:37.426536+0000","last_undegraded":"2023-10-08T09:14:37.426536+0000","last_fullsized":"2023-10-08T09:14:37.426536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:26.894614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425970+0000","last_change":"2023-10-08T09:13:07.871322+0000","last_active":"2023-10-08T09:14:37.425970+0000","last_peered":"2023-10-08T09:14:37.425970+0000","last_clean":"2023-10-08T09:14:37.425970+0000","last_became_active":"2023-10-08T09:13:07.870828+0000","last_became_peered":"2023-10-08T09:13:07.870828+0000","last_unstale":"2023-10-08T09:14:37.425970+0000","last_undegraded":"2023-10-08T09:14:37.425970+0000","last_fullsized":"2023-10-08T09:14:37.425970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:37.145705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.558590+0000","last_change":"2023-10-08T09:13:02.373510+0000","last_active":"2023-10-08T09:14:57.558590+0000","last_peered":"2023-10-08T09:14:57.558590+0000","last_clean":"2023-10-08T09:14:57.558590+0000","last_became_active":"2023-10-08T09:13:02.373174+0000","last_became_peered":"2023-10-08T09:13:02.373174+0000","last_unstale":"2023-10-08T09:14:57.558590+0000","last_undegraded":"2023-10-08T09:14:57.558590+0000","last_fullsized":"2023-10-08T09:14:57.558590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429497+0000","last_change":"2023-10-08T09:13:02.835195+0000","last_active":"2023-10-08T09:14:37.429497+0000","last_peered":"2023-10-08T09:14:37.429497+0000","last_clean":"2023-10-08T09:14:37.429497+0000","last_became_active":"2023-10-08T09:13:02.834893+0000","last_became_peered":"2023-10-08T09:13:02.834893+0000","last_unstale":"2023-10-08T09:14:37.429497+0000","last_undegraded":"2023-10-08T09:14:37.429497+0000","last_fullsized":"2023-10-08T09:14:37.429497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:35.835017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513867+0000","last_change":"2023-10-08T09:13:07.871418+0000","last_active":"2023-10-08T09:14:37.513867+0000","last_peered":"2023-10-08T09:14:37.513867+0000","last_clean":"2023-10-08T09:14:37.513867+0000","last_became_active":"2023-10-08T09:13:07.870961+0000","last_became_peered":"2023-10-08T09:13:07.870961+0000","last_unstale":"2023-10-08T09:14:37.513867+0000","last_undegraded":"2023-10-08T09:14:37.513867+0000","last_fullsized":"2023-10-08T09:14:37.513867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:31.723498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.546688+0000","last_change":"2023-10-08T09:13:02.370728+0000","last_active":"2023-10-08T09:14:38.546688+0000","last_peered":"2023-10-08T09:14:38.546688+0000","last_clean":"2023-10-08T09:14:38.546688+0000","last_became_active":"2023-10-08T09:13:02.370396+0000","last_became_peered":"2023-10-08T09:13:02.370396+0000","last_unstale":"2023-10-08T09:14:38.546688+0000","last_undegraded":"2023-10-08T09:14:38.546688+0000","last_fullsized":"2023-10-08T09:14:38.546688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:32.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737684+0000","last_change":"2023-10-08T09:13:02.827823+0000","last_active":"2023-10-08T09:14:37.737684+0000","last_peered":"2023-10-08T09:14:37.737684+0000","last_clean":"2023-10-08T09:14:37.737684+0000","last_became_active":"2023-10-08T09:13:02.827417+0000","last_became_peered":"2023-10-08T09:13:02.827417+0000","last_unstale":"2023-10-08T09:14:37.737684+0000","last_undegraded":"2023-10-08T09:14:37.737684+0000","last_fullsized":"2023-10-08T09:14:37.737684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:38.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513945+0000","last_change":"2023-10-08T09:13:37.455325+0000","last_active":"2023-10-08T09:14:37.513945+0000","last_peered":"2023-10-08T09:14:37.513945+0000","last_clean":"2023-10-08T09:14:37.513945+0000","last_became_active":"2023-10-08T09:13:37.454985+0000","last_became_peered":"2023-10-08T09:13:37.454985+0000","last_unstale":"2023-10-08T09:14:37.513945+0000","last_undegraded":"2023-10-08T09:14:37.513945+0000","last_fullsized":"2023-10-08T09:14:37.513945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:13.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535311+0000","last_change":"2023-10-08T09:13:02.375244+0000","last_active":"2023-10-08T09:14:37.535311+0000","last_peered":"2023-10-08T09:14:37.535311+0000","last_clean":"2023-10-08T09:14:37.535311+0000","last_became_active":"2023-10-08T09:13:02.374526+0000","last_became_peered":"2023-10-08T09:13:02.374526+0000","last_unstale":"2023-10-08T09:14:37.535311+0000","last_undegraded":"2023-10-08T09:14:37.535311+0000","last_fullsized":"2023-10-08T09:14:37.535311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:36.380491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737863+0000","last_change":"2023-10-08T09:13:02.838373+0000","last_active":"2023-10-08T09:14:37.737863+0000","last_peered":"2023-10-08T09:14:37.737863+0000","last_clean":"2023-10-08T09:14:37.737863+0000","last_became_active":"2023-10-08T09:13:02.838100+0000","last_became_peered":"2023-10-08T09:13:02.838100+0000","last_unstale":"2023-10-08T09:14:37.737863+0000","last_undegraded":"2023-10-08T09:14:37.737863+0000","last_fullsized":"2023-10-08T09:14:37.737863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:59.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513906+0000","last_change":"2023-10-08T09:13:07.868929+0000","last_active":"2023-10-08T09:14:37.513906+0000","last_peered":"2023-10-08T09:14:37.513906+0000","last_clean":"2023-10-08T09:14:37.513906+0000","last_became_active":"2023-10-08T09:13:07.868451+0000","last_became_peered":"2023-10-08T09:13:07.868451+0000","last_unstale":"2023-10-08T09:14:37.513906+0000","last_undegraded":"2023-10-08T09:14:37.513906+0000","last_fullsized":"2023-10-08T09:14:37.513906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:36.540319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535135+0000","last_change":"2023-10-08T09:13:02.366404+0000","last_active":"2023-10-08T09:14:37.535135+0000","last_peered":"2023-10-08T09:14:37.535135+0000","last_clean":"2023-10-08T09:14:37.535135+0000","last_became_active":"2023-10-08T09:13:02.366025+0000","last_became_peered":"2023-10-08T09:13:02.366025+0000","last_unstale":"2023-10-08T09:14:37.535135+0000","last_undegraded":"2023-10-08T09:14:37.535135+0000","last_fullsized":"2023-10-08T09:14:37.535135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:51.916565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515611+0000","last_change":"2023-10-08T09:13:02.832852+0000","last_active":"2023-10-08T09:14:37.515611+0000","last_peered":"2023-10-08T09:14:37.515611+0000","last_clean":"2023-10-08T09:14:37.515611+0000","last_became_active":"2023-10-08T09:13:02.832530+0000","last_became_peered":"2023-10-08T09:13:02.832530+0000","last_unstale":"2023-10-08T09:14:37.515611+0000","last_undegraded":"2023-10-08T09:14:37.515611+0000","last_fullsized":"2023-10-08T09:14:37.515611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:20.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426351+0000","last_change":"2023-10-08T09:13:07.863864+0000","last_active":"2023-10-08T09:14:37.426351+0000","last_peered":"2023-10-08T09:14:37.426351+0000","last_clean":"2023-10-08T09:14:37.426351+0000","last_became_active":"2023-10-08T09:13:07.863509+0000","last_became_peered":"2023-10-08T09:13:07.863509+0000","last_unstale":"2023-10-08T09:14:37.426351+0000","last_undegraded":"2023-10-08T09:14:37.426351+0000","last_fullsized":"2023-10-08T09:14:37.426351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:22.933327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":117,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.548141+0000","last_change":"2023-10-08T09:13:02.374215+0000","last_active":"2023-10-08T09:14:57.548141+0000","last_peered":"2023-10-08T09:14:57.548141+0000","last_clean":"2023-10-08T09:14:57.548141+0000","last_became_active":"2023-10-08T09:13:02.373912+0000","last_became_peered":"2023-10-08T09:13:02.373912+0000","last_unstale":"2023-10-08T09:14:57.548141+0000","last_undegraded":"2023-10-08T09:14:57.548141+0000","last_fullsized":"2023-10-08T09:14:57.548141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:18.315244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535090+0000","last_change":"2023-10-08T09:13:02.830651+0000","last_active":"2023-10-08T09:14:37.535090+0000","last_peered":"2023-10-08T09:14:37.535090+0000","last_clean":"2023-10-08T09:14:37.535090+0000","last_became_active":"2023-10-08T09:13:02.830324+0000","last_became_peered":"2023-10-08T09:13:02.830324+0000","last_unstale":"2023-10-08T09:14:37.535090+0000","last_undegraded":"2023-10-08T09:14:37.535090+0000","last_fullsized":"2023-10-08T09:14:37.535090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:02.143794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426203+0000","last_change":"2023-10-08T09:13:37.454054+0000","last_active":"2023-10-08T09:14:37.426203+0000","last_peered":"2023-10-08T09:14:37.426203+0000","last_clean":"2023-10-08T09:14:37.426203+0000","last_became_active":"2023-10-08T09:13:37.453482+0000","last_became_peered":"2023-10-08T09:13:37.453482+0000","last_unstale":"2023-10-08T09:14:37.426203+0000","last_undegraded":"2023-10-08T09:14:37.426203+0000","last_fullsized":"2023-10-08T09:14:37.426203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:31.550837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512273+0000","last_change":"2023-10-08T09:13:02.370334+0000","last_active":"2023-10-08T09:14:37.512273+0000","last_peered":"2023-10-08T09:14:37.512273+0000","last_clean":"2023-10-08T09:14:37.512273+0000","last_became_active":"2023-10-08T09:13:02.370068+0000","last_became_peered":"2023-10-08T09:13:02.370068+0000","last_unstale":"2023-10-08T09:14:37.512273+0000","last_undegraded":"2023-10-08T09:14:37.512273+0000","last_fullsized":"2023-10-08T09:14:37.512273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.962779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515743+0000","last_change":"2023-10-08T09:13:02.832346+0000","last_active":"2023-10-08T09:14:37.515743+0000","last_peered":"2023-10-08T09:14:37.515743+0000","last_clean":"2023-10-08T09:14:37.515743+0000","last_became_active":"2023-10-08T09:13:02.832059+0000","last_became_peered":"2023-10-08T09:13:02.832059+0000","last_unstale":"2023-10-08T09:14:37.515743+0000","last_undegraded":"2023-10-08T09:14:37.515743+0000","last_fullsized":"2023-10-08T09:14:37.515743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:30.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429192+0000","last_change":"2023-10-08T09:13:07.863466+0000","last_active":"2023-10-08T09:14:37.429192+0000","last_peered":"2023-10-08T09:14:37.429192+0000","last_clean":"2023-10-08T09:14:37.429192+0000","last_became_active":"2023-10-08T09:13:07.863015+0000","last_became_peered":"2023-10-08T09:13:07.863015+0000","last_unstale":"2023-10-08T09:14:37.429192+0000","last_undegraded":"2023-10-08T09:14:37.429192+0000","last_fullsized":"2023-10-08T09:14:37.429192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:30.316625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511291+0000","last_change":"2023-10-08T09:13:02.369444+0000","last_active":"2023-10-08T09:14:37.511291+0000","last_peered":"2023-10-08T09:14:37.511291+0000","last_clean":"2023-10-08T09:14:37.511291+0000","last_became_active":"2023-10-08T09:13:02.369142+0000","last_became_peered":"2023-10-08T09:13:02.369142+0000","last_unstale":"2023-10-08T09:14:37.511291+0000","last_undegraded":"2023-10-08T09:14:37.511291+0000","last_fullsized":"2023-10-08T09:14:37.511291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:28.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428912+0000","last_change":"2023-10-08T09:13:02.836125+0000","last_active":"2023-10-08T09:14:37.428912+0000","last_peered":"2023-10-08T09:14:37.428912+0000","last_clean":"2023-10-08T09:14:37.428912+0000","last_became_active":"2023-10-08T09:13:02.835809+0000","last_became_peered":"2023-10-08T09:13:02.835809+0000","last_unstale":"2023-10-08T09:14:37.428912+0000","last_undegraded":"2023-10-08T09:14:37.428912+0000","last_fullsized":"2023-10-08T09:14:37.428912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:15.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737758+0000","last_change":"2023-10-08T09:13:07.861633+0000","last_active":"2023-10-08T09:14:37.737758+0000","last_peered":"2023-10-08T09:14:37.737758+0000","last_clean":"2023-10-08T09:14:37.737758+0000","last_became_active":"2023-10-08T09:13:07.861168+0000","last_became_peered":"2023-10-08T09:13:07.861168+0000","last_unstale":"2023-10-08T09:14:37.737758+0000","last_undegraded":"2023-10-08T09:14:37.737758+0000","last_fullsized":"2023-10-08T09:14:37.737758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:47.283892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515340+0000","last_change":"2023-10-08T09:13:02.365703+0000","last_active":"2023-10-08T09:14:37.515340+0000","last_peered":"2023-10-08T09:14:37.515340+0000","last_clean":"2023-10-08T09:14:37.515340+0000","last_became_active":"2023-10-08T09:13:02.365421+0000","last_became_peered":"2023-10-08T09:13:02.365421+0000","last_unstale":"2023-10-08T09:14:37.515340+0000","last_undegraded":"2023-10-08T09:14:37.515340+0000","last_fullsized":"2023-10-08T09:14:37.515340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:43.124511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425689+0000","last_change":"2023-10-08T09:13:02.840941+0000","last_active":"2023-10-08T09:14:37.425689+0000","last_peered":"2023-10-08T09:14:37.425689+0000","last_clean":"2023-10-08T09:14:37.425689+0000","last_became_active":"2023-10-08T09:13:02.840605+0000","last_became_peered":"2023-10-08T09:13:02.840605+0000","last_unstale":"2023-10-08T09:14:37.425689+0000","last_undegraded":"2023-10-08T09:14:37.425689+0000","last_fullsized":"2023-10-08T09:14:37.425689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:24:13.417028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429080+0000","last_change":"2023-10-08T09:13:07.868950+0000","last_active":"2023-10-08T09:14:37.429080+0000","last_peered":"2023-10-08T09:14:37.429080+0000","last_clean":"2023-10-08T09:14:37.429080+0000","last_became_active":"2023-10-08T09:13:07.868542+0000","last_became_peered":"2023-10-08T09:13:07.868542+0000","last_unstale":"2023-10-08T09:14:37.429080+0000","last_undegraded":"2023-10-08T09:14:37.429080+0000","last_fullsized":"2023-10-08T09:14:37.429080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:11.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":201,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549263+0000","last_change":"2023-10-08T09:13:02.376188+0000","last_active":"2023-10-08T09:14:58.549263+0000","last_peered":"2023-10-08T09:14:58.549263+0000","last_clean":"2023-10-08T09:14:58.549263+0000","last_became_active":"2023-10-08T09:13:02.375914+0000","last_became_peered":"2023-10-08T09:13:02.375914+0000","last_unstale":"2023-10-08T09:14:58.549263+0000","last_undegraded":"2023-10-08T09:14:58.549263+0000","last_fullsized":"2023-10-08T09:14:58.549263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:10.744085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.738033+0000","last_change":"2023-10-08T09:13:02.833429+0000","last_active":"2023-10-08T09:14:37.738033+0000","last_peered":"2023-10-08T09:14:37.738033+0000","last_clean":"2023-10-08T09:14:37.738033+0000","last_became_active":"2023-10-08T09:13:02.832792+0000","last_became_peered":"2023-10-08T09:13:02.832792+0000","last_unstale":"2023-10-08T09:14:37.738033+0000","last_undegraded":"2023-10-08T09:14:37.738033+0000","last_fullsized":"2023-10-08T09:14:37.738033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:24.563447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425404+0000","last_change":"2023-10-08T09:13:07.862688+0000","last_active":"2023-10-08T09:14:37.425404+0000","last_peered":"2023-10-08T09:14:37.425404+0000","last_clean":"2023-10-08T09:14:37.425404+0000","last_became_active":"2023-10-08T09:13:07.862197+0000","last_became_peered":"2023-10-08T09:13:07.862197+0000","last_unstale":"2023-10-08T09:14:37.425404+0000","last_undegraded":"2023-10-08T09:14:37.425404+0000","last_fullsized":"2023-10-08T09:14:37.425404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:56.792045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427056+0000","last_change":"2023-10-08T09:13:02.380283+0000","last_active":"2023-10-08T09:14:37.427056+0000","last_peered":"2023-10-08T09:14:37.427056+0000","last_clean":"2023-10-08T09:14:37.427056+0000","last_became_active":"2023-10-08T09:13:02.379844+0000","last_became_peered":"2023-10-08T09:13:02.379844+0000","last_unstale":"2023-10-08T09:14:37.427056+0000","last_undegraded":"2023-10-08T09:14:37.427056+0000","last_fullsized":"2023-10-08T09:14:37.427056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:18.162437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737756+0000","last_change":"2023-10-08T09:13:02.839211+0000","last_active":"2023-10-08T09:14:37.737756+0000","last_peered":"2023-10-08T09:14:37.737756+0000","last_clean":"2023-10-08T09:14:37.737756+0000","last_became_active":"2023-10-08T09:13:02.838968+0000","last_became_peered":"2023-10-08T09:13:02.838968+0000","last_unstale":"2023-10-08T09:14:37.737756+0000","last_undegraded":"2023-10-08T09:14:37.737756+0000","last_fullsized":"2023-10-08T09:14:37.737756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:24:02.541749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"86'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514099+0000","last_change":"2023-10-08T09:13:37.457701+0000","last_active":"2023-10-08T09:14:37.514099+0000","last_peered":"2023-10-08T09:14:37.514099+0000","last_clean":"2023-10-08T09:14:37.514099+0000","last_became_active":"2023-10-08T09:13:37.452309+0000","last_became_peered":"2023-10-08T09:13:37.452309+0000","last_unstale":"2023-10-08T09:14:37.514099+0000","last_undegraded":"2023-10-08T09:14:37.514099+0000","last_fullsized":"2023-10-08T09:14:37.514099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:14.898542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515550+0000","last_change":"2023-10-08T09:13:02.377027+0000","last_active":"2023-10-08T09:14:37.515550+0000","last_peered":"2023-10-08T09:14:37.515550+0000","last_clean":"2023-10-08T09:14:37.515550+0000","last_became_active":"2023-10-08T09:13:02.376746+0000","last_became_peered":"2023-10-08T09:13:02.376746+0000","last_unstale":"2023-10-08T09:14:37.515550+0000","last_undegraded":"2023-10-08T09:14:37.515550+0000","last_fullsized":"2023-10-08T09:14:37.515550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:19.062296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737868+0000","last_change":"2023-10-08T09:13:02.837279+0000","last_active":"2023-10-08T09:14:37.737868+0000","last_peered":"2023-10-08T09:14:37.737868+0000","last_clean":"2023-10-08T09:14:37.737868+0000","last_became_active":"2023-10-08T09:13:02.836966+0000","last_became_peered":"2023-10-08T09:13:02.836966+0000","last_unstale":"2023-10-08T09:14:37.737868+0000","last_undegraded":"2023-10-08T09:14:37.737868+0000","last_fullsized":"2023-10-08T09:14:37.737868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:25.462835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425457+0000","last_change":"2023-10-08T09:13:07.865538+0000","last_active":"2023-10-08T09:14:37.425457+0000","last_peered":"2023-10-08T09:14:37.425457+0000","last_clean":"2023-10-08T09:14:37.425457+0000","last_became_active":"2023-10-08T09:13:07.865118+0000","last_became_peered":"2023-10-08T09:13:07.865118+0000","last_unstale":"2023-10-08T09:14:37.425457+0000","last_undegraded":"2023-10-08T09:14:37.425457+0000","last_fullsized":"2023-10-08T09:14:37.425457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:32.167439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":246,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549867+0000","last_change":"2023-10-08T09:13:02.378173+0000","last_active":"2023-10-08T09:14:58.549867+0000","last_peered":"2023-10-08T09:14:58.549867+0000","last_clean":"2023-10-08T09:14:58.549867+0000","last_became_active":"2023-10-08T09:13:02.377880+0000","last_became_peered":"2023-10-08T09:13:02.377880+0000","last_unstale":"2023-10-08T09:14:58.549867+0000","last_undegraded":"2023-10-08T09:14:58.549867+0000","last_fullsized":"2023-10-08T09:14:58.549867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59058,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":151,"num_write":40,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535159+0000","last_change":"2023-10-08T09:13:02.829874+0000","last_active":"2023-10-08T09:14:37.535159+0000","last_peered":"2023-10-08T09:14:37.535159+0000","last_clean":"2023-10-08T09:14:37.535159+0000","last_became_active":"2023-10-08T09:13:02.829552+0000","last_became_peered":"2023-10-08T09:13:02.829552+0000","last_unstale":"2023-10-08T09:14:37.535159+0000","last_undegraded":"2023-10-08T09:14:37.535159+0000","last_fullsized":"2023-10-08T09:14:37.535159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:29.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515922+0000","last_change":"2023-10-08T09:13:07.869661+0000","last_active":"2023-10-08T09:14:37.515922+0000","last_peered":"2023-10-08T09:14:37.515922+0000","last_clean":"2023-10-08T09:14:37.515922+0000","last_became_active":"2023-10-08T09:13:07.869236+0000","last_became_peered":"2023-10-08T09:13:07.869236+0000","last_unstale":"2023-10-08T09:14:37.515922+0000","last_undegraded":"2023-10-08T09:14:37.515922+0000","last_fullsized":"2023-10-08T09:14:37.515922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:26.354461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426116+0000","last_change":"2023-10-08T09:13:02.376941+0000","last_active":"2023-10-08T09:14:37.426116+0000","last_peered":"2023-10-08T09:14:37.426116+0000","last_clean":"2023-10-08T09:14:37.426116+0000","last_became_active":"2023-10-08T09:13:02.375810+0000","last_became_peered":"2023-10-08T09:13:02.375810+0000","last_unstale":"2023-10-08T09:14:37.426116+0000","last_undegraded":"2023-10-08T09:14:37.426116+0000","last_fullsized":"2023-10-08T09:14:37.426116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:17.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511365+0000","last_change":"2023-10-08T09:13:02.836781+0000","last_active":"2023-10-08T09:14:37.511365+0000","last_peered":"2023-10-08T09:14:37.511365+0000","last_clean":"2023-10-08T09:14:37.511365+0000","last_became_active":"2023-10-08T09:13:02.836493+0000","last_became_peered":"2023-10-08T09:13:02.836493+0000","last_unstale":"2023-10-08T09:14:37.511365+0000","last_undegraded":"2023-10-08T09:14:37.511365+0000","last_fullsized":"2023-10-08T09:14:37.511365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:03.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428895+0000","last_change":"2023-10-08T09:13:07.859104+0000","last_active":"2023-10-08T09:14:37.428895+0000","last_peered":"2023-10-08T09:14:37.428895+0000","last_clean":"2023-10-08T09:14:37.428895+0000","last_became_active":"2023-10-08T09:13:07.858691+0000","last_became_peered":"2023-10-08T09:13:07.858691+0000","last_unstale":"2023-10-08T09:14:37.428895+0000","last_undegraded":"2023-10-08T09:14:37.428895+0000","last_fullsized":"2023-10-08T09:14:37.428895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:33.256044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515740+0000","last_change":"2023-10-08T09:13:02.364472+0000","last_active":"2023-10-08T09:14:37.515740+0000","last_peered":"2023-10-08T09:14:37.515740+0000","last_clean":"2023-10-08T09:14:37.515740+0000","last_became_active":"2023-10-08T09:13:02.364130+0000","last_became_peered":"2023-10-08T09:13:02.364130+0000","last_unstale":"2023-10-08T09:14:37.515740+0000","last_undegraded":"2023-10-08T09:14:37.515740+0000","last_fullsized":"2023-10-08T09:14:37.515740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:18.156186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427207+0000","last_change":"2023-10-08T09:13:02.836119+0000","last_active":"2023-10-08T09:14:37.427207+0000","last_peered":"2023-10-08T09:14:37.427207+0000","last_clean":"2023-10-08T09:14:37.427207+0000","last_became_active":"2023-10-08T09:13:02.835548+0000","last_became_peered":"2023-10-08T09:13:02.835548+0000","last_unstale":"2023-10-08T09:14:37.427207+0000","last_undegraded":"2023-10-08T09:14:37.427207+0000","last_fullsized":"2023-10-08T09:14:37.427207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:34.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426091+0000","last_change":"2023-10-08T09:13:07.871447+0000","last_active":"2023-10-08T09:14:37.426091+0000","last_peered":"2023-10-08T09:14:37.426091+0000","last_clean":"2023-10-08T09:14:37.426091+0000","last_became_active":"2023-10-08T09:13:07.871018+0000","last_became_peered":"2023-10-08T09:13:07.871018+0000","last_unstale":"2023-10-08T09:14:37.426091+0000","last_undegraded":"2023-10-08T09:14:37.426091+0000","last_fullsized":"2023-10-08T09:14:37.426091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:34.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.547617+0000","last_change":"2023-10-08T09:13:02.378954+0000","last_active":"2023-10-08T09:14:57.547617+0000","last_peered":"2023-10-08T09:14:57.547617+0000","last_clean":"2023-10-08T09:14:57.547617+0000","last_became_active":"2023-10-08T09:13:02.378635+0000","last_became_peered":"2023-10-08T09:13:02.378635+0000","last_unstale":"2023-10-08T09:14:57.547617+0000","last_undegraded":"2023-10-08T09:14:57.547617+0000","last_fullsized":"2023-10-08T09:14:57.547617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:09.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":906,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426816+0000","last_change":"2023-10-08T09:13:02.828732+0000","last_active":"2023-10-08T09:14:37.426816+0000","last_peered":"2023-10-08T09:14:37.426816+0000","last_clean":"2023-10-08T09:14:37.426816+0000","last_became_active":"2023-10-08T09:13:02.828449+0000","last_became_peered":"2023-10-08T09:13:02.828449+0000","last_unstale":"2023-10-08T09:14:37.426816+0000","last_undegraded":"2023-10-08T09:14:37.426816+0000","last_fullsized":"2023-10-08T09:14:37.426816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:09.883908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511958+0000","last_change":"2023-10-08T09:13:07.869610+0000","last_active":"2023-10-08T09:14:37.511958+0000","last_peered":"2023-10-08T09:14:37.511958+0000","last_clean":"2023-10-08T09:14:37.511958+0000","last_became_active":"2023-10-08T09:13:07.869225+0000","last_became_peered":"2023-10-08T09:13:07.869225+0000","last_unstale":"2023-10-08T09:14:37.511958+0000","last_undegraded":"2023-10-08T09:14:37.511958+0000","last_fullsized":"2023-10-08T09:14:37.511958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:35.225452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534680+0000","last_change":"2023-10-08T09:13:02.375658+0000","last_active":"2023-10-08T09:14:37.534680+0000","last_peered":"2023-10-08T09:14:37.534680+0000","last_clean":"2023-10-08T09:14:37.534680+0000","last_became_active":"2023-10-08T09:13:02.375383+0000","last_became_peered":"2023-10-08T09:13:02.375383+0000","last_unstale":"2023-10-08T09:14:37.534680+0000","last_undegraded":"2023-10-08T09:14:37.534680+0000","last_fullsized":"2023-10-08T09:14:37.534680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:52.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515917+0000","last_change":"2023-10-08T09:13:37.455645+0000","last_active":"2023-10-08T09:14:37.515917+0000","last_peered":"2023-10-08T09:14:37.515917+0000","last_clean":"2023-10-08T09:14:37.515917+0000","last_became_active":"2023-10-08T09:13:37.455207+0000","last_became_peered":"2023-10-08T09:13:37.455207+0000","last_unstale":"2023-10-08T09:14:37.515917+0000","last_undegraded":"2023-10-08T09:14:37.515917+0000","last_fullsized":"2023-10-08T09:14:37.515917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:18.021608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515352+0000","last_change":"2023-10-08T09:13:07.871441+0000","last_active":"2023-10-08T09:14:37.515352+0000","last_peered":"2023-10-08T09:14:37.515352+0000","last_clean":"2023-10-08T09:14:37.515352+0000","last_became_active":"2023-10-08T09:13:07.870968+0000","last_became_peered":"2023-10-08T09:13:07.870968+0000","last_unstale":"2023-10-08T09:14:37.515352+0000","last_undegraded":"2023-10-08T09:14:37.515352+0000","last_fullsized":"2023-10-08T09:14:37.515352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:48.746147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535222+0000","last_change":"2023-10-08T09:13:02.372514+0000","last_active":"2023-10-08T09:14:37.535222+0000","last_peered":"2023-10-08T09:14:37.535222+0000","last_clean":"2023-10-08T09:14:37.535222+0000","last_became_active":"2023-10-08T09:13:02.372181+0000","last_became_peered":"2023-10-08T09:13:02.372181+0000","last_unstale":"2023-10-08T09:14:37.535222+0000","last_undegraded":"2023-10-08T09:14:37.535222+0000","last_fullsized":"2023-10-08T09:14:37.535222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:28.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429168+0000","last_change":"2023-10-08T09:13:02.835220+0000","last_active":"2023-10-08T09:14:37.429168+0000","last_peered":"2023-10-08T09:14:37.429168+0000","last_clean":"2023-10-08T09:14:37.429168+0000","last_became_active":"2023-10-08T09:13:02.834914+0000","last_became_peered":"2023-10-08T09:13:02.834914+0000","last_unstale":"2023-10-08T09:14:37.429168+0000","last_undegraded":"2023-10-08T09:14:37.429168+0000","last_fullsized":"2023-10-08T09:14:37.429168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:37.447418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535013+0000","last_change":"2023-10-08T09:13:08.096146+0000","last_active":"2023-10-08T09:14:37.535013+0000","last_peered":"2023-10-08T09:14:37.535013+0000","last_clean":"2023-10-08T09:14:37.535013+0000","last_became_active":"2023-10-08T09:13:08.095703+0000","last_became_peered":"2023-10-08T09:13:08.095703+0000","last_unstale":"2023-10-08T09:14:37.535013+0000","last_undegraded":"2023-10-08T09:14:37.535013+0000","last_fullsized":"2023-10-08T09:14:37.535013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:37.511094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425782+0000","last_change":"2023-10-08T09:13:02.372836+0000","last_active":"2023-10-08T09:14:37.425782+0000","last_peered":"2023-10-08T09:14:37.425782+0000","last_clean":"2023-10-08T09:14:37.425782+0000","last_became_active":"2023-10-08T09:13:02.372488+0000","last_became_peered":"2023-10-08T09:13:02.372488+0000","last_unstale":"2023-10-08T09:14:37.425782+0000","last_undegraded":"2023-10-08T09:14:37.425782+0000","last_fullsized":"2023-10-08T09:14:37.425782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:19.269539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512564+0000","last_change":"2023-10-08T09:13:02.827306+0000","last_active":"2023-10-08T09:14:37.512564+0000","last_peered":"2023-10-08T09:14:37.512564+0000","last_clean":"2023-10-08T09:14:37.512564+0000","last_became_active":"2023-10-08T09:13:02.826965+0000","last_became_peered":"2023-10-08T09:13:02.826965+0000","last_unstale":"2023-10-08T09:14:37.512564+0000","last_undegraded":"2023-10-08T09:14:37.512564+0000","last_fullsized":"2023-10-08T09:14:37.512564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:27.513860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511590+0000","last_change":"2023-10-08T09:13:07.862637+0000","last_active":"2023-10-08T09:14:37.511590+0000","last_peered":"2023-10-08T09:14:37.511590+0000","last_clean":"2023-10-08T09:14:37.511590+0000","last_became_active":"2023-10-08T09:13:07.862117+0000","last_became_peered":"2023-10-08T09:13:07.862117+0000","last_unstale":"2023-10-08T09:14:37.511590+0000","last_undegraded":"2023-10-08T09:14:37.511590+0000","last_fullsized":"2023-10-08T09:14:37.511590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:14.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:59.561747+0000","last_change":"2023-10-08T09:13:02.379585+0000","last_active":"2023-10-08T09:14:59.561747+0000","last_peered":"2023-10-08T09:14:59.561747+0000","last_clean":"2023-10-08T09:14:59.561747+0000","last_became_active":"2023-10-08T09:13:02.379324+0000","last_became_peered":"2023-10-08T09:13:02.379324+0000","last_unstale":"2023-10-08T09:14:59.561747+0000","last_undegraded":"2023-10-08T09:14:59.561747+0000","last_fullsized":"2023-10-08T09:14:59.561747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:31.609558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512613+0000","last_change":"2023-10-08T09:13:02.833416+0000","last_active":"2023-10-08T09:14:37.512613+0000","last_peered":"2023-10-08T09:14:37.512613+0000","last_clean":"2023-10-08T09:14:37.512613+0000","last_became_active":"2023-10-08T09:13:02.833040+0000","last_became_peered":"2023-10-08T09:13:02.833040+0000","last_unstale":"2023-10-08T09:14:37.512613+0000","last_undegraded":"2023-10-08T09:14:37.512613+0000","last_fullsized":"2023-10-08T09:14:37.512613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:44.480445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511906+0000","last_change":"2023-10-08T09:13:08.095723+0000","last_active":"2023-10-08T09:14:37.511906+0000","last_peered":"2023-10-08T09:14:37.511906+0000","last_clean":"2023-10-08T09:14:37.511906+0000","last_became_active":"2023-10-08T09:13:08.095271+0000","last_became_peered":"2023-10-08T09:13:08.095271+0000","last_unstale":"2023-10-08T09:14:37.511906+0000","last_undegraded":"2023-10-08T09:14:37.511906+0000","last_fullsized":"2023-10-08T09:14:37.511906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:04.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515464+0000","last_change":"2023-10-08T09:13:02.829663+0000","last_active":"2023-10-08T09:14:37.515464+0000","last_peered":"2023-10-08T09:14:37.515464+0000","last_clean":"2023-10-08T09:14:37.515464+0000","last_became_active":"2023-10-08T09:13:02.829354+0000","last_became_peered":"2023-10-08T09:13:02.829354+0000","last_unstale":"2023-10-08T09:14:37.515464+0000","last_undegraded":"2023-10-08T09:14:37.515464+0000","last_fullsized":"2023-10-08T09:14:37.515464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:55.332346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549710+0000","last_change":"2023-10-08T09:13:02.372114+0000","last_active":"2023-10-08T09:14:58.549710+0000","last_peered":"2023-10-08T09:14:58.549710+0000","last_clean":"2023-10-08T09:14:58.549710+0000","last_became_active":"2023-10-08T09:13:02.371603+0000","last_became_peered":"2023-10-08T09:13:02.371603+0000","last_unstale":"2023-10-08T09:14:58.549710+0000","last_undegraded":"2023-10-08T09:14:58.549710+0000","last_fullsized":"2023-10-08T09:14:58.549710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:39.078307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427099+0000","last_change":"2023-10-08T09:13:07.863492+0000","last_active":"2023-10-08T09:14:37.427099+0000","last_peered":"2023-10-08T09:14:37.427099+0000","last_clean":"2023-10-08T09:14:37.427099+0000","last_became_active":"2023-10-08T09:13:07.862803+0000","last_became_peered":"2023-10-08T09:13:07.862803+0000","last_unstale":"2023-10-08T09:14:37.427099+0000","last_undegraded":"2023-10-08T09:14:37.427099+0000","last_fullsized":"2023-10-08T09:14:37.427099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:39.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428936+0000","last_change":"2023-10-08T09:13:02.836596+0000","last_active":"2023-10-08T09:14:37.428936+0000","last_peered":"2023-10-08T09:14:37.428936+0000","last_clean":"2023-10-08T09:14:37.428936+0000","last_became_active":"2023-10-08T09:13:02.836283+0000","last_became_peered":"2023-10-08T09:13:02.836283+0000","last_unstale":"2023-10-08T09:14:37.428936+0000","last_undegraded":"2023-10-08T09:14:37.428936+0000","last_fullsized":"2023-10-08T09:14:37.428936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:02.393575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514133+0000","last_change":"2023-10-08T09:13:02.379516+0000","last_active":"2023-10-08T09:14:37.514133+0000","last_peered":"2023-10-08T09:14:37.514133+0000","last_clean":"2023-10-08T09:14:37.514133+0000","last_became_active":"2023-10-08T09:13:02.379229+0000","last_became_peered":"2023-10-08T09:13:02.379229+0000","last_unstale":"2023-10-08T09:14:37.514133+0000","last_undegraded":"2023-10-08T09:14:37.514133+0000","last_fullsized":"2023-10-08T09:14:37.514133+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:55.294651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.533330+0000","last_change":"2023-10-08T09:12:02.620769+0000","last_active":"2023-10-08T09:14:38.533330+0000","last_peered":"2023-10-08T09:14:38.533330+0000","last_clean":"2023-10-08T09:14:38.533330+0000","last_became_active":"2023-10-08T09:12:02.620388+0000","last_became_peered":"2023-10-08T09:12:02.620388+0000","last_unstale":"2023-10-08T09:14:38.533330+0000","last_undegraded":"2023-10-08T09:14:38.533330+0000","last_fullsized":"2023-10-08T09:14:38.533330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_clean_scrub_stamp":"2023-10-08T09:09:10.746288+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:55:35.713060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515785+0000","last_change":"2023-10-08T09:13:08.096318+0000","last_active":"2023-10-08T09:14:37.515785+0000","last_peered":"2023-10-08T09:14:37.515785+0000","last_clean":"2023-10-08T09:14:37.515785+0000","last_became_active":"2023-10-08T09:13:08.095774+0000","last_became_peered":"2023-10-08T09:13:08.095774+0000","last_unstale":"2023-10-08T09:14:37.515785+0000","last_undegraded":"2023-10-08T09:14:37.515785+0000","last_fullsized":"2023-10-08T09:14:37.515785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:53.089468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511157+0000","last_change":"2023-10-08T09:13:02.835167+0000","last_active":"2023-10-08T09:14:37.511157+0000","last_peered":"2023-10-08T09:14:37.511157+0000","last_clean":"2023-10-08T09:14:37.511157+0000","last_became_active":"2023-10-08T09:13:02.834850+0000","last_became_peered":"2023-10-08T09:13:02.834850+0000","last_unstale":"2023-10-08T09:14:37.511157+0000","last_undegraded":"2023-10-08T09:14:37.511157+0000","last_fullsized":"2023-10-08T09:14:37.511157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:19.422372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426957+0000","last_change":"2023-10-08T09:13:02.377746+0000","last_active":"2023-10-08T09:14:37.426957+0000","last_peered":"2023-10-08T09:14:37.426957+0000","last_clean":"2023-10-08T09:14:37.426957+0000","last_became_active":"2023-10-08T09:13:02.377422+0000","last_became_peered":"2023-10-08T09:13:02.377422+0000","last_unstale":"2023-10-08T09:14:37.426957+0000","last_undegraded":"2023-10-08T09:14:37.426957+0000","last_fullsized":"2023-10-08T09:14:37.426957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:13.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428886+0000","last_change":"2023-10-08T09:13:07.864678+0000","last_active":"2023-10-08T09:14:37.428886+0000","last_peered":"2023-10-08T09:14:37.428886+0000","last_clean":"2023-10-08T09:14:37.428886+0000","last_became_active":"2023-10-08T09:13:07.864226+0000","last_became_peered":"2023-10-08T09:13:07.864226+0000","last_unstale":"2023-10-08T09:14:37.428886+0000","last_undegraded":"2023-10-08T09:14:37.428886+0000","last_fullsized":"2023-10-08T09:14:37.428886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:43:51.179853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535266+0000","last_change":"2023-10-08T09:13:02.370014+0000","last_active":"2023-10-08T09:14:37.535266+0000","last_peered":"2023-10-08T09:14:37.535266+0000","last_clean":"2023-10-08T09:14:37.535266+0000","last_became_active":"2023-10-08T09:13:02.369704+0000","last_became_peered":"2023-10-08T09:13:02.369704+0000","last_unstale":"2023-10-08T09:14:37.535266+0000","last_undegraded":"2023-10-08T09:14:37.535266+0000","last_fullsized":"2023-10-08T09:14:37.535266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:28.902274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511727+0000","last_change":"2023-10-08T09:13:02.833859+0000","last_active":"2023-10-08T09:14:37.511727+0000","last_peered":"2023-10-08T09:14:37.511727+0000","last_clean":"2023-10-08T09:14:37.511727+0000","last_became_active":"2023-10-08T09:13:02.833546+0000","last_became_peered":"2023-10-08T09:13:02.833546+0000","last_unstale":"2023-10-08T09:14:37.511727+0000","last_undegraded":"2023-10-08T09:14:37.511727+0000","last_fullsized":"2023-10-08T09:14:37.511727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:35.534588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428821+0000","last_change":"2023-10-08T09:13:08.095622+0000","last_active":"2023-10-08T09:14:37.428821+0000","last_peered":"2023-10-08T09:14:37.428821+0000","last_clean":"2023-10-08T09:14:37.428821+0000","last_became_active":"2023-10-08T09:13:08.095143+0000","last_became_peered":"2023-10-08T09:13:08.095143+0000","last_unstale":"2023-10-08T09:14:37.428821+0000","last_undegraded":"2023-10-08T09:14:37.428821+0000","last_fullsized":"2023-10-08T09:14:37.428821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:46.738102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426121+0000","last_change":"2023-10-08T09:13:37.459185+0000","last_active":"2023-10-08T09:14:37.426121+0000","last_peered":"2023-10-08T09:14:37.426121+0000","last_clean":"2023-10-08T09:14:37.426121+0000","last_became_active":"2023-10-08T09:13:37.451105+0000","last_became_peered":"2023-10-08T09:13:37.451105+0000","last_unstale":"2023-10-08T09:14:37.426121+0000","last_undegraded":"2023-10-08T09:14:37.426121+0000","last_fullsized":"2023-10-08T09:14:37.426121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.768269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429624+0000","last_change":"2023-10-08T09:13:02.834053+0000","last_active":"2023-10-08T09:14:37.429624+0000","last_peered":"2023-10-08T09:14:37.429624+0000","last_clean":"2023-10-08T09:14:37.429624+0000","last_became_active":"2023-10-08T09:13:02.833707+0000","last_became_peered":"2023-10-08T09:13:02.833707+0000","last_unstale":"2023-10-08T09:14:37.429624+0000","last_undegraded":"2023-10-08T09:14:37.429624+0000","last_fullsized":"2023-10-08T09:14:37.429624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:31.372003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425862+0000","last_change":"2023-10-08T09:13:07.855734+0000","last_active":"2023-10-08T09:14:37.425862+0000","last_peered":"2023-10-08T09:14:37.425862+0000","last_clean":"2023-10-08T09:14:37.425862+0000","last_became_active":"2023-10-08T09:13:07.855313+0000","last_became_peered":"2023-10-08T09:13:07.855313+0000","last_unstale":"2023-10-08T09:14:37.425862+0000","last_undegraded":"2023-10-08T09:14:37.425862+0000","last_fullsized":"2023-10-08T09:14:37.425862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:15.207337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514427+0000","last_change":"2023-10-08T09:13:02.371319+0000","last_active":"2023-10-08T09:14:37.514427+0000","last_peered":"2023-10-08T09:14:37.514427+0000","last_clean":"2023-10-08T09:14:37.514427+0000","last_became_active":"2023-10-08T09:13:02.371049+0000","last_became_peered":"2023-10-08T09:13:02.371049+0000","last_unstale":"2023-10-08T09:14:37.514427+0000","last_undegraded":"2023-10-08T09:14:37.514427+0000","last_fullsized":"2023-10-08T09:14:37.514427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:28.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737818+0000","last_change":"2023-10-08T09:13:02.838682+0000","last_active":"2023-10-08T09:14:37.737818+0000","last_peered":"2023-10-08T09:14:37.737818+0000","last_clean":"2023-10-08T09:14:37.737818+0000","last_became_active":"2023-10-08T09:13:02.838389+0000","last_became_peered":"2023-10-08T09:13:02.838389+0000","last_unstale":"2023-10-08T09:14:37.737818+0000","last_undegraded":"2023-10-08T09:14:37.737818+0000","last_fullsized":"2023-10-08T09:14:37.737818+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:52.937723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514169+0000","last_change":"2023-10-08T09:13:08.098131+0000","last_active":"2023-10-08T09:14:37.514169+0000","last_peered":"2023-10-08T09:14:37.514169+0000","last_clean":"2023-10-08T09:14:37.514169+0000","last_became_active":"2023-10-08T09:13:08.097352+0000","last_became_peered":"2023-10-08T09:13:08.097352+0000","last_unstale":"2023-10-08T09:14:37.514169+0000","last_undegraded":"2023-10-08T09:14:37.514169+0000","last_fullsized":"2023-10-08T09:14:37.514169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:30.388875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737576+0000","last_change":"2023-10-08T09:13:37.461067+0000","last_active":"2023-10-08T09:14:37.737576+0000","last_peered":"2023-10-08T09:14:37.737576+0000","last_clean":"2023-10-08T09:14:37.737576+0000","last_became_active":"2023-10-08T09:13:37.451739+0000","last_became_peered":"2023-10-08T09:13:37.451739+0000","last_unstale":"2023-10-08T09:14:37.737576+0000","last_undegraded":"2023-10-08T09:14:37.737576+0000","last_fullsized":"2023-10-08T09:14:37.737576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:26.603798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534742+0000","last_change":"2023-10-08T09:13:02.830017+0000","last_active":"2023-10-08T09:14:37.534742+0000","last_peered":"2023-10-08T09:14:37.534742+0000","last_clean":"2023-10-08T09:14:37.534742+0000","last_became_active":"2023-10-08T09:13:02.829700+0000","last_became_peered":"2023-10-08T09:13:02.829700+0000","last_unstale":"2023-10-08T09:14:37.534742+0000","last_undegraded":"2023-10-08T09:14:37.534742+0000","last_fullsized":"2023-10-08T09:14:37.534742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:31:31.387049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426196+0000","last_change":"2023-10-08T09:13:07.869097+0000","last_active":"2023-10-08T09:14:37.426196+0000","last_peered":"2023-10-08T09:14:37.426196+0000","last_clean":"2023-10-08T09:14:37.426196+0000","last_became_active":"2023-10-08T09:13:07.868710+0000","last_became_peered":"2023-10-08T09:13:07.868710+0000","last_unstale":"2023-10-08T09:14:37.426196+0000","last_undegraded":"2023-10-08T09:14:37.426196+0000","last_fullsized":"2023-10-08T09:14:37.426196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:37:27.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429207+0000","last_change":"2023-10-08T09:13:02.370694+0000","last_active":"2023-10-08T09:14:37.429207+0000","last_peered":"2023-10-08T09:14:37.429207+0000","last_clean":"2023-10-08T09:14:37.429207+0000","last_became_active":"2023-10-08T09:13:02.370161+0000","last_became_peered":"2023-10-08T09:13:02.370161+0000","last_unstale":"2023-10-08T09:14:37.429207+0000","last_undegraded":"2023-10-08T09:14:37.429207+0000","last_fullsized":"2023-10-08T09:14:37.429207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:04.809703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428791+0000","last_change":"2023-10-08T09:13:02.831026+0000","last_active":"2023-10-08T09:14:37.428791+0000","last_peered":"2023-10-08T09:14:37.428791+0000","last_clean":"2023-10-08T09:14:37.428791+0000","last_became_active":"2023-10-08T09:13:02.830715+0000","last_became_peered":"2023-10-08T09:13:02.830715+0000","last_unstale":"2023-10-08T09:14:37.428791+0000","last_undegraded":"2023-10-08T09:14:37.428791+0000","last_fullsized":"2023-10-08T09:14:37.428791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:47.709820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514173+0000","last_change":"2023-10-08T09:13:07.860464+0000","last_active":"2023-10-08T09:14:37.514173+0000","last_peered":"2023-10-08T09:14:37.514173+0000","last_clean":"2023-10-08T09:14:37.514173+0000","last_became_active":"2023-10-08T09:13:07.859962+0000","last_became_peered":"2023-10-08T09:13:07.859962+0000","last_unstale":"2023-10-08T09:14:37.514173+0000","last_undegraded":"2023-10-08T09:14:37.514173+0000","last_fullsized":"2023-10-08T09:14:37.514173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:04.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426052+0000","last_change":"2023-10-08T09:13:02.371308+0000","last_active":"2023-10-08T09:14:37.426052+0000","last_peered":"2023-10-08T09:14:37.426052+0000","last_clean":"2023-10-08T09:14:37.426052+0000","last_became_active":"2023-10-08T09:13:02.370968+0000","last_became_peered":"2023-10-08T09:13:02.370968+0000","last_unstale":"2023-10-08T09:14:37.426052+0000","last_undegraded":"2023-10-08T09:14:37.426052+0000","last_fullsized":"2023-10-08T09:14:37.426052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:58.491112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429125+0000","last_change":"2023-10-08T09:13:02.826511+0000","last_active":"2023-10-08T09:14:37.429125+0000","last_peered":"2023-10-08T09:14:37.429125+0000","last_clean":"2023-10-08T09:14:37.429125+0000","last_became_active":"2023-10-08T09:13:02.826205+0000","last_became_peered":"2023-10-08T09:13:02.826205+0000","last_unstale":"2023-10-08T09:14:37.429125+0000","last_undegraded":"2023-10-08T09:14:37.429125+0000","last_fullsized":"2023-10-08T09:14:37.429125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:12.270176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514117+0000","last_change":"2023-10-08T09:13:08.096360+0000","last_active":"2023-10-08T09:14:37.514117+0000","last_peered":"2023-10-08T09:14:37.514117+0000","last_clean":"2023-10-08T09:14:37.514117+0000","last_became_active":"2023-10-08T09:13:08.095915+0000","last_became_peered":"2023-10-08T09:13:08.095915+0000","last_unstale":"2023-10-08T09:14:37.514117+0000","last_undegraded":"2023-10-08T09:14:37.514117+0000","last_fullsized":"2023-10-08T09:14:37.514117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:59.412471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534927+0000","last_change":"2023-10-08T09:13:02.375287+0000","last_active":"2023-10-08T09:14:37.534927+0000","last_peered":"2023-10-08T09:14:37.534927+0000","last_clean":"2023-10-08T09:14:37.534927+0000","last_became_active":"2023-10-08T09:13:02.374668+0000","last_became_peered":"2023-10-08T09:13:02.374668+0000","last_unstale":"2023-10-08T09:14:37.534927+0000","last_undegraded":"2023-10-08T09:14:37.534927+0000","last_fullsized":"2023-10-08T09:14:37.534927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:22.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426986+0000","last_change":"2023-10-08T09:13:02.828240+0000","last_active":"2023-10-08T09:14:37.426986+0000","last_peered":"2023-10-08T09:14:37.426986+0000","last_clean":"2023-10-08T09:14:37.426986+0000","last_became_active":"2023-10-08T09:13:02.827951+0000","last_became_peered":"2023-10-08T09:13:02.827951+0000","last_unstale":"2023-10-08T09:14:37.426986+0000","last_undegraded":"2023-10-08T09:14:37.426986+0000","last_fullsized":"2023-10-08T09:14:37.426986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:37.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737603+0000","last_change":"2023-10-08T09:13:07.865846+0000","last_active":"2023-10-08T09:14:37.737603+0000","last_peered":"2023-10-08T09:14:37.737603+0000","last_clean":"2023-10-08T09:14:37.737603+0000","last_became_active":"2023-10-08T09:13:07.865529+0000","last_became_peered":"2023-10-08T09:13:07.865529+0000","last_unstale":"2023-10-08T09:14:37.737603+0000","last_undegraded":"2023-10-08T09:14:37.737603+0000","last_fullsized":"2023-10-08T09:14:37.737603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:34:03.408036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:59.571238+0000","last_change":"2023-10-08T09:13:02.380128+0000","last_active":"2023-10-08T09:14:59.571238+0000","last_peered":"2023-10-08T09:14:59.571238+0000","last_clean":"2023-10-08T09:14:59.571238+0000","last_became_active":"2023-10-08T09:13:02.379800+0000","last_became_peered":"2023-10-08T09:13:02.379800+0000","last_unstale":"2023-10-08T09:14:59.571238+0000","last_undegraded":"2023-10-08T09:14:59.571238+0000","last_fullsized":"2023-10-08T09:14:59.571238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:01.546289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427243+0000","last_change":"2023-10-08T09:13:02.829474+0000","last_active":"2023-10-08T09:14:37.427243+0000","last_peered":"2023-10-08T09:14:37.427243+0000","last_clean":"2023-10-08T09:14:37.427243+0000","last_became_active":"2023-10-08T09:13:02.829173+0000","last_became_peered":"2023-10-08T09:13:02.829173+0000","last_unstale":"2023-10-08T09:14:37.427243+0000","last_undegraded":"2023-10-08T09:14:37.427243+0000","last_fullsized":"2023-10-08T09:14:37.427243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:02.469158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737465+0000","last_change":"2023-10-08T09:13:37.453778+0000","last_active":"2023-10-08T09:14:37.737465+0000","last_peered":"2023-10-08T09:14:37.737465+0000","last_clean":"2023-10-08T09:14:37.737465+0000","last_became_active":"2023-10-08T09:13:37.453354+0000","last_became_peered":"2023-10-08T09:13:37.453354+0000","last_unstale":"2023-10-08T09:14:37.737465+0000","last_undegraded":"2023-10-08T09:14:37.737465+0000","last_fullsized":"2023-10-08T09:14:37.737465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:50.035588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515804+0000","last_change":"2023-10-08T09:13:02.376641+0000","last_active":"2023-10-08T09:14:37.515804+0000","last_peered":"2023-10-08T09:14:37.515804+0000","last_clean":"2023-10-08T09:14:37.515804+0000","last_became_active":"2023-10-08T09:13:02.376348+0000","last_became_peered":"2023-10-08T09:13:02.376348+0000","last_unstale":"2023-10-08T09:14:37.515804+0000","last_undegraded":"2023-10-08T09:14:37.515804+0000","last_fullsized":"2023-10-08T09:14:37.515804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:42.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427333+0000","last_change":"2023-10-08T09:13:02.836201+0000","last_active":"2023-10-08T09:14:37.427333+0000","last_peered":"2023-10-08T09:14:37.427333+0000","last_clean":"2023-10-08T09:14:37.427333+0000","last_became_active":"2023-10-08T09:13:02.835885+0000","last_became_peered":"2023-10-08T09:13:02.835885+0000","last_unstale":"2023-10-08T09:14:37.427333+0000","last_undegraded":"2023-10-08T09:14:37.427333+0000","last_fullsized":"2023-10-08T09:14:37.427333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:34.310854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515138+0000","last_change":"2023-10-08T09:13:07.863854+0000","last_active":"2023-10-08T09:14:37.515138+0000","last_peered":"2023-10-08T09:14:37.515138+0000","last_clean":"2023-10-08T09:14:37.515138+0000","last_became_active":"2023-10-08T09:13:07.863460+0000","last_became_peered":"2023-10-08T09:13:07.863460+0000","last_unstale":"2023-10-08T09:14:37.515138+0000","last_undegraded":"2023-10-08T09:14:37.515138+0000","last_fullsized":"2023-10-08T09:14:37.515138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:03.093858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511956+0000","last_change":"2023-10-08T09:13:02.372336+0000","last_active":"2023-10-08T09:14:37.511956+0000","last_peered":"2023-10-08T09:14:37.511956+0000","last_clean":"2023-10-08T09:14:37.511956+0000","last_became_active":"2023-10-08T09:13:02.372022+0000","last_became_peered":"2023-10-08T09:13:02.372022+0000","last_unstale":"2023-10-08T09:14:37.511956+0000","last_undegraded":"2023-10-08T09:14:37.511956+0000","last_fullsized":"2023-10-08T09:14:37.511956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:10.610332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427293+0000","last_change":"2023-10-08T09:13:02.832430+0000","last_active":"2023-10-08T09:14:37.427293+0000","last_peered":"2023-10-08T09:14:37.427293+0000","last_clean":"2023-10-08T09:14:37.427293+0000","last_became_active":"2023-10-08T09:13:02.832092+0000","last_became_peered":"2023-10-08T09:13:02.832092+0000","last_unstale":"2023-10-08T09:14:37.427293+0000","last_undegraded":"2023-10-08T09:14:37.427293+0000","last_fullsized":"2023-10-08T09:14:37.427293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:00.798900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428785+0000","last_change":"2023-10-08T09:13:02.833114+0000","last_active":"2023-10-08T09:14:37.428785+0000","last_peered":"2023-10-08T09:14:37.428785+0000","last_clean":"2023-10-08T09:14:37.428785+0000","last_became_active":"2023-10-08T09:13:02.830959+0000","last_became_peered":"2023-10-08T09:13:02.830959+0000","last_unstale":"2023-10-08T09:14:37.428785+0000","last_undegraded":"2023-10-08T09:14:37.428785+0000","last_fullsized":"2023-10-08T09:14:37.428785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:13.853585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426056+0000","last_change":"2023-10-08T09:13:02.836046+0000","last_active":"2023-10-08T09:14:37.426056+0000","last_peered":"2023-10-08T09:14:37.426056+0000","last_clean":"2023-10-08T09:14:37.426056+0000","last_became_active":"2023-10-08T09:13:02.835722+0000","last_became_peered":"2023-10-08T09:13:02.835722+0000","last_unstale":"2023-10-08T09:14:37.426056+0000","last_undegraded":"2023-10-08T09:14:37.426056+0000","last_fullsized":"2023-10-08T09:14:37.426056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:00.316259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425931+0000","last_change":"2023-10-08T09:13:02.830021+0000","last_active":"2023-10-08T09:14:37.425931+0000","last_peered":"2023-10-08T09:14:37.425931+0000","last_clean":"2023-10-08T09:14:37.425931+0000","last_became_active":"2023-10-08T09:13:02.829397+0000","last_became_peered":"2023-10-08T09:13:02.829397+0000","last_unstale":"2023-10-08T09:14:37.425931+0000","last_undegraded":"2023-10-08T09:14:37.425931+0000","last_fullsized":"2023-10-08T09:14:37.425931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:29.153457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737569+0000","last_change":"2023-10-08T09:13:02.832002+0000","last_active":"2023-10-08T09:14:37.737569+0000","last_peered":"2023-10-08T09:14:37.737569+0000","last_clean":"2023-10-08T09:14:37.737569+0000","last_became_active":"2023-10-08T09:13:02.831666+0000","last_became_peered":"2023-10-08T09:13:02.831666+0000","last_unstale":"2023-10-08T09:14:37.737569+0000","last_undegraded":"2023-10-08T09:14:37.737569+0000","last_fullsized":"2023-10-08T09:14:37.737569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:16.414127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511793+0000","last_change":"2023-10-08T09:13:02.835881+0000","last_active":"2023-10-08T09:14:37.511793+0000","last_peered":"2023-10-08T09:14:37.511793+0000","last_clean":"2023-10-08T09:14:37.511793+0000","last_became_active":"2023-10-08T09:13:02.835587+0000","last_became_peered":"2023-10-08T09:13:02.835587+0000","last_unstale":"2023-10-08T09:14:37.511793+0000","last_undegraded":"2023-10-08T09:14:37.511793+0000","last_fullsized":"2023-10-08T09:14:37.511793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:52.323234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429154+0000","last_change":"2023-10-08T09:13:02.826371+0000","last_active":"2023-10-08T09:14:37.429154+0000","last_peered":"2023-10-08T09:14:37.429154+0000","last_clean":"2023-10-08T09:14:37.429154+0000","last_became_active":"2023-10-08T09:13:02.826093+0000","last_became_peered":"2023-10-08T09:13:02.826093+0000","last_unstale":"2023-10-08T09:14:37.429154+0000","last_undegraded":"2023-10-08T09:14:37.429154+0000","last_fullsized":"2023-10-08T09:14:37.429154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:08.113739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426543+0000","last_change":"2023-10-08T09:13:02.829936+0000","last_active":"2023-10-08T09:14:37.426543+0000","last_peered":"2023-10-08T09:14:37.426543+0000","last_clean":"2023-10-08T09:14:37.426543+0000","last_became_active":"2023-10-08T09:13:02.829498+0000","last_became_peered":"2023-10-08T09:13:02.829498+0000","last_unstale":"2023-10-08T09:14:37.426543+0000","last_undegraded":"2023-10-08T09:14:37.426543+0000","last_fullsized":"2023-10-08T09:14:37.426543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:42.200791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429728+0000","last_change":"2023-10-08T09:13:02.840415+0000","last_active":"2023-10-08T09:14:37.429728+0000","last_peered":"2023-10-08T09:14:37.429728+0000","last_clean":"2023-10-08T09:14:37.429728+0000","last_became_active":"2023-10-08T09:13:02.840075+0000","last_became_peered":"2023-10-08T09:13:02.840075+0000","last_unstale":"2023-10-08T09:14:37.429728+0000","last_undegraded":"2023-10-08T09:14:37.429728+0000","last_fullsized":"2023-10-08T09:14:37.429728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:48.959372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515335+0000","last_change":"2023-10-08T09:13:02.828337+0000","last_active":"2023-10-08T09:14:37.515335+0000","last_peered":"2023-10-08T09:14:37.515335+0000","last_clean":"2023-10-08T09:14:37.515335+0000","last_became_active":"2023-10-08T09:13:02.827998+0000","last_became_peered":"2023-10-08T09:13:02.827998+0000","last_unstale":"2023-10-08T09:14:37.515335+0000","last_undegraded":"2023-10-08T09:14:37.515335+0000","last_fullsized":"2023-10-08T09:14:37.515335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:27.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.547574+0000","last_change":"2023-10-08T09:14:38.547245+0000","last_active":"2023-10-08T09:14:38.547574+0000","last_peered":"2023-10-08T09:14:38.547574+0000","last_clean":"2023-10-08T09:14:38.547574+0000","last_became_active":"2023-10-08T09:14:38.546964+0000","last_became_peered":"2023-10-08T09:14:38.546964+0000","last_unstale":"2023-10-08T09:14:38.547574+0000","last_undegraded":"2023-10-08T09:14:38.547574+0000","last_fullsized":"2023-10-08T09:14:38.547574+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:40.120946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737724+0000","last_change":"2023-10-08T09:13:02.840921+0000","last_active":"2023-10-08T09:14:37.737724+0000","last_peered":"2023-10-08T09:14:37.737724+0000","last_clean":"2023-10-08T09:14:37.737724+0000","last_became_active":"2023-10-08T09:13:02.840566+0000","last_became_peered":"2023-10-08T09:13:02.840566+0000","last_unstale":"2023-10-08T09:14:37.737724+0000","last_undegraded":"2023-10-08T09:14:37.737724+0000","last_fullsized":"2023-10-08T09:14:37.737724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:20.219581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511631+0000","last_change":"2023-10-08T09:13:02.834491+0000","last_active":"2023-10-08T09:14:37.511631+0000","last_peered":"2023-10-08T09:14:37.511631+0000","last_clean":"2023-10-08T09:14:37.511631+0000","last_became_active":"2023-10-08T09:13:02.834058+0000","last_became_peered":"2023-10-08T09:13:02.834058+0000","last_unstale":"2023-10-08T09:14:37.511631+0000","last_undegraded":"2023-10-08T09:14:37.511631+0000","last_fullsized":"2023-10-08T09:14:37.511631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:47.699423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428850+0000","last_change":"2023-10-08T09:13:02.836433+0000","last_active":"2023-10-08T09:14:37.428850+0000","last_peered":"2023-10-08T09:14:37.428850+0000","last_clean":"2023-10-08T09:14:37.428850+0000","last_became_active":"2023-10-08T09:13:02.835147+0000","last_became_peered":"2023-10-08T09:13:02.835147+0000","last_unstale":"2023-10-08T09:14:37.428850+0000","last_undegraded":"2023-10-08T09:14:37.428850+0000","last_fullsized":"2023-10-08T09:14:37.428850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:51.233662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247538,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66860,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":919,"num_read_kb":533,"num_write":247,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514532,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":507696,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69999999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59699999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90100000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.81799999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86699999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.77400000000000002,"5min":1.147,"15min":1.147},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.62,"15min":0.63},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83799999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.78700000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0389999999999999,"5min":2.222,"15min":2.222},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.77200000000000002,"15min":0.79100000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.85099999999999998,"15min":0.86199999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.98199999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0009999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.745},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.83499999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82899999999999996,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98099999999999998,"5min":1.679,"15min":1.679},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.008,"5min":1.752,"15min":1.752},"last":0.85999999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.331,"15min":1.331},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.86799999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.78000000000000003}]}]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1157453,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66400000000000003}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72299999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.752}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73499999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83299999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.505,"5min":0.434,"15min":0.434},"max":{"1min":0.96199999999999997,"5min":1.01,"15min":1.01},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80700000000000005,"5min":1.115,"15min":1.115},"last":0.80600000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.441},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74299999999999999,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89300000000000002,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.125,"5min":1.534,"15min":1.534},"last":0.91300000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.879,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0840000000000001,"5min":1.649,"15min":1.649},"last":0.93799999999999994}]},{"osd":11,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.85399999999999998,"5min":1.04,"15min":1.04},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.98199999999999998}]}]},{"osd":9,"up_from":64,"seq":274877906988,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":507095,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.42599999999999999,"15min":0.44},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.51700000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.45500000000000002,"15min":0.48099999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.45700000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.35599999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85399999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69299999999999995,"5min":1.012,"15min":1.012},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77300000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.56999999999999995,"15min":0.61399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.54200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80600000000000005,"5min":1.165,"15min":1.165},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.48299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.79400000000000004,"5min":1.216,"15min":1.216},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78200000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.79200000000000004,"15min":0.83599999999999997},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79000000000000004,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.77000000000000002,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66700000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.72499999999999998,"15min":0.76800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":1.337,"15min":1.337},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.81000000000000005,"15min":0.84799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88400000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.61799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.81200000000000006,"15min":0.82999999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96299999999999997,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79500000000000004,"5min":1.024,"15min":1.024},"last":0.63200000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64200000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":503046,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58399999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82299999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.48999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.876,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72599999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.441},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71499999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.496,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.505}]},{"osd":3,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.42499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.88100000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90200000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.50600000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.96999999999999997,"5min":0.999,"15min":0.999},"last":0.78900000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.58499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.0129999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.70799999999999996,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.026,"5min":1.149,"15min":1.149},"last":0.94899999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.77300000000000002,"15min":0.746},"min":{"1min":0.622,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0680000000000001,"5min":1.179,"15min":1.179},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.64200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.071,"15min":1.071},"last":0.878}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.754,"15min":0.72899999999999998},"min":{"1min":0.76200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.69499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91500000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":1},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.93999999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":1100332,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.41199999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54700000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.745},"last":0.42499999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83599999999999997,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.518,"15min":1.518},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81599999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.505,"15min":0.47699999999999998},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.587,"15min":1.587},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":1.496,"15min":1.496},"last":0.85299999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.71399999999999997,"15min":0.68100000000000005},"min":{"1min":0.54300000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.48799999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69199999999999995,"15min":0.66700000000000004},"min":{"1min":0.52100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91700000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47899999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95799999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76000000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.88500000000000001,"15min":0.81299999999999994},"min":{"1min":0.59299999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.707000000000001,"5min":11.707000000000001,"15min":11.707000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.079,"5min":0.82699999999999996,"15min":0.78300000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":6.5899999999999999,"5min":6.5899999999999999,"15min":6.5899999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91700000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78100000000000003}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83399999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561081,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":503130,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.54500000000000004,"15min":0.42299999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.849,"5min":16.849,"15min":16.849},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.38600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.67000000000000004,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":9.093,"5min":9.093,"15min":9.093},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.65400000000000003,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":5.9980000000000002,"5min":5.9980000000000002,"15min":5.9980000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.68500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":9.0050000000000008,"5min":9.0050000000000008,"15min":9.0050000000000008},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.58399999999999996,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0270000000000001,"5min":6.0270000000000001,"15min":6.0270000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.623},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.745,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.1500000000000004,"5min":9.1500000000000004,"15min":9.1500000000000004},"last":0.70099999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.70699999999999996,"15min":0.65700000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":5.8499999999999996,"5min":5.8499999999999996,"15min":5.8499999999999996},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.68999999999999995,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.84699999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":0.81299999999999994,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.1229999999999993,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.74299999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.83899999999999997,"15min":0.79200000000000004},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":9.0589999999999993,"5min":9.0589999999999993,"15min":9.0589999999999993},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.89200000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.83699999999999997,"15min":0.82299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.81299999999999994}]},{"osd":10,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.61299999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.25,"5min":0.96999999999999997,"15min":0.92400000000000004},"min":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":9.1769999999999996,"5min":9.1769999999999996,"15min":9.1769999999999996},"last":0.86099999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":5.9610000000000003,"5min":5.9610000000000003,"15min":5.9610000000000003},"last":0.95899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":567041,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91500000000000004,"5min":0.999,"15min":0.999},"last":0.44}]},{"osd":2,"last update":"Sun Oct 8 09:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.45400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60699999999999998,"15min":0.54200000000000004},"min":{"1min":0.49099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65900000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.50700000000000001},"min":{"1min":0.41299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81499999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61199999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":0.751,"15min":0.751},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.42099999999999999},"min":{"1min":0.41799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.60599999999999998},"min":{"1min":0.58499999999999996,"5min":0.377,"15min":0.377},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.59299999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80000000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64700000000000002,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.59299999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.53700000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.63900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.78000000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.879},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.86199999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.73599999999999999,"15min":0.71399999999999997},"min":{"1min":0.72899999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84199999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.752},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":504586,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1819999999999999,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.63800000000000001,"15min":0.54100000000000004},"min":{"1min":0.45000000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1739999999999999,"5min":3.468,"15min":3.468},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.56399999999999995},"min":{"1min":0.47899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2190000000000001,"5min":3.4409999999999998,"15min":3.4409999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Oct 8 09:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.622,"15min":0.59299999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60799999999999998,"15min":0.57399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0920000000000001,"5min":1.095,"15min":1.095},"last":0.61099999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81499999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53800000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.71499999999999997},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97399999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.621}]},{"osd":10,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.79400000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.4569999999999999,"5min":3.4569999999999999,"15min":3.4569999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.58499999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90700000000000003,"5min":1.008,"15min":1.008},"last":0.59399999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.60199999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1459999999999999,"5min":3.427,"15min":3.427},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.63800000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68000000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":567534,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72799999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52300000000000002,"15min":0.49099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69799999999999995,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.58699999999999997,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0049999999999999,"5min":1.071,"15min":1.071},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94499999999999995,"5min":1.758,"15min":1.758},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.622,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":1.772,"15min":1.772},"last":0.72199999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.48199999999999998,"15min":0.44700000000000001},"min":{"1min":0.314,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.65200000000000002,"15min":0.629},"min":{"1min":0.47499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.026,"5min":1.839,"15min":1.839},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68799999999999994,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":1.492,"15min":1.492},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.89100000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.63500000000000001,"15min":0.59899999999999998},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.113,"5min":1.127,"15min":1.127},"last":0.875},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.58999999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.748}]},{"osd":9,"last update":"Sun Oct 8 09:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.0760000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.95199999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.877,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.96999999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.92100000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":501976,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.379},"min":{"1min":0.314,"5min":0.217,"15min":0.217},"max":{"1min":0.55900000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.53600000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":1,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81499999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73699999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60699999999999998,"15min":0.58899999999999997},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82199999999999995,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.49399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72399999999999998,"5min":1.494,"15min":1.494},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61499999999999999,"15min":0.56100000000000005},"min":{"1min":0.48099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.87,"15min":0.87},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64100000000000001,"5min":1.01,"15min":1.01},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65900000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70899999999999996,"15min":0.72799999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80200000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.55300000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81599999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.022},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.73299999999999998},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64100000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92500000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.81799999999999995,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":1.125,"15min":1.125},"last":0.81899999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73899999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":506673,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.499,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":1.466,"15min":1.466},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.36799999999999999},"min":{"1min":0.30199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.29999999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.52700000000000002},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.89300000000000002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.52500000000000002},"min":{"1min":0.36899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77100000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":4.7850000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.505},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.56000000000000005},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":1.429,"15min":1.429},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63700000000000001,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86299999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78900000000000003,"5min":0.873,"15min":0.873},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.40999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.82299999999999995,"5min":1.353,"15min":1.353},"last":0.47699999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.68200000000000005},"min":{"1min":0.51800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97699999999999998,"5min":1.373,"15min":1.373},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.628}]},{"osd":9,"last update":"Sun Oct 8 09:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93600000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.91300000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":4.8259999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.48699999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.50900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.51600000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70699999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1099161,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.52600000000000002,"15min":0.53600000000000003},"min":{"1min":0.32300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.53300000000000003},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.998,"15min":0.998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.44700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.41599999999999998,"15min":0.40200000000000002},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.625},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.432,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79100000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77200000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70299999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.57099999999999995},"min":{"1min":0.51200000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.47799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.77900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.61199999999999999},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77900000000000003,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.748},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76100000000000001,"5min":1.175,"15min":1.175},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.87},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.52400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.016,"15min":1.016},"last":0.85499999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.83999999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.56399999999999995,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91300000000000003,"5min":1.071,"15min":1.071},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93200000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.93600000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.006},"last":0.88300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:15:02.663 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-08T09:15:02.663 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T09:15:02.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:02 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/641410598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:02.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:02 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/641410598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 -- 172.21.15.49:0/2298204374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec40ff5a0 msgr2=0x7fcec40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 --2- 172.21.15.49:0/2298204374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec40ff5a0 0x7fcec40ff9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcec00099c0 tx=0x7fcec0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 -- 172.21.15.49:0/2298204374 shutdown_connections 2023-10-08T09:15:02.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 --2- 172.21.15.49:0/2298204374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcec4101130 0x7fcec41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 --2- 172.21.15.49:0/2298204374 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec40ff5a0 0x7fcec40ff9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 --2- 172.21.15.49:0/2298204374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec4100790 0x7fcec4100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 -- 172.21.15.49:0/2298204374 >> 172.21.15.49:0/2298204374 conn(0x7fcec40fab10 msgr2=0x7fcec40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 -- 172.21.15.49:0/2298204374 shutdown_connections 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 -- 172.21.15.49:0/2298204374 wait complete. 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 Processor -- start 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.814+0000 7fcecbc31700 1 -- start start 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcec40ff5a0 0x7fcec41954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec4100790 0x7fcec4195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec4101130 0x7fcec419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419cff0 con 0x7fcec4100790 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419d130 con 0x7fcec40ff5a0 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419d2c0 con 0x7fcec4101130 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec4100790 0x7fcec4195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec4100790 0x7fcec4195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50002/0 (socket says 172.21.15.49:50002) 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 -- 172.21.15.49:0/899883232 learned_addr learned my addr 172.21.15.49:0/899883232 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fceca1ce700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec4101130 0x7fcec419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec99cd700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcec40ff5a0 0x7fcec41954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 -- 172.21.15.49:0/899883232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec4101130 msgr2=0x7fcec419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec4101130 0x7fcec419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 -- 172.21.15.49:0/899883232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcec40ff5a0 msgr2=0x7fcec41954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:02.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcec40ff5a0 0x7fcec41954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 -- 172.21.15.49:0/899883232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00095f0 con 0x7fcec4100790 2023-10-08T09:15:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcec91cc700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec4100790 0x7fcec4195a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d950 tx=0x7fceb400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcebaffd700 1 -- 172.21.15.49:0/899883232 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4009940 con 0x7fcec4100790 2023-10-08T09:15:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcebaffd700 1 -- 172.21.15.49:0/899883232 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceb4010460 con 0x7fcec4100790 2023-10-08T09:15:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec419d5a0 con 0x7fcec4100790 2023-10-08T09:15:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcebaffd700 1 -- 172.21.15.49:0/899883232 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400f620 con 0x7fcec4100790 2023-10-08T09:15:02.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.815+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec419db00 con 0x7fcec4100790 2023-10-08T09:15:02.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.816+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec4106970 con 0x7fcec4100790 2023-10-08T09:15:02.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.819+0000 7fcebaffd700 1 -- 172.21.15.49:0/899883232 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fceb40108d0 con 0x7fcec4100790 2023-10-08T09:15:02.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.819+0000 7fcebaffd700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fceb00a1500 0x7fceb00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:02.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.819+0000 7fcebaffd700 1 -- 172.21.15.49:0/899883232 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fceb40cce00 con 0x7fcec4100790 2023-10-08T09:15:02.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.819+0000 7fcec99cd700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fceb00a1500 0x7fceb00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:02.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.820+0000 7fcec99cd700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fceb00a1500 0x7fceb00a39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcec000b580 tx=0x7fcec0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:02.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:02.822+0000 7fcebaffd700 1 -- 172.21.15.49:0/899883232 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fceb4096130 con 0x7fcec4100790 2023-10-08T09:15:02.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:02 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/641410598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.023+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 --> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcec402cf80 con 0x7fceb00a1500 2023-10-08T09:15:03.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.039+0000 7fcebaffd700 1 -- 172.21.15.49:0/899883232 <== mgr.14152 v2:172.21.15.49:6800/1314466902 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427132 (secure 0 0 0) 0x7fcec402cf80 con 0x7fceb00a1500 2023-10-08T09:15:03.044 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fceb00a1500 msgr2=0x7fceb00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fceb00a1500 0x7fceb00a39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcec000b580 tx=0x7fcec0005fb0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec4100790 msgr2=0x7fcec4195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec4100790 0x7fcec4195a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fceb400d950 tx=0x7fceb400dc60 comp rx=0 tx=0).stop 2023-10-08T09:15:03.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 shutdown_connections 2023-10-08T09:15:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fceb00a1500 0x7fceb00a39c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec4101130 0x7fcec419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcec40ff5a0 0x7fcec41954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 --2- 172.21.15.49:0/899883232 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcec4100790 0x7fcec4195a00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 >> 172.21.15.49:0/899883232 conn(0x7fcec40fab10 msgr2=0x7fcec4103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:03.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 shutdown_connections 2023-10-08T09:15:03.050 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.043+0000 7fcecbc31700 1 -- 172.21.15.49:0/899883232 wait complete. 2023-10-08T09:15:03.050 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2023-10-08T09:15:03.056 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2023-10-08T09:15:01.884548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657474,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1045,"num_read_kb":642,"num_write":430,"num_write_kb":3083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180859,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":357,"ondisk_log_size":357,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336424,"kb_used_data":12188,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124605400,"statfs":{"total":1151940427776,"available":1151595929600,"internally_reserved":0,"allocated":12480512,"data_stored":8025723,"data_compressed":391095,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":636,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":57,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003618"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429290+0000","last_change":"2023-10-08T09:13:02.831436+0000","last_active":"2023-10-08T09:14:37.429290+0000","last_peered":"2023-10-08T09:14:37.429290+0000","last_clean":"2023-10-08T09:14:37.429290+0000","last_became_active":"2023-10-08T09:13:02.831123+0000","last_became_peered":"2023-10-08T09:13:02.831123+0000","last_unstale":"2023-10-08T09:14:37.429290+0000","last_undegraded":"2023-10-08T09:14:37.429290+0000","last_fullsized":"2023-10-08T09:14:37.429290+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:33.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737779+0000","last_change":"2023-10-08T09:13:02.832286+0000","last_active":"2023-10-08T09:14:37.737779+0000","last_peered":"2023-10-08T09:14:37.737779+0000","last_clean":"2023-10-08T09:14:37.737779+0000","last_became_active":"2023-10-08T09:13:02.831881+0000","last_became_peered":"2023-10-08T09:13:02.831881+0000","last_unstale":"2023-10-08T09:14:37.737779+0000","last_undegraded":"2023-10-08T09:14:37.737779+0000","last_fullsized":"2023-10-08T09:14:37.737779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:31.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737613+0000","last_change":"2023-10-08T09:13:02.831454+0000","last_active":"2023-10-08T09:14:37.737613+0000","last_peered":"2023-10-08T09:14:37.737613+0000","last_clean":"2023-10-08T09:14:37.737613+0000","last_became_active":"2023-10-08T09:13:02.831040+0000","last_became_peered":"2023-10-08T09:13:02.831040+0000","last_unstale":"2023-10-08T09:14:37.737613+0000","last_undegraded":"2023-10-08T09:14:37.737613+0000","last_fullsized":"2023-10-08T09:14:37.737613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:16:02.788279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514426+0000","last_change":"2023-10-08T09:13:02.832699+0000","last_active":"2023-10-08T09:14:37.514426+0000","last_peered":"2023-10-08T09:14:37.514426+0000","last_clean":"2023-10-08T09:14:37.514426+0000","last_became_active":"2023-10-08T09:13:02.832345+0000","last_became_peered":"2023-10-08T09:13:02.832345+0000","last_unstale":"2023-10-08T09:14:37.514426+0000","last_undegraded":"2023-10-08T09:14:37.514426+0000","last_fullsized":"2023-10-08T09:14:37.514426+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:01.590555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534951+0000","last_change":"2023-10-08T09:13:02.827767+0000","last_active":"2023-10-08T09:14:37.534951+0000","last_peered":"2023-10-08T09:14:37.534951+0000","last_clean":"2023-10-08T09:14:37.534951+0000","last_became_active":"2023-10-08T09:13:02.827454+0000","last_became_peered":"2023-10-08T09:13:02.827454+0000","last_unstale":"2023-10-08T09:14:37.534951+0000","last_undegraded":"2023-10-08T09:14:37.534951+0000","last_fullsized":"2023-10-08T09:14:37.534951+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:43.122347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512454+0000","last_change":"2023-10-08T09:13:37.452701+0000","last_active":"2023-10-08T09:14:37.512454+0000","last_peered":"2023-10-08T09:14:37.512454+0000","last_clean":"2023-10-08T09:14:37.512454+0000","last_became_active":"2023-10-08T09:13:37.452261+0000","last_became_peered":"2023-10-08T09:13:37.452261+0000","last_unstale":"2023-10-08T09:14:37.512454+0000","last_undegraded":"2023-10-08T09:14:37.512454+0000","last_fullsized":"2023-10-08T09:14:37.512454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:56.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426872+0000","last_change":"2023-10-08T09:13:02.831765+0000","last_active":"2023-10-08T09:14:37.426872+0000","last_peered":"2023-10-08T09:14:37.426872+0000","last_clean":"2023-10-08T09:14:37.426872+0000","last_became_active":"2023-10-08T09:13:02.831446+0000","last_became_peered":"2023-10-08T09:13:02.831446+0000","last_unstale":"2023-10-08T09:14:37.426872+0000","last_undegraded":"2023-10-08T09:14:37.426872+0000","last_fullsized":"2023-10-08T09:14:37.426872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:41.463161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428811+0000","last_change":"2023-10-08T09:13:02.817586+0000","last_active":"2023-10-08T09:14:37.428811+0000","last_peered":"2023-10-08T09:14:37.428811+0000","last_clean":"2023-10-08T09:14:37.428811+0000","last_became_active":"2023-10-08T09:13:02.817231+0000","last_became_peered":"2023-10-08T09:13:02.817231+0000","last_unstale":"2023-10-08T09:14:37.428811+0000","last_undegraded":"2023-10-08T09:14:37.428811+0000","last_fullsized":"2023-10-08T09:14:37.428811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:23.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.738138+0000","last_change":"2023-10-08T09:13:02.831925+0000","last_active":"2023-10-08T09:14:37.738138+0000","last_peered":"2023-10-08T09:14:37.738138+0000","last_clean":"2023-10-08T09:14:37.738138+0000","last_became_active":"2023-10-08T09:13:02.831625+0000","last_became_peered":"2023-10-08T09:13:02.831625+0000","last_unstale":"2023-10-08T09:14:37.738138+0000","last_undegraded":"2023-10-08T09:14:37.738138+0000","last_fullsized":"2023-10-08T09:14:37.738138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:39.115740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512758+0000","last_change":"2023-10-08T09:13:02.831752+0000","last_active":"2023-10-08T09:14:37.512758+0000","last_peered":"2023-10-08T09:14:37.512758+0000","last_clean":"2023-10-08T09:14:37.512758+0000","last_became_active":"2023-10-08T09:13:02.831468+0000","last_became_peered":"2023-10-08T09:13:02.831468+0000","last_unstale":"2023-10-08T09:14:37.512758+0000","last_undegraded":"2023-10-08T09:14:37.512758+0000","last_fullsized":"2023-10-08T09:14:37.512758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:05.758017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427202+0000","last_change":"2023-10-08T09:13:02.828838+0000","last_active":"2023-10-08T09:14:37.427202+0000","last_peered":"2023-10-08T09:14:37.427202+0000","last_clean":"2023-10-08T09:14:37.427202+0000","last_became_active":"2023-10-08T09:13:02.828598+0000","last_became_peered":"2023-10-08T09:13:02.828598+0000","last_unstale":"2023-10-08T09:14:37.427202+0000","last_undegraded":"2023-10-08T09:14:37.427202+0000","last_fullsized":"2023-10-08T09:14:37.427202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:04.317649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425720+0000","last_change":"2023-10-08T09:13:37.450251+0000","last_active":"2023-10-08T09:14:37.425720+0000","last_peered":"2023-10-08T09:14:37.425720+0000","last_clean":"2023-10-08T09:14:37.425720+0000","last_became_active":"2023-10-08T09:13:37.449848+0000","last_became_peered":"2023-10-08T09:13:37.449848+0000","last_unstale":"2023-10-08T09:14:37.425720+0000","last_undegraded":"2023-10-08T09:14:37.425720+0000","last_fullsized":"2023-10-08T09:14:37.425720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:09:16.541785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511528+0000","last_change":"2023-10-08T09:13:37.450381+0000","last_active":"2023-10-08T09:14:37.511528+0000","last_peered":"2023-10-08T09:14:37.511528+0000","last_clean":"2023-10-08T09:14:37.511528+0000","last_became_active":"2023-10-08T09:13:37.450044+0000","last_became_peered":"2023-10-08T09:13:37.450044+0000","last_unstale":"2023-10-08T09:14:37.511528+0000","last_undegraded":"2023-10-08T09:14:37.511528+0000","last_fullsized":"2023-10-08T09:14:37.511528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:03.203894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737338+0000","last_change":"2023-10-08T09:13:02.829027+0000","last_active":"2023-10-08T09:14:37.737338+0000","last_peered":"2023-10-08T09:14:37.737338+0000","last_clean":"2023-10-08T09:14:37.737338+0000","last_became_active":"2023-10-08T09:13:02.828714+0000","last_became_peered":"2023-10-08T09:13:02.828714+0000","last_unstale":"2023-10-08T09:14:37.737338+0000","last_undegraded":"2023-10-08T09:14:37.737338+0000","last_fullsized":"2023-10-08T09:14:37.737338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:13.121274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737347+0000","last_change":"2023-10-08T09:13:02.838640+0000","last_active":"2023-10-08T09:14:37.737347+0000","last_peered":"2023-10-08T09:14:37.737347+0000","last_clean":"2023-10-08T09:14:37.737347+0000","last_became_active":"2023-10-08T09:13:02.838302+0000","last_became_peered":"2023-10-08T09:13:02.838302+0000","last_unstale":"2023-10-08T09:14:37.737347+0000","last_undegraded":"2023-10-08T09:14:37.737347+0000","last_fullsized":"2023-10-08T09:14:37.737347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:54.268315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737546+0000","last_change":"2023-10-08T09:13:02.834027+0000","last_active":"2023-10-08T09:14:37.737546+0000","last_peered":"2023-10-08T09:14:37.737546+0000","last_clean":"2023-10-08T09:14:37.737546+0000","last_became_active":"2023-10-08T09:13:02.833733+0000","last_became_peered":"2023-10-08T09:13:02.833733+0000","last_unstale":"2023-10-08T09:14:37.737546+0000","last_undegraded":"2023-10-08T09:14:37.737546+0000","last_fullsized":"2023-10-08T09:14:37.737546+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:03.644640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429275+0000","last_change":"2023-10-08T09:13:02.832040+0000","last_active":"2023-10-08T09:14:37.429275+0000","last_peered":"2023-10-08T09:14:37.429275+0000","last_clean":"2023-10-08T09:14:37.429275+0000","last_became_active":"2023-10-08T09:13:02.831752+0000","last_became_peered":"2023-10-08T09:13:02.831752+0000","last_unstale":"2023-10-08T09:14:37.429275+0000","last_undegraded":"2023-10-08T09:14:37.429275+0000","last_fullsized":"2023-10-08T09:14:37.429275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:38.938703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426182+0000","last_change":"2023-10-08T09:13:02.830535+0000","last_active":"2023-10-08T09:14:37.426182+0000","last_peered":"2023-10-08T09:14:37.426182+0000","last_clean":"2023-10-08T09:14:37.426182+0000","last_became_active":"2023-10-08T09:13:02.830144+0000","last_became_peered":"2023-10-08T09:13:02.830144+0000","last_unstale":"2023-10-08T09:14:37.426182+0000","last_undegraded":"2023-10-08T09:14:37.426182+0000","last_fullsized":"2023-10-08T09:14:37.426182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:33.850815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426536+0000","last_change":"2023-10-08T09:13:02.830938+0000","last_active":"2023-10-08T09:14:37.426536+0000","last_peered":"2023-10-08T09:14:37.426536+0000","last_clean":"2023-10-08T09:14:37.426536+0000","last_became_active":"2023-10-08T09:13:02.830551+0000","last_became_peered":"2023-10-08T09:13:02.830551+0000","last_unstale":"2023-10-08T09:14:37.426536+0000","last_undegraded":"2023-10-08T09:14:37.426536+0000","last_fullsized":"2023-10-08T09:14:37.426536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:26.894614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425970+0000","last_change":"2023-10-08T09:13:07.871322+0000","last_active":"2023-10-08T09:14:37.425970+0000","last_peered":"2023-10-08T09:14:37.425970+0000","last_clean":"2023-10-08T09:14:37.425970+0000","last_became_active":"2023-10-08T09:13:07.870828+0000","last_became_peered":"2023-10-08T09:13:07.870828+0000","last_unstale":"2023-10-08T09:14:37.425970+0000","last_undegraded":"2023-10-08T09:14:37.425970+0000","last_fullsized":"2023-10-08T09:14:37.425970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:37.145705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.558590+0000","last_change":"2023-10-08T09:13:02.373510+0000","last_active":"2023-10-08T09:14:57.558590+0000","last_peered":"2023-10-08T09:14:57.558590+0000","last_clean":"2023-10-08T09:14:57.558590+0000","last_became_active":"2023-10-08T09:13:02.373174+0000","last_became_peered":"2023-10-08T09:13:02.373174+0000","last_unstale":"2023-10-08T09:14:57.558590+0000","last_undegraded":"2023-10-08T09:14:57.558590+0000","last_fullsized":"2023-10-08T09:14:57.558590+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.306454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":87,"num_read_kb":87,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429497+0000","last_change":"2023-10-08T09:13:02.835195+0000","last_active":"2023-10-08T09:14:37.429497+0000","last_peered":"2023-10-08T09:14:37.429497+0000","last_clean":"2023-10-08T09:14:37.429497+0000","last_became_active":"2023-10-08T09:13:02.834893+0000","last_became_peered":"2023-10-08T09:13:02.834893+0000","last_unstale":"2023-10-08T09:14:37.429497+0000","last_undegraded":"2023-10-08T09:14:37.429497+0000","last_fullsized":"2023-10-08T09:14:37.429497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:35.835017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513867+0000","last_change":"2023-10-08T09:13:07.871418+0000","last_active":"2023-10-08T09:14:37.513867+0000","last_peered":"2023-10-08T09:14:37.513867+0000","last_clean":"2023-10-08T09:14:37.513867+0000","last_became_active":"2023-10-08T09:13:07.870961+0000","last_became_peered":"2023-10-08T09:13:07.870961+0000","last_unstale":"2023-10-08T09:14:37.513867+0000","last_undegraded":"2023-10-08T09:14:37.513867+0000","last_fullsized":"2023-10-08T09:14:37.513867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:31.723498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.546688+0000","last_change":"2023-10-08T09:13:02.370728+0000","last_active":"2023-10-08T09:14:38.546688+0000","last_peered":"2023-10-08T09:14:38.546688+0000","last_clean":"2023-10-08T09:14:38.546688+0000","last_became_active":"2023-10-08T09:13:02.370396+0000","last_became_peered":"2023-10-08T09:13:02.370396+0000","last_unstale":"2023-10-08T09:14:38.546688+0000","last_undegraded":"2023-10-08T09:14:38.546688+0000","last_fullsized":"2023-10-08T09:14:38.546688+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:32.271348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737684+0000","last_change":"2023-10-08T09:13:02.827823+0000","last_active":"2023-10-08T09:14:37.737684+0000","last_peered":"2023-10-08T09:14:37.737684+0000","last_clean":"2023-10-08T09:14:37.737684+0000","last_became_active":"2023-10-08T09:13:02.827417+0000","last_became_peered":"2023-10-08T09:13:02.827417+0000","last_unstale":"2023-10-08T09:14:37.737684+0000","last_undegraded":"2023-10-08T09:14:37.737684+0000","last_fullsized":"2023-10-08T09:14:37.737684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:38.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513945+0000","last_change":"2023-10-08T09:13:37.455325+0000","last_active":"2023-10-08T09:14:37.513945+0000","last_peered":"2023-10-08T09:14:37.513945+0000","last_clean":"2023-10-08T09:14:37.513945+0000","last_became_active":"2023-10-08T09:13:37.454985+0000","last_became_peered":"2023-10-08T09:13:37.454985+0000","last_unstale":"2023-10-08T09:14:37.513945+0000","last_undegraded":"2023-10-08T09:14:37.513945+0000","last_fullsized":"2023-10-08T09:14:37.513945+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:13.566831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535311+0000","last_change":"2023-10-08T09:13:02.375244+0000","last_active":"2023-10-08T09:14:37.535311+0000","last_peered":"2023-10-08T09:14:37.535311+0000","last_clean":"2023-10-08T09:14:37.535311+0000","last_became_active":"2023-10-08T09:13:02.374526+0000","last_became_peered":"2023-10-08T09:13:02.374526+0000","last_unstale":"2023-10-08T09:14:37.535311+0000","last_undegraded":"2023-10-08T09:14:37.535311+0000","last_fullsized":"2023-10-08T09:14:37.535311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:16:36.380491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737863+0000","last_change":"2023-10-08T09:13:02.838373+0000","last_active":"2023-10-08T09:14:37.737863+0000","last_peered":"2023-10-08T09:14:37.737863+0000","last_clean":"2023-10-08T09:14:37.737863+0000","last_became_active":"2023-10-08T09:13:02.838100+0000","last_became_peered":"2023-10-08T09:13:02.838100+0000","last_unstale":"2023-10-08T09:14:37.737863+0000","last_undegraded":"2023-10-08T09:14:37.737863+0000","last_fullsized":"2023-10-08T09:14:37.737863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:59.547989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.513906+0000","last_change":"2023-10-08T09:13:07.868929+0000","last_active":"2023-10-08T09:14:37.513906+0000","last_peered":"2023-10-08T09:14:37.513906+0000","last_clean":"2023-10-08T09:14:37.513906+0000","last_became_active":"2023-10-08T09:13:07.868451+0000","last_became_peered":"2023-10-08T09:13:07.868451+0000","last_unstale":"2023-10-08T09:14:37.513906+0000","last_undegraded":"2023-10-08T09:14:37.513906+0000","last_fullsized":"2023-10-08T09:14:37.513906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:36.540319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535135+0000","last_change":"2023-10-08T09:13:02.366404+0000","last_active":"2023-10-08T09:14:37.535135+0000","last_peered":"2023-10-08T09:14:37.535135+0000","last_clean":"2023-10-08T09:14:37.535135+0000","last_became_active":"2023-10-08T09:13:02.366025+0000","last_became_peered":"2023-10-08T09:13:02.366025+0000","last_unstale":"2023-10-08T09:14:37.535135+0000","last_undegraded":"2023-10-08T09:14:37.535135+0000","last_fullsized":"2023-10-08T09:14:37.535135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:51.916565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515611+0000","last_change":"2023-10-08T09:13:02.832852+0000","last_active":"2023-10-08T09:14:37.515611+0000","last_peered":"2023-10-08T09:14:37.515611+0000","last_clean":"2023-10-08T09:14:37.515611+0000","last_became_active":"2023-10-08T09:13:02.832530+0000","last_became_peered":"2023-10-08T09:13:02.832530+0000","last_unstale":"2023-10-08T09:14:37.515611+0000","last_undegraded":"2023-10-08T09:14:37.515611+0000","last_fullsized":"2023-10-08T09:14:37.515611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:20.887377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426351+0000","last_change":"2023-10-08T09:13:07.863864+0000","last_active":"2023-10-08T09:14:37.426351+0000","last_peered":"2023-10-08T09:14:37.426351+0000","last_clean":"2023-10-08T09:14:37.426351+0000","last_became_active":"2023-10-08T09:13:07.863509+0000","last_became_peered":"2023-10-08T09:13:07.863509+0000","last_unstale":"2023-10-08T09:14:37.426351+0000","last_undegraded":"2023-10-08T09:14:37.426351+0000","last_fullsized":"2023-10-08T09:14:37.426351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:22.933327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":117,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.548141+0000","last_change":"2023-10-08T09:13:02.374215+0000","last_active":"2023-10-08T09:14:57.548141+0000","last_peered":"2023-10-08T09:14:57.548141+0000","last_clean":"2023-10-08T09:14:57.548141+0000","last_became_active":"2023-10-08T09:13:02.373912+0000","last_became_peered":"2023-10-08T09:13:02.373912+0000","last_unstale":"2023-10-08T09:14:57.548141+0000","last_undegraded":"2023-10-08T09:14:57.548141+0000","last_fullsized":"2023-10-08T09:14:57.548141+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:18.315244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535090+0000","last_change":"2023-10-08T09:13:02.830651+0000","last_active":"2023-10-08T09:14:37.535090+0000","last_peered":"2023-10-08T09:14:37.535090+0000","last_clean":"2023-10-08T09:14:37.535090+0000","last_became_active":"2023-10-08T09:13:02.830324+0000","last_became_peered":"2023-10-08T09:13:02.830324+0000","last_unstale":"2023-10-08T09:14:37.535090+0000","last_undegraded":"2023-10-08T09:14:37.535090+0000","last_fullsized":"2023-10-08T09:14:37.535090+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:17:02.143794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426203+0000","last_change":"2023-10-08T09:13:37.454054+0000","last_active":"2023-10-08T09:14:37.426203+0000","last_peered":"2023-10-08T09:14:37.426203+0000","last_clean":"2023-10-08T09:14:37.426203+0000","last_became_active":"2023-10-08T09:13:37.453482+0000","last_became_peered":"2023-10-08T09:13:37.453482+0000","last_unstale":"2023-10-08T09:14:37.426203+0000","last_undegraded":"2023-10-08T09:14:37.426203+0000","last_fullsized":"2023-10-08T09:14:37.426203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:31.550837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512273+0000","last_change":"2023-10-08T09:13:02.370334+0000","last_active":"2023-10-08T09:14:37.512273+0000","last_peered":"2023-10-08T09:14:37.512273+0000","last_clean":"2023-10-08T09:14:37.512273+0000","last_became_active":"2023-10-08T09:13:02.370068+0000","last_became_peered":"2023-10-08T09:13:02.370068+0000","last_unstale":"2023-10-08T09:14:37.512273+0000","last_undegraded":"2023-10-08T09:14:37.512273+0000","last_fullsized":"2023-10-08T09:14:37.512273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.962779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515743+0000","last_change":"2023-10-08T09:13:02.832346+0000","last_active":"2023-10-08T09:14:37.515743+0000","last_peered":"2023-10-08T09:14:37.515743+0000","last_clean":"2023-10-08T09:14:37.515743+0000","last_became_active":"2023-10-08T09:13:02.832059+0000","last_became_peered":"2023-10-08T09:13:02.832059+0000","last_unstale":"2023-10-08T09:14:37.515743+0000","last_undegraded":"2023-10-08T09:14:37.515743+0000","last_fullsized":"2023-10-08T09:14:37.515743+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:30.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429192+0000","last_change":"2023-10-08T09:13:07.863466+0000","last_active":"2023-10-08T09:14:37.429192+0000","last_peered":"2023-10-08T09:14:37.429192+0000","last_clean":"2023-10-08T09:14:37.429192+0000","last_became_active":"2023-10-08T09:13:07.863015+0000","last_became_peered":"2023-10-08T09:13:07.863015+0000","last_unstale":"2023-10-08T09:14:37.429192+0000","last_undegraded":"2023-10-08T09:14:37.429192+0000","last_fullsized":"2023-10-08T09:14:37.429192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:30.316625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511291+0000","last_change":"2023-10-08T09:13:02.369444+0000","last_active":"2023-10-08T09:14:37.511291+0000","last_peered":"2023-10-08T09:14:37.511291+0000","last_clean":"2023-10-08T09:14:37.511291+0000","last_became_active":"2023-10-08T09:13:02.369142+0000","last_became_peered":"2023-10-08T09:13:02.369142+0000","last_unstale":"2023-10-08T09:14:37.511291+0000","last_undegraded":"2023-10-08T09:14:37.511291+0000","last_fullsized":"2023-10-08T09:14:37.511291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:28.069205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428912+0000","last_change":"2023-10-08T09:13:02.836125+0000","last_active":"2023-10-08T09:14:37.428912+0000","last_peered":"2023-10-08T09:14:37.428912+0000","last_clean":"2023-10-08T09:14:37.428912+0000","last_became_active":"2023-10-08T09:13:02.835809+0000","last_became_peered":"2023-10-08T09:13:02.835809+0000","last_unstale":"2023-10-08T09:14:37.428912+0000","last_undegraded":"2023-10-08T09:14:37.428912+0000","last_fullsized":"2023-10-08T09:14:37.428912+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:15.548386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737758+0000","last_change":"2023-10-08T09:13:07.861633+0000","last_active":"2023-10-08T09:14:37.737758+0000","last_peered":"2023-10-08T09:14:37.737758+0000","last_clean":"2023-10-08T09:14:37.737758+0000","last_became_active":"2023-10-08T09:13:07.861168+0000","last_became_peered":"2023-10-08T09:13:07.861168+0000","last_unstale":"2023-10-08T09:14:37.737758+0000","last_undegraded":"2023-10-08T09:14:37.737758+0000","last_fullsized":"2023-10-08T09:14:37.737758+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:47.283892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515340+0000","last_change":"2023-10-08T09:13:02.365703+0000","last_active":"2023-10-08T09:14:37.515340+0000","last_peered":"2023-10-08T09:14:37.515340+0000","last_clean":"2023-10-08T09:14:37.515340+0000","last_became_active":"2023-10-08T09:13:02.365421+0000","last_became_peered":"2023-10-08T09:13:02.365421+0000","last_unstale":"2023-10-08T09:14:37.515340+0000","last_undegraded":"2023-10-08T09:14:37.515340+0000","last_fullsized":"2023-10-08T09:14:37.515340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:43.124511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425689+0000","last_change":"2023-10-08T09:13:02.840941+0000","last_active":"2023-10-08T09:14:37.425689+0000","last_peered":"2023-10-08T09:14:37.425689+0000","last_clean":"2023-10-08T09:14:37.425689+0000","last_became_active":"2023-10-08T09:13:02.840605+0000","last_became_peered":"2023-10-08T09:13:02.840605+0000","last_unstale":"2023-10-08T09:14:37.425689+0000","last_undegraded":"2023-10-08T09:14:37.425689+0000","last_fullsized":"2023-10-08T09:14:37.425689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:24:13.417028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429080+0000","last_change":"2023-10-08T09:13:07.868950+0000","last_active":"2023-10-08T09:14:37.429080+0000","last_peered":"2023-10-08T09:14:37.429080+0000","last_clean":"2023-10-08T09:14:37.429080+0000","last_became_active":"2023-10-08T09:13:07.868542+0000","last_became_peered":"2023-10-08T09:13:07.868542+0000","last_unstale":"2023-10-08T09:14:37.429080+0000","last_undegraded":"2023-10-08T09:14:37.429080+0000","last_fullsized":"2023-10-08T09:14:37.429080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:11.452695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":201,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549263+0000","last_change":"2023-10-08T09:13:02.376188+0000","last_active":"2023-10-08T09:14:58.549263+0000","last_peered":"2023-10-08T09:14:58.549263+0000","last_clean":"2023-10-08T09:14:58.549263+0000","last_became_active":"2023-10-08T09:13:02.375914+0000","last_became_peered":"2023-10-08T09:13:02.375914+0000","last_unstale":"2023-10-08T09:14:58.549263+0000","last_undegraded":"2023-10-08T09:14:58.549263+0000","last_fullsized":"2023-10-08T09:14:58.549263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:10.744085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.738033+0000","last_change":"2023-10-08T09:13:02.833429+0000","last_active":"2023-10-08T09:14:37.738033+0000","last_peered":"2023-10-08T09:14:37.738033+0000","last_clean":"2023-10-08T09:14:37.738033+0000","last_became_active":"2023-10-08T09:13:02.832792+0000","last_became_peered":"2023-10-08T09:13:02.832792+0000","last_unstale":"2023-10-08T09:14:37.738033+0000","last_undegraded":"2023-10-08T09:14:37.738033+0000","last_fullsized":"2023-10-08T09:14:37.738033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:24.563447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425404+0000","last_change":"2023-10-08T09:13:07.862688+0000","last_active":"2023-10-08T09:14:37.425404+0000","last_peered":"2023-10-08T09:14:37.425404+0000","last_clean":"2023-10-08T09:14:37.425404+0000","last_became_active":"2023-10-08T09:13:07.862197+0000","last_became_peered":"2023-10-08T09:13:07.862197+0000","last_unstale":"2023-10-08T09:14:37.425404+0000","last_undegraded":"2023-10-08T09:14:37.425404+0000","last_fullsized":"2023-10-08T09:14:37.425404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:56.792045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427056+0000","last_change":"2023-10-08T09:13:02.380283+0000","last_active":"2023-10-08T09:14:37.427056+0000","last_peered":"2023-10-08T09:14:37.427056+0000","last_clean":"2023-10-08T09:14:37.427056+0000","last_became_active":"2023-10-08T09:13:02.379844+0000","last_became_peered":"2023-10-08T09:13:02.379844+0000","last_unstale":"2023-10-08T09:14:37.427056+0000","last_undegraded":"2023-10-08T09:14:37.427056+0000","last_fullsized":"2023-10-08T09:14:37.427056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:18.162437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737756+0000","last_change":"2023-10-08T09:13:02.839211+0000","last_active":"2023-10-08T09:14:37.737756+0000","last_peered":"2023-10-08T09:14:37.737756+0000","last_clean":"2023-10-08T09:14:37.737756+0000","last_became_active":"2023-10-08T09:13:02.838968+0000","last_became_peered":"2023-10-08T09:13:02.838968+0000","last_unstale":"2023-10-08T09:14:37.737756+0000","last_undegraded":"2023-10-08T09:14:37.737756+0000","last_fullsized":"2023-10-08T09:14:37.737756+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:24:02.541749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"86'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514099+0000","last_change":"2023-10-08T09:13:37.457701+0000","last_active":"2023-10-08T09:14:37.514099+0000","last_peered":"2023-10-08T09:14:37.514099+0000","last_clean":"2023-10-08T09:14:37.514099+0000","last_became_active":"2023-10-08T09:13:37.452309+0000","last_became_peered":"2023-10-08T09:13:37.452309+0000","last_unstale":"2023-10-08T09:14:37.514099+0000","last_undegraded":"2023-10-08T09:14:37.514099+0000","last_fullsized":"2023-10-08T09:14:37.514099+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:14.898542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515550+0000","last_change":"2023-10-08T09:13:02.377027+0000","last_active":"2023-10-08T09:14:37.515550+0000","last_peered":"2023-10-08T09:14:37.515550+0000","last_clean":"2023-10-08T09:14:37.515550+0000","last_became_active":"2023-10-08T09:13:02.376746+0000","last_became_peered":"2023-10-08T09:13:02.376746+0000","last_unstale":"2023-10-08T09:14:37.515550+0000","last_undegraded":"2023-10-08T09:14:37.515550+0000","last_fullsized":"2023-10-08T09:14:37.515550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:19.062296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737868+0000","last_change":"2023-10-08T09:13:02.837279+0000","last_active":"2023-10-08T09:14:37.737868+0000","last_peered":"2023-10-08T09:14:37.737868+0000","last_clean":"2023-10-08T09:14:37.737868+0000","last_became_active":"2023-10-08T09:13:02.836966+0000","last_became_peered":"2023-10-08T09:13:02.836966+0000","last_unstale":"2023-10-08T09:14:37.737868+0000","last_undegraded":"2023-10-08T09:14:37.737868+0000","last_fullsized":"2023-10-08T09:14:37.737868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:25.462835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425457+0000","last_change":"2023-10-08T09:13:07.865538+0000","last_active":"2023-10-08T09:14:37.425457+0000","last_peered":"2023-10-08T09:14:37.425457+0000","last_clean":"2023-10-08T09:14:37.425457+0000","last_became_active":"2023-10-08T09:13:07.865118+0000","last_became_peered":"2023-10-08T09:13:07.865118+0000","last_unstale":"2023-10-08T09:14:37.425457+0000","last_undegraded":"2023-10-08T09:14:37.425457+0000","last_fullsized":"2023-10-08T09:14:37.425457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:32.167439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":246,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549867+0000","last_change":"2023-10-08T09:13:02.378173+0000","last_active":"2023-10-08T09:14:58.549867+0000","last_peered":"2023-10-08T09:14:58.549867+0000","last_clean":"2023-10-08T09:14:58.549867+0000","last_became_active":"2023-10-08T09:13:02.377880+0000","last_became_peered":"2023-10-08T09:13:02.377880+0000","last_unstale":"2023-10-08T09:14:58.549867+0000","last_undegraded":"2023-10-08T09:14:58.549867+0000","last_fullsized":"2023-10-08T09:14:58.549867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:43.492910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59058,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":151,"num_write":40,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535159+0000","last_change":"2023-10-08T09:13:02.829874+0000","last_active":"2023-10-08T09:14:37.535159+0000","last_peered":"2023-10-08T09:14:37.535159+0000","last_clean":"2023-10-08T09:14:37.535159+0000","last_became_active":"2023-10-08T09:13:02.829552+0000","last_became_peered":"2023-10-08T09:13:02.829552+0000","last_unstale":"2023-10-08T09:14:37.535159+0000","last_undegraded":"2023-10-08T09:14:37.535159+0000","last_fullsized":"2023-10-08T09:14:37.535159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:29.698812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515922+0000","last_change":"2023-10-08T09:13:07.869661+0000","last_active":"2023-10-08T09:14:37.515922+0000","last_peered":"2023-10-08T09:14:37.515922+0000","last_clean":"2023-10-08T09:14:37.515922+0000","last_became_active":"2023-10-08T09:13:07.869236+0000","last_became_peered":"2023-10-08T09:13:07.869236+0000","last_unstale":"2023-10-08T09:14:37.515922+0000","last_undegraded":"2023-10-08T09:14:37.515922+0000","last_fullsized":"2023-10-08T09:14:37.515922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:21:26.354461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426116+0000","last_change":"2023-10-08T09:13:02.376941+0000","last_active":"2023-10-08T09:14:37.426116+0000","last_peered":"2023-10-08T09:14:37.426116+0000","last_clean":"2023-10-08T09:14:37.426116+0000","last_became_active":"2023-10-08T09:13:02.375810+0000","last_became_peered":"2023-10-08T09:13:02.375810+0000","last_unstale":"2023-10-08T09:14:37.426116+0000","last_undegraded":"2023-10-08T09:14:37.426116+0000","last_fullsized":"2023-10-08T09:14:37.426116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:17.250153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511365+0000","last_change":"2023-10-08T09:13:02.836781+0000","last_active":"2023-10-08T09:14:37.511365+0000","last_peered":"2023-10-08T09:14:37.511365+0000","last_clean":"2023-10-08T09:14:37.511365+0000","last_became_active":"2023-10-08T09:13:02.836493+0000","last_became_peered":"2023-10-08T09:13:02.836493+0000","last_unstale":"2023-10-08T09:14:37.511365+0000","last_undegraded":"2023-10-08T09:14:37.511365+0000","last_fullsized":"2023-10-08T09:14:37.511365+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:03.189574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428895+0000","last_change":"2023-10-08T09:13:07.859104+0000","last_active":"2023-10-08T09:14:37.428895+0000","last_peered":"2023-10-08T09:14:37.428895+0000","last_clean":"2023-10-08T09:14:37.428895+0000","last_became_active":"2023-10-08T09:13:07.858691+0000","last_became_peered":"2023-10-08T09:13:07.858691+0000","last_unstale":"2023-10-08T09:14:37.428895+0000","last_undegraded":"2023-10-08T09:14:37.428895+0000","last_fullsized":"2023-10-08T09:14:37.428895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:33.256044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515740+0000","last_change":"2023-10-08T09:13:02.364472+0000","last_active":"2023-10-08T09:14:37.515740+0000","last_peered":"2023-10-08T09:14:37.515740+0000","last_clean":"2023-10-08T09:14:37.515740+0000","last_became_active":"2023-10-08T09:13:02.364130+0000","last_became_peered":"2023-10-08T09:13:02.364130+0000","last_unstale":"2023-10-08T09:14:37.515740+0000","last_undegraded":"2023-10-08T09:14:37.515740+0000","last_fullsized":"2023-10-08T09:14:37.515740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:18.156186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427207+0000","last_change":"2023-10-08T09:13:02.836119+0000","last_active":"2023-10-08T09:14:37.427207+0000","last_peered":"2023-10-08T09:14:37.427207+0000","last_clean":"2023-10-08T09:14:37.427207+0000","last_became_active":"2023-10-08T09:13:02.835548+0000","last_became_peered":"2023-10-08T09:13:02.835548+0000","last_unstale":"2023-10-08T09:14:37.427207+0000","last_undegraded":"2023-10-08T09:14:37.427207+0000","last_fullsized":"2023-10-08T09:14:37.427207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:34.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426091+0000","last_change":"2023-10-08T09:13:07.871447+0000","last_active":"2023-10-08T09:14:37.426091+0000","last_peered":"2023-10-08T09:14:37.426091+0000","last_clean":"2023-10-08T09:14:37.426091+0000","last_became_active":"2023-10-08T09:13:07.871018+0000","last_became_peered":"2023-10-08T09:13:07.871018+0000","last_unstale":"2023-10-08T09:14:37.426091+0000","last_undegraded":"2023-10-08T09:14:37.426091+0000","last_fullsized":"2023-10-08T09:14:37.426091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:34.361895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:57.547617+0000","last_change":"2023-10-08T09:13:02.378954+0000","last_active":"2023-10-08T09:14:57.547617+0000","last_peered":"2023-10-08T09:14:57.547617+0000","last_clean":"2023-10-08T09:14:57.547617+0000","last_became_active":"2023-10-08T09:13:02.378635+0000","last_became_peered":"2023-10-08T09:13:02.378635+0000","last_unstale":"2023-10-08T09:14:57.547617+0000","last_undegraded":"2023-10-08T09:14:57.547617+0000","last_fullsized":"2023-10-08T09:14:57.547617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:09.536215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":906,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426816+0000","last_change":"2023-10-08T09:13:02.828732+0000","last_active":"2023-10-08T09:14:37.426816+0000","last_peered":"2023-10-08T09:14:37.426816+0000","last_clean":"2023-10-08T09:14:37.426816+0000","last_became_active":"2023-10-08T09:13:02.828449+0000","last_became_peered":"2023-10-08T09:13:02.828449+0000","last_unstale":"2023-10-08T09:14:37.426816+0000","last_undegraded":"2023-10-08T09:14:37.426816+0000","last_fullsized":"2023-10-08T09:14:37.426816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:09.883908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511958+0000","last_change":"2023-10-08T09:13:07.869610+0000","last_active":"2023-10-08T09:14:37.511958+0000","last_peered":"2023-10-08T09:14:37.511958+0000","last_clean":"2023-10-08T09:14:37.511958+0000","last_became_active":"2023-10-08T09:13:07.869225+0000","last_became_peered":"2023-10-08T09:13:07.869225+0000","last_unstale":"2023-10-08T09:14:37.511958+0000","last_undegraded":"2023-10-08T09:14:37.511958+0000","last_fullsized":"2023-10-08T09:14:37.511958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:35.225452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534680+0000","last_change":"2023-10-08T09:13:02.375658+0000","last_active":"2023-10-08T09:14:37.534680+0000","last_peered":"2023-10-08T09:14:37.534680+0000","last_clean":"2023-10-08T09:14:37.534680+0000","last_became_active":"2023-10-08T09:13:02.375383+0000","last_became_peered":"2023-10-08T09:13:02.375383+0000","last_unstale":"2023-10-08T09:14:37.534680+0000","last_undegraded":"2023-10-08T09:14:37.534680+0000","last_fullsized":"2023-10-08T09:14:37.534680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:52.758096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515917+0000","last_change":"2023-10-08T09:13:37.455645+0000","last_active":"2023-10-08T09:14:37.515917+0000","last_peered":"2023-10-08T09:14:37.515917+0000","last_clean":"2023-10-08T09:14:37.515917+0000","last_became_active":"2023-10-08T09:13:37.455207+0000","last_became_peered":"2023-10-08T09:13:37.455207+0000","last_unstale":"2023-10-08T09:14:37.515917+0000","last_undegraded":"2023-10-08T09:14:37.515917+0000","last_fullsized":"2023-10-08T09:14:37.515917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:18.021608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515352+0000","last_change":"2023-10-08T09:13:07.871441+0000","last_active":"2023-10-08T09:14:37.515352+0000","last_peered":"2023-10-08T09:14:37.515352+0000","last_clean":"2023-10-08T09:14:37.515352+0000","last_became_active":"2023-10-08T09:13:07.870968+0000","last_became_peered":"2023-10-08T09:13:07.870968+0000","last_unstale":"2023-10-08T09:14:37.515352+0000","last_undegraded":"2023-10-08T09:14:37.515352+0000","last_fullsized":"2023-10-08T09:14:37.515352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:48.746147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535222+0000","last_change":"2023-10-08T09:13:02.372514+0000","last_active":"2023-10-08T09:14:37.535222+0000","last_peered":"2023-10-08T09:14:37.535222+0000","last_clean":"2023-10-08T09:14:37.535222+0000","last_became_active":"2023-10-08T09:13:02.372181+0000","last_became_peered":"2023-10-08T09:13:02.372181+0000","last_unstale":"2023-10-08T09:14:37.535222+0000","last_undegraded":"2023-10-08T09:14:37.535222+0000","last_fullsized":"2023-10-08T09:14:37.535222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:28.563157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429168+0000","last_change":"2023-10-08T09:13:02.835220+0000","last_active":"2023-10-08T09:14:37.429168+0000","last_peered":"2023-10-08T09:14:37.429168+0000","last_clean":"2023-10-08T09:14:37.429168+0000","last_became_active":"2023-10-08T09:13:02.834914+0000","last_became_peered":"2023-10-08T09:13:02.834914+0000","last_unstale":"2023-10-08T09:14:37.429168+0000","last_undegraded":"2023-10-08T09:14:37.429168+0000","last_fullsized":"2023-10-08T09:14:37.429168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:37.447418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535013+0000","last_change":"2023-10-08T09:13:08.096146+0000","last_active":"2023-10-08T09:14:37.535013+0000","last_peered":"2023-10-08T09:14:37.535013+0000","last_clean":"2023-10-08T09:14:37.535013+0000","last_became_active":"2023-10-08T09:13:08.095703+0000","last_became_peered":"2023-10-08T09:13:08.095703+0000","last_unstale":"2023-10-08T09:14:37.535013+0000","last_undegraded":"2023-10-08T09:14:37.535013+0000","last_fullsized":"2023-10-08T09:14:37.535013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:37.511094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425782+0000","last_change":"2023-10-08T09:13:02.372836+0000","last_active":"2023-10-08T09:14:37.425782+0000","last_peered":"2023-10-08T09:14:37.425782+0000","last_clean":"2023-10-08T09:14:37.425782+0000","last_became_active":"2023-10-08T09:13:02.372488+0000","last_became_peered":"2023-10-08T09:13:02.372488+0000","last_unstale":"2023-10-08T09:14:37.425782+0000","last_undegraded":"2023-10-08T09:14:37.425782+0000","last_fullsized":"2023-10-08T09:14:37.425782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:19.269539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512564+0000","last_change":"2023-10-08T09:13:02.827306+0000","last_active":"2023-10-08T09:14:37.512564+0000","last_peered":"2023-10-08T09:14:37.512564+0000","last_clean":"2023-10-08T09:14:37.512564+0000","last_became_active":"2023-10-08T09:13:02.826965+0000","last_became_peered":"2023-10-08T09:13:02.826965+0000","last_unstale":"2023-10-08T09:14:37.512564+0000","last_undegraded":"2023-10-08T09:14:37.512564+0000","last_fullsized":"2023-10-08T09:14:37.512564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:27.513860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511590+0000","last_change":"2023-10-08T09:13:07.862637+0000","last_active":"2023-10-08T09:14:37.511590+0000","last_peered":"2023-10-08T09:14:37.511590+0000","last_clean":"2023-10-08T09:14:37.511590+0000","last_became_active":"2023-10-08T09:13:07.862117+0000","last_became_peered":"2023-10-08T09:13:07.862117+0000","last_unstale":"2023-10-08T09:14:37.511590+0000","last_undegraded":"2023-10-08T09:14:37.511590+0000","last_fullsized":"2023-10-08T09:14:37.511590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:30:14.475228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'7","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:59.561747+0000","last_change":"2023-10-08T09:13:02.379585+0000","last_active":"2023-10-08T09:14:59.561747+0000","last_peered":"2023-10-08T09:14:59.561747+0000","last_clean":"2023-10-08T09:14:59.561747+0000","last_became_active":"2023-10-08T09:13:02.379324+0000","last_became_peered":"2023-10-08T09:13:02.379324+0000","last_unstale":"2023-10-08T09:14:59.561747+0000","last_undegraded":"2023-10-08T09:14:59.561747+0000","last_fullsized":"2023-10-08T09:14:59.561747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:31.609558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.512613+0000","last_change":"2023-10-08T09:13:02.833416+0000","last_active":"2023-10-08T09:14:37.512613+0000","last_peered":"2023-10-08T09:14:37.512613+0000","last_clean":"2023-10-08T09:14:37.512613+0000","last_became_active":"2023-10-08T09:13:02.833040+0000","last_became_peered":"2023-10-08T09:13:02.833040+0000","last_unstale":"2023-10-08T09:14:37.512613+0000","last_undegraded":"2023-10-08T09:14:37.512613+0000","last_fullsized":"2023-10-08T09:14:37.512613+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:44.480445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511906+0000","last_change":"2023-10-08T09:13:08.095723+0000","last_active":"2023-10-08T09:14:37.511906+0000","last_peered":"2023-10-08T09:14:37.511906+0000","last_clean":"2023-10-08T09:14:37.511906+0000","last_became_active":"2023-10-08T09:13:08.095271+0000","last_became_peered":"2023-10-08T09:13:08.095271+0000","last_unstale":"2023-10-08T09:14:37.511906+0000","last_undegraded":"2023-10-08T09:14:37.511906+0000","last_fullsized":"2023-10-08T09:14:37.511906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:04.021426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515464+0000","last_change":"2023-10-08T09:13:02.829663+0000","last_active":"2023-10-08T09:14:37.515464+0000","last_peered":"2023-10-08T09:14:37.515464+0000","last_clean":"2023-10-08T09:14:37.515464+0000","last_became_active":"2023-10-08T09:13:02.829354+0000","last_became_peered":"2023-10-08T09:13:02.829354+0000","last_unstale":"2023-10-08T09:14:37.515464+0000","last_undegraded":"2023-10-08T09:14:37.515464+0000","last_fullsized":"2023-10-08T09:14:37.515464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:55.332346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:58.549710+0000","last_change":"2023-10-08T09:13:02.372114+0000","last_active":"2023-10-08T09:14:58.549710+0000","last_peered":"2023-10-08T09:14:58.549710+0000","last_clean":"2023-10-08T09:14:58.549710+0000","last_became_active":"2023-10-08T09:13:02.371603+0000","last_became_peered":"2023-10-08T09:13:02.371603+0000","last_unstale":"2023-10-08T09:14:58.549710+0000","last_undegraded":"2023-10-08T09:14:58.549710+0000","last_fullsized":"2023-10-08T09:14:58.549710+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:39.078307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427099+0000","last_change":"2023-10-08T09:13:07.863492+0000","last_active":"2023-10-08T09:14:37.427099+0000","last_peered":"2023-10-08T09:14:37.427099+0000","last_clean":"2023-10-08T09:14:37.427099+0000","last_became_active":"2023-10-08T09:13:07.862803+0000","last_became_peered":"2023-10-08T09:13:07.862803+0000","last_unstale":"2023-10-08T09:14:37.427099+0000","last_undegraded":"2023-10-08T09:14:37.427099+0000","last_fullsized":"2023-10-08T09:14:37.427099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:36:39.618214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428936+0000","last_change":"2023-10-08T09:13:02.836596+0000","last_active":"2023-10-08T09:14:37.428936+0000","last_peered":"2023-10-08T09:14:37.428936+0000","last_clean":"2023-10-08T09:14:37.428936+0000","last_became_active":"2023-10-08T09:13:02.836283+0000","last_became_peered":"2023-10-08T09:13:02.836283+0000","last_unstale":"2023-10-08T09:14:37.428936+0000","last_undegraded":"2023-10-08T09:14:37.428936+0000","last_fullsized":"2023-10-08T09:14:37.428936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:02.393575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514133+0000","last_change":"2023-10-08T09:13:02.379516+0000","last_active":"2023-10-08T09:14:37.514133+0000","last_peered":"2023-10-08T09:14:37.514133+0000","last_clean":"2023-10-08T09:14:37.514133+0000","last_became_active":"2023-10-08T09:13:02.379229+0000","last_became_peered":"2023-10-08T09:13:02.379229+0000","last_unstale":"2023-10-08T09:14:37.514133+0000","last_undegraded":"2023-10-08T09:14:37.514133+0000","last_fullsized":"2023-10-08T09:14:37.514133+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:55.294651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":315,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.533330+0000","last_change":"2023-10-08T09:12:02.620769+0000","last_active":"2023-10-08T09:14:38.533330+0000","last_peered":"2023-10-08T09:14:38.533330+0000","last_clean":"2023-10-08T09:14:38.533330+0000","last_became_active":"2023-10-08T09:12:02.620388+0000","last_became_peered":"2023-10-08T09:12:02.620388+0000","last_unstale":"2023-10-08T09:14:38.533330+0000","last_undegraded":"2023-10-08T09:14:38.533330+0000","last_fullsized":"2023-10-08T09:14:38.533330+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:09:10.746288+0000","last_clean_scrub_stamp":"2023-10-08T09:09:10.746288+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:55:35.713060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515785+0000","last_change":"2023-10-08T09:13:08.096318+0000","last_active":"2023-10-08T09:14:37.515785+0000","last_peered":"2023-10-08T09:14:37.515785+0000","last_clean":"2023-10-08T09:14:37.515785+0000","last_became_active":"2023-10-08T09:13:08.095774+0000","last_became_peered":"2023-10-08T09:13:08.095774+0000","last_unstale":"2023-10-08T09:14:37.515785+0000","last_undegraded":"2023-10-08T09:14:37.515785+0000","last_fullsized":"2023-10-08T09:14:37.515785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:53.089468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511157+0000","last_change":"2023-10-08T09:13:02.835167+0000","last_active":"2023-10-08T09:14:37.511157+0000","last_peered":"2023-10-08T09:14:37.511157+0000","last_clean":"2023-10-08T09:14:37.511157+0000","last_became_active":"2023-10-08T09:13:02.834850+0000","last_became_peered":"2023-10-08T09:13:02.834850+0000","last_unstale":"2023-10-08T09:14:37.511157+0000","last_undegraded":"2023-10-08T09:14:37.511157+0000","last_fullsized":"2023-10-08T09:14:37.511157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:19.422372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426957+0000","last_change":"2023-10-08T09:13:02.377746+0000","last_active":"2023-10-08T09:14:37.426957+0000","last_peered":"2023-10-08T09:14:37.426957+0000","last_clean":"2023-10-08T09:14:37.426957+0000","last_became_active":"2023-10-08T09:13:02.377422+0000","last_became_peered":"2023-10-08T09:13:02.377422+0000","last_unstale":"2023-10-08T09:14:37.426957+0000","last_undegraded":"2023-10-08T09:14:37.426957+0000","last_fullsized":"2023-10-08T09:14:37.426957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:13.130329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428886+0000","last_change":"2023-10-08T09:13:07.864678+0000","last_active":"2023-10-08T09:14:37.428886+0000","last_peered":"2023-10-08T09:14:37.428886+0000","last_clean":"2023-10-08T09:14:37.428886+0000","last_became_active":"2023-10-08T09:13:07.864226+0000","last_became_peered":"2023-10-08T09:13:07.864226+0000","last_unstale":"2023-10-08T09:14:37.428886+0000","last_undegraded":"2023-10-08T09:14:37.428886+0000","last_fullsized":"2023-10-08T09:14:37.428886+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:43:51.179853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.535266+0000","last_change":"2023-10-08T09:13:02.370014+0000","last_active":"2023-10-08T09:14:37.535266+0000","last_peered":"2023-10-08T09:14:37.535266+0000","last_clean":"2023-10-08T09:14:37.535266+0000","last_became_active":"2023-10-08T09:13:02.369704+0000","last_became_peered":"2023-10-08T09:13:02.369704+0000","last_unstale":"2023-10-08T09:14:37.535266+0000","last_undegraded":"2023-10-08T09:14:37.535266+0000","last_fullsized":"2023-10-08T09:14:37.535266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:28.902274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511727+0000","last_change":"2023-10-08T09:13:02.833859+0000","last_active":"2023-10-08T09:14:37.511727+0000","last_peered":"2023-10-08T09:14:37.511727+0000","last_clean":"2023-10-08T09:14:37.511727+0000","last_became_active":"2023-10-08T09:13:02.833546+0000","last_became_peered":"2023-10-08T09:13:02.833546+0000","last_unstale":"2023-10-08T09:14:37.511727+0000","last_undegraded":"2023-10-08T09:14:37.511727+0000","last_fullsized":"2023-10-08T09:14:37.511727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:35.534588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428821+0000","last_change":"2023-10-08T09:13:08.095622+0000","last_active":"2023-10-08T09:14:37.428821+0000","last_peered":"2023-10-08T09:14:37.428821+0000","last_clean":"2023-10-08T09:14:37.428821+0000","last_became_active":"2023-10-08T09:13:08.095143+0000","last_became_peered":"2023-10-08T09:13:08.095143+0000","last_unstale":"2023-10-08T09:14:37.428821+0000","last_undegraded":"2023-10-08T09:14:37.428821+0000","last_fullsized":"2023-10-08T09:14:37.428821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:46.738102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426121+0000","last_change":"2023-10-08T09:13:37.459185+0000","last_active":"2023-10-08T09:14:37.426121+0000","last_peered":"2023-10-08T09:14:37.426121+0000","last_clean":"2023-10-08T09:14:37.426121+0000","last_became_active":"2023-10-08T09:13:37.451105+0000","last_became_peered":"2023-10-08T09:13:37.451105+0000","last_unstale":"2023-10-08T09:14:37.426121+0000","last_undegraded":"2023-10-08T09:14:37.426121+0000","last_fullsized":"2023-10-08T09:14:37.426121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:03.768269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429624+0000","last_change":"2023-10-08T09:13:02.834053+0000","last_active":"2023-10-08T09:14:37.429624+0000","last_peered":"2023-10-08T09:14:37.429624+0000","last_clean":"2023-10-08T09:14:37.429624+0000","last_became_active":"2023-10-08T09:13:02.833707+0000","last_became_peered":"2023-10-08T09:13:02.833707+0000","last_unstale":"2023-10-08T09:14:37.429624+0000","last_undegraded":"2023-10-08T09:14:37.429624+0000","last_fullsized":"2023-10-08T09:14:37.429624+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:31.372003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425862+0000","last_change":"2023-10-08T09:13:07.855734+0000","last_active":"2023-10-08T09:14:37.425862+0000","last_peered":"2023-10-08T09:14:37.425862+0000","last_clean":"2023-10-08T09:14:37.425862+0000","last_became_active":"2023-10-08T09:13:07.855313+0000","last_became_peered":"2023-10-08T09:13:07.855313+0000","last_unstale":"2023-10-08T09:14:37.425862+0000","last_undegraded":"2023-10-08T09:14:37.425862+0000","last_fullsized":"2023-10-08T09:14:37.425862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:15.207337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514427+0000","last_change":"2023-10-08T09:13:02.371319+0000","last_active":"2023-10-08T09:14:37.514427+0000","last_peered":"2023-10-08T09:14:37.514427+0000","last_clean":"2023-10-08T09:14:37.514427+0000","last_became_active":"2023-10-08T09:13:02.371049+0000","last_became_peered":"2023-10-08T09:13:02.371049+0000","last_unstale":"2023-10-08T09:14:37.514427+0000","last_undegraded":"2023-10-08T09:14:37.514427+0000","last_fullsized":"2023-10-08T09:14:37.514427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:28.581408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737818+0000","last_change":"2023-10-08T09:13:02.838682+0000","last_active":"2023-10-08T09:14:37.737818+0000","last_peered":"2023-10-08T09:14:37.737818+0000","last_clean":"2023-10-08T09:14:37.737818+0000","last_became_active":"2023-10-08T09:13:02.838389+0000","last_became_peered":"2023-10-08T09:13:02.838389+0000","last_unstale":"2023-10-08T09:14:37.737818+0000","last_undegraded":"2023-10-08T09:14:37.737818+0000","last_fullsized":"2023-10-08T09:14:37.737818+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:52.937723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514169+0000","last_change":"2023-10-08T09:13:08.098131+0000","last_active":"2023-10-08T09:14:37.514169+0000","last_peered":"2023-10-08T09:14:37.514169+0000","last_clean":"2023-10-08T09:14:37.514169+0000","last_became_active":"2023-10-08T09:13:08.097352+0000","last_became_peered":"2023-10-08T09:13:08.097352+0000","last_unstale":"2023-10-08T09:14:37.514169+0000","last_undegraded":"2023-10-08T09:14:37.514169+0000","last_fullsized":"2023-10-08T09:14:37.514169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:30.388875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737576+0000","last_change":"2023-10-08T09:13:37.461067+0000","last_active":"2023-10-08T09:14:37.737576+0000","last_peered":"2023-10-08T09:14:37.737576+0000","last_clean":"2023-10-08T09:14:37.737576+0000","last_became_active":"2023-10-08T09:13:37.451739+0000","last_became_peered":"2023-10-08T09:13:37.451739+0000","last_unstale":"2023-10-08T09:14:37.737576+0000","last_undegraded":"2023-10-08T09:14:37.737576+0000","last_fullsized":"2023-10-08T09:14:37.737576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:26.603798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534742+0000","last_change":"2023-10-08T09:13:02.830017+0000","last_active":"2023-10-08T09:14:37.534742+0000","last_peered":"2023-10-08T09:14:37.534742+0000","last_clean":"2023-10-08T09:14:37.534742+0000","last_became_active":"2023-10-08T09:13:02.829700+0000","last_became_peered":"2023-10-08T09:13:02.829700+0000","last_unstale":"2023-10-08T09:14:37.534742+0000","last_undegraded":"2023-10-08T09:14:37.534742+0000","last_fullsized":"2023-10-08T09:14:37.534742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:31:31.387049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426196+0000","last_change":"2023-10-08T09:13:07.869097+0000","last_active":"2023-10-08T09:14:37.426196+0000","last_peered":"2023-10-08T09:14:37.426196+0000","last_clean":"2023-10-08T09:14:37.426196+0000","last_became_active":"2023-10-08T09:13:07.868710+0000","last_became_peered":"2023-10-08T09:13:07.868710+0000","last_unstale":"2023-10-08T09:14:37.426196+0000","last_undegraded":"2023-10-08T09:14:37.426196+0000","last_fullsized":"2023-10-08T09:14:37.426196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:37:27.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429207+0000","last_change":"2023-10-08T09:13:02.370694+0000","last_active":"2023-10-08T09:14:37.429207+0000","last_peered":"2023-10-08T09:14:37.429207+0000","last_clean":"2023-10-08T09:14:37.429207+0000","last_became_active":"2023-10-08T09:13:02.370161+0000","last_became_peered":"2023-10-08T09:13:02.370161+0000","last_unstale":"2023-10-08T09:14:37.429207+0000","last_undegraded":"2023-10-08T09:14:37.429207+0000","last_fullsized":"2023-10-08T09:14:37.429207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:04.809703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428791+0000","last_change":"2023-10-08T09:13:02.831026+0000","last_active":"2023-10-08T09:14:37.428791+0000","last_peered":"2023-10-08T09:14:37.428791+0000","last_clean":"2023-10-08T09:14:37.428791+0000","last_became_active":"2023-10-08T09:13:02.830715+0000","last_became_peered":"2023-10-08T09:13:02.830715+0000","last_unstale":"2023-10-08T09:14:37.428791+0000","last_undegraded":"2023-10-08T09:14:37.428791+0000","last_fullsized":"2023-10-08T09:14:37.428791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:47.709820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514173+0000","last_change":"2023-10-08T09:13:07.860464+0000","last_active":"2023-10-08T09:14:37.514173+0000","last_peered":"2023-10-08T09:14:37.514173+0000","last_clean":"2023-10-08T09:14:37.514173+0000","last_became_active":"2023-10-08T09:13:07.859962+0000","last_became_peered":"2023-10-08T09:13:07.859962+0000","last_unstale":"2023-10-08T09:14:37.514173+0000","last_undegraded":"2023-10-08T09:14:37.514173+0000","last_fullsized":"2023-10-08T09:14:37.514173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:04.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426052+0000","last_change":"2023-10-08T09:13:02.371308+0000","last_active":"2023-10-08T09:14:37.426052+0000","last_peered":"2023-10-08T09:14:37.426052+0000","last_clean":"2023-10-08T09:14:37.426052+0000","last_became_active":"2023-10-08T09:13:02.370968+0000","last_became_peered":"2023-10-08T09:13:02.370968+0000","last_unstale":"2023-10-08T09:14:37.426052+0000","last_undegraded":"2023-10-08T09:14:37.426052+0000","last_fullsized":"2023-10-08T09:14:37.426052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:58.491112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429125+0000","last_change":"2023-10-08T09:13:02.826511+0000","last_active":"2023-10-08T09:14:37.429125+0000","last_peered":"2023-10-08T09:14:37.429125+0000","last_clean":"2023-10-08T09:14:37.429125+0000","last_became_active":"2023-10-08T09:13:02.826205+0000","last_became_peered":"2023-10-08T09:13:02.826205+0000","last_unstale":"2023-10-08T09:14:37.429125+0000","last_undegraded":"2023-10-08T09:14:37.429125+0000","last_fullsized":"2023-10-08T09:14:37.429125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:12.270176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.514117+0000","last_change":"2023-10-08T09:13:08.096360+0000","last_active":"2023-10-08T09:14:37.514117+0000","last_peered":"2023-10-08T09:14:37.514117+0000","last_clean":"2023-10-08T09:14:37.514117+0000","last_became_active":"2023-10-08T09:13:08.095915+0000","last_became_peered":"2023-10-08T09:13:08.095915+0000","last_unstale":"2023-10-08T09:14:37.514117+0000","last_undegraded":"2023-10-08T09:14:37.514117+0000","last_fullsized":"2023-10-08T09:14:37.514117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:59.412471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.534927+0000","last_change":"2023-10-08T09:13:02.375287+0000","last_active":"2023-10-08T09:14:37.534927+0000","last_peered":"2023-10-08T09:14:37.534927+0000","last_clean":"2023-10-08T09:14:37.534927+0000","last_became_active":"2023-10-08T09:13:02.374668+0000","last_became_peered":"2023-10-08T09:13:02.374668+0000","last_unstale":"2023-10-08T09:14:37.534927+0000","last_undegraded":"2023-10-08T09:14:37.534927+0000","last_fullsized":"2023-10-08T09:14:37.534927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:22.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426986+0000","last_change":"2023-10-08T09:13:02.828240+0000","last_active":"2023-10-08T09:14:37.426986+0000","last_peered":"2023-10-08T09:14:37.426986+0000","last_clean":"2023-10-08T09:14:37.426986+0000","last_became_active":"2023-10-08T09:13:02.827951+0000","last_became_peered":"2023-10-08T09:13:02.827951+0000","last_unstale":"2023-10-08T09:14:37.426986+0000","last_undegraded":"2023-10-08T09:14:37.426986+0000","last_fullsized":"2023-10-08T09:14:37.426986+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:37.980713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737603+0000","last_change":"2023-10-08T09:13:07.865846+0000","last_active":"2023-10-08T09:14:37.737603+0000","last_peered":"2023-10-08T09:14:37.737603+0000","last_clean":"2023-10-08T09:14:37.737603+0000","last_became_active":"2023-10-08T09:13:07.865529+0000","last_became_peered":"2023-10-08T09:13:07.865529+0000","last_unstale":"2023-10-08T09:14:37.737603+0000","last_undegraded":"2023-10-08T09:14:37.737603+0000","last_fullsized":"2023-10-08T09:14:37.737603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:34:03.408036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'7","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:59.571238+0000","last_change":"2023-10-08T09:13:02.380128+0000","last_active":"2023-10-08T09:14:59.571238+0000","last_peered":"2023-10-08T09:14:59.571238+0000","last_clean":"2023-10-08T09:14:59.571238+0000","last_became_active":"2023-10-08T09:13:02.379800+0000","last_became_peered":"2023-10-08T09:13:02.379800+0000","last_unstale":"2023-10-08T09:14:59.571238+0000","last_undegraded":"2023-10-08T09:14:59.571238+0000","last_fullsized":"2023-10-08T09:14:59.571238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:01.546289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427243+0000","last_change":"2023-10-08T09:13:02.829474+0000","last_active":"2023-10-08T09:14:37.427243+0000","last_peered":"2023-10-08T09:14:37.427243+0000","last_clean":"2023-10-08T09:14:37.427243+0000","last_became_active":"2023-10-08T09:13:02.829173+0000","last_became_peered":"2023-10-08T09:13:02.829173+0000","last_unstale":"2023-10-08T09:14:37.427243+0000","last_undegraded":"2023-10-08T09:14:37.427243+0000","last_fullsized":"2023-10-08T09:14:37.427243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:02.469158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737465+0000","last_change":"2023-10-08T09:13:37.453778+0000","last_active":"2023-10-08T09:14:37.737465+0000","last_peered":"2023-10-08T09:14:37.737465+0000","last_clean":"2023-10-08T09:14:37.737465+0000","last_became_active":"2023-10-08T09:13:37.453354+0000","last_became_peered":"2023-10-08T09:13:37.453354+0000","last_unstale":"2023-10-08T09:14:37.737465+0000","last_undegraded":"2023-10-08T09:14:37.737465+0000","last_fullsized":"2023-10-08T09:14:37.737465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:50.035588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515804+0000","last_change":"2023-10-08T09:13:02.376641+0000","last_active":"2023-10-08T09:14:37.515804+0000","last_peered":"2023-10-08T09:14:37.515804+0000","last_clean":"2023-10-08T09:14:37.515804+0000","last_became_active":"2023-10-08T09:13:02.376348+0000","last_became_peered":"2023-10-08T09:13:02.376348+0000","last_unstale":"2023-10-08T09:14:37.515804+0000","last_undegraded":"2023-10-08T09:14:37.515804+0000","last_fullsized":"2023-10-08T09:14:37.515804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:42.787837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427333+0000","last_change":"2023-10-08T09:13:02.836201+0000","last_active":"2023-10-08T09:14:37.427333+0000","last_peered":"2023-10-08T09:14:37.427333+0000","last_clean":"2023-10-08T09:14:37.427333+0000","last_became_active":"2023-10-08T09:13:02.835885+0000","last_became_peered":"2023-10-08T09:13:02.835885+0000","last_unstale":"2023-10-08T09:14:37.427333+0000","last_undegraded":"2023-10-08T09:14:37.427333+0000","last_fullsized":"2023-10-08T09:14:37.427333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:34.310854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515138+0000","last_change":"2023-10-08T09:13:07.863854+0000","last_active":"2023-10-08T09:14:37.515138+0000","last_peered":"2023-10-08T09:14:37.515138+0000","last_clean":"2023-10-08T09:14:37.515138+0000","last_became_active":"2023-10-08T09:13:07.863460+0000","last_became_peered":"2023-10-08T09:13:07.863460+0000","last_unstale":"2023-10-08T09:14:37.515138+0000","last_undegraded":"2023-10-08T09:14:37.515138+0000","last_fullsized":"2023-10-08T09:14:37.515138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:06.832902+0000","last_clean_scrub_stamp":"2023-10-08T09:13:06.832902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:03.093858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511956+0000","last_change":"2023-10-08T09:13:02.372336+0000","last_active":"2023-10-08T09:14:37.511956+0000","last_peered":"2023-10-08T09:14:37.511956+0000","last_clean":"2023-10-08T09:14:37.511956+0000","last_became_active":"2023-10-08T09:13:02.372022+0000","last_became_peered":"2023-10-08T09:13:02.372022+0000","last_unstale":"2023-10-08T09:14:37.511956+0000","last_undegraded":"2023-10-08T09:14:37.511956+0000","last_fullsized":"2023-10-08T09:14:37.511956+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:01.343291+0000","last_clean_scrub_stamp":"2023-10-08T09:13:01.343291+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:10.610332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.427293+0000","last_change":"2023-10-08T09:13:02.832430+0000","last_active":"2023-10-08T09:14:37.427293+0000","last_peered":"2023-10-08T09:14:37.427293+0000","last_clean":"2023-10-08T09:14:37.427293+0000","last_became_active":"2023-10-08T09:13:02.832092+0000","last_became_peered":"2023-10-08T09:13:02.832092+0000","last_unstale":"2023-10-08T09:14:37.427293+0000","last_undegraded":"2023-10-08T09:14:37.427293+0000","last_fullsized":"2023-10-08T09:14:37.427293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:00.798900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428785+0000","last_change":"2023-10-08T09:13:02.833114+0000","last_active":"2023-10-08T09:14:37.428785+0000","last_peered":"2023-10-08T09:14:37.428785+0000","last_clean":"2023-10-08T09:14:37.428785+0000","last_became_active":"2023-10-08T09:13:02.830959+0000","last_became_peered":"2023-10-08T09:13:02.830959+0000","last_unstale":"2023-10-08T09:14:37.428785+0000","last_undegraded":"2023-10-08T09:14:37.428785+0000","last_fullsized":"2023-10-08T09:14:37.428785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:13.853585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426056+0000","last_change":"2023-10-08T09:13:02.836046+0000","last_active":"2023-10-08T09:14:37.426056+0000","last_peered":"2023-10-08T09:14:37.426056+0000","last_clean":"2023-10-08T09:14:37.426056+0000","last_became_active":"2023-10-08T09:13:02.835722+0000","last_became_peered":"2023-10-08T09:13:02.835722+0000","last_unstale":"2023-10-08T09:14:37.426056+0000","last_undegraded":"2023-10-08T09:14:37.426056+0000","last_fullsized":"2023-10-08T09:14:37.426056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:00.316259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.425931+0000","last_change":"2023-10-08T09:13:02.830021+0000","last_active":"2023-10-08T09:14:37.425931+0000","last_peered":"2023-10-08T09:14:37.425931+0000","last_clean":"2023-10-08T09:14:37.425931+0000","last_became_active":"2023-10-08T09:13:02.829397+0000","last_became_peered":"2023-10-08T09:13:02.829397+0000","last_unstale":"2023-10-08T09:14:37.425931+0000","last_undegraded":"2023-10-08T09:14:37.425931+0000","last_fullsized":"2023-10-08T09:14:37.425931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:29.153457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737569+0000","last_change":"2023-10-08T09:13:02.832002+0000","last_active":"2023-10-08T09:14:37.737569+0000","last_peered":"2023-10-08T09:14:37.737569+0000","last_clean":"2023-10-08T09:14:37.737569+0000","last_became_active":"2023-10-08T09:13:02.831666+0000","last_became_peered":"2023-10-08T09:13:02.831666+0000","last_unstale":"2023-10-08T09:14:37.737569+0000","last_undegraded":"2023-10-08T09:14:37.737569+0000","last_fullsized":"2023-10-08T09:14:37.737569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:16.414127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511793+0000","last_change":"2023-10-08T09:13:02.835881+0000","last_active":"2023-10-08T09:14:37.511793+0000","last_peered":"2023-10-08T09:14:37.511793+0000","last_clean":"2023-10-08T09:14:37.511793+0000","last_became_active":"2023-10-08T09:13:02.835587+0000","last_became_peered":"2023-10-08T09:13:02.835587+0000","last_unstale":"2023-10-08T09:14:37.511793+0000","last_undegraded":"2023-10-08T09:14:37.511793+0000","last_fullsized":"2023-10-08T09:14:37.511793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:52.323234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429154+0000","last_change":"2023-10-08T09:13:02.826371+0000","last_active":"2023-10-08T09:14:37.429154+0000","last_peered":"2023-10-08T09:14:37.429154+0000","last_clean":"2023-10-08T09:14:37.429154+0000","last_became_active":"2023-10-08T09:13:02.826093+0000","last_became_peered":"2023-10-08T09:13:02.826093+0000","last_unstale":"2023-10-08T09:14:37.429154+0000","last_undegraded":"2023-10-08T09:14:37.429154+0000","last_fullsized":"2023-10-08T09:14:37.429154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:17:08.113739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.426543+0000","last_change":"2023-10-08T09:13:02.829936+0000","last_active":"2023-10-08T09:14:37.426543+0000","last_peered":"2023-10-08T09:14:37.426543+0000","last_clean":"2023-10-08T09:14:37.426543+0000","last_became_active":"2023-10-08T09:13:02.829498+0000","last_became_peered":"2023-10-08T09:13:02.829498+0000","last_unstale":"2023-10-08T09:14:37.426543+0000","last_undegraded":"2023-10-08T09:14:37.426543+0000","last_fullsized":"2023-10-08T09:14:37.426543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:42.200791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.429728+0000","last_change":"2023-10-08T09:13:02.840415+0000","last_active":"2023-10-08T09:14:37.429728+0000","last_peered":"2023-10-08T09:14:37.429728+0000","last_clean":"2023-10-08T09:14:37.429728+0000","last_became_active":"2023-10-08T09:13:02.840075+0000","last_became_peered":"2023-10-08T09:13:02.840075+0000","last_unstale":"2023-10-08T09:14:37.429728+0000","last_undegraded":"2023-10-08T09:14:37.429728+0000","last_fullsized":"2023-10-08T09:14:37.429728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:48.959372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.515335+0000","last_change":"2023-10-08T09:13:02.828337+0000","last_active":"2023-10-08T09:14:37.515335+0000","last_peered":"2023-10-08T09:14:37.515335+0000","last_clean":"2023-10-08T09:14:37.515335+0000","last_became_active":"2023-10-08T09:13:02.827998+0000","last_became_peered":"2023-10-08T09:13:02.827998+0000","last_unstale":"2023-10-08T09:14:37.515335+0000","last_undegraded":"2023-10-08T09:14:37.515335+0000","last_fullsized":"2023-10-08T09:14:37.515335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:27.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T09:14:38.547574+0000","last_change":"2023-10-08T09:14:38.547245+0000","last_active":"2023-10-08T09:14:38.547574+0000","last_peered":"2023-10-08T09:14:38.547574+0000","last_clean":"2023-10-08T09:14:38.547574+0000","last_became_active":"2023-10-08T09:14:38.546964+0000","last_became_peered":"2023-10-08T09:14:38.546964+0000","last_unstale":"2023-10-08T09:14:38.547574+0000","last_undegraded":"2023-10-08T09:14:38.547574+0000","last_fullsized":"2023-10-08T09:14:38.547574+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:40.120946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.737724+0000","last_change":"2023-10-08T09:13:02.840921+0000","last_active":"2023-10-08T09:14:37.737724+0000","last_peered":"2023-10-08T09:14:37.737724+0000","last_clean":"2023-10-08T09:14:37.737724+0000","last_became_active":"2023-10-08T09:13:02.840566+0000","last_became_peered":"2023-10-08T09:13:02.840566+0000","last_unstale":"2023-10-08T09:14:37.737724+0000","last_undegraded":"2023-10-08T09:14:37.737724+0000","last_fullsized":"2023-10-08T09:14:37.737724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:20.219581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.511631+0000","last_change":"2023-10-08T09:13:02.834491+0000","last_active":"2023-10-08T09:14:37.511631+0000","last_peered":"2023-10-08T09:14:37.511631+0000","last_clean":"2023-10-08T09:14:37.511631+0000","last_became_active":"2023-10-08T09:13:02.834058+0000","last_became_peered":"2023-10-08T09:13:02.834058+0000","last_unstale":"2023-10-08T09:14:37.511631+0000","last_undegraded":"2023-10-08T09:14:37.511631+0000","last_fullsized":"2023-10-08T09:14:37.511631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:47.699423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T09:14:37.428850+0000","last_change":"2023-10-08T09:13:02.836433+0000","last_active":"2023-10-08T09:14:37.428850+0000","last_peered":"2023-10-08T09:14:37.428850+0000","last_clean":"2023-10-08T09:14:37.428850+0000","last_became_active":"2023-10-08T09:13:02.835147+0000","last_became_peered":"2023-10-08T09:13:02.835147+0000","last_unstale":"2023-10-08T09:14:37.428850+0000","last_undegraded":"2023-10-08T09:14:37.428850+0000","last_fullsized":"2023-10-08T09:14:37.428850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T09:13:02.352095+0000","last_clean_scrub_stamp":"2023-10-08T09:13:02.352095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:51:51.233662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":247538,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66860,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":919,"num_read_kb":533,"num_write":247,"num_write_kb":143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":11,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514532,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":507696,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.69999999999999996,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59699999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90100000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.66300000000000003,"15min":0.67000000000000004},"min":{"1min":0.42599999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79300000000000004,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.81799999999999995}]},{"osd":2,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":4,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86699999999999999,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.47799999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.77400000000000002,"5min":1.147,"15min":1.147},"last":0.86899999999999999}]},{"osd":5,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.62,"15min":0.63},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68999999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70599999999999996,"15min":0.71399999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83799999999999997,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.78700000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0389999999999999,"5min":2.222,"15min":2.222},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.77200000000000002,"15min":0.79100000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.90400000000000003,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.85099999999999998,"15min":0.86199999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.98199999999999998,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0009999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.745},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.83499999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82899999999999996,"15min":0.83999999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98099999999999998,"5min":1.679,"15min":1.679},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.51800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.008,"5min":1.752,"15min":1.752},"last":0.85999999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.87,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.331,"15min":1.331},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.86799999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.96299999999999997,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.78000000000000003}]}]},{"osd":10,"up_from":69,"seq":296352743463,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1157453,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66400000000000003}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.29299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.252,"15min":0.252},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58399999999999996,"15min":0.57599999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81799999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80800000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72299999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.42599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.77500000000000002,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.752}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40699999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.73499999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.35899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83299999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66700000000000004,"15min":0.65700000000000003},"min":{"1min":0.505,"5min":0.434,"15min":0.434},"max":{"1min":0.96199999999999997,"5min":1.01,"15min":1.01},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67600000000000005,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80700000000000005,"5min":1.115,"15min":1.115},"last":0.80600000000000005}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89200000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.53400000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.88400000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:13:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.441},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54800000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74299999999999999,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89300000000000002,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.56000000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.125,"5min":1.534,"15min":1.534},"last":0.91300000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78300000000000003,"15min":0.78600000000000003},"min":{"1min":0.621,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.879,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.56599999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0840000000000001,"5min":1.649,"15min":1.649},"last":0.93799999999999994}]},{"osd":11,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.85399999999999998,"5min":1.04,"15min":1.04},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.98199999999999998}]}]},{"osd":9,"up_from":64,"seq":274877906988,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":507095,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.42599999999999999,"15min":0.44},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.51700000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.45500000000000002,"15min":0.48099999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.45700000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.35599999999999998}]},{"osd":1,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85399999999999998,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.63600000000000001,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69299999999999995,"5min":1.012,"15min":1.012},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.64600000000000002,"15min":0.67000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77300000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.54000000000000004,"15min":0.56999999999999995},"min":{"1min":0.34499999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.55400000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.373},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.56999999999999995,"15min":0.61399999999999999},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.54200000000000004,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.39700000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80200000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68899999999999995,"15min":0.71099999999999997},"min":{"1min":0.48499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.80600000000000005,"5min":1.165,"15min":1.165},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71799999999999997,"15min":0.74199999999999999},"min":{"1min":0.48299999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.79400000000000004,"5min":1.216,"15min":1.216},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.73399999999999999,"15min":0.76800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78200000000000003,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.79200000000000004,"15min":0.83599999999999997},"min":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.79000000000000004,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.77100000000000002,"15min":0.81699999999999995},"min":{"1min":0.48699999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.77000000000000002,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66700000000000004}]},{"osd":8,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.72499999999999998,"15min":0.76800000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.80100000000000005,"5min":1.337,"15min":1.337},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.81000000000000005,"15min":0.84799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88400000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.61799999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.81200000000000006,"15min":0.82999999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.96299999999999997,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.73199999999999998,"15min":0.75},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79500000000000004,"5min":1.024,"15min":1.024},"last":0.63200000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64200000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168624,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":503046,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56000000000000005,"15min":0.54600000000000004},"min":{"1min":0.38900000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58399999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82299999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59299999999999997,"15min":0.57699999999999996},"min":{"1min":0.48999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.876,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72599999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.441},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71499999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.496,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.505}]},{"osd":3,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.44900000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64500000000000002,"15min":0.63100000000000001},"min":{"1min":0.42499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.63700000000000001,"15min":0.61299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65800000000000003,"15min":0.63600000000000001},"min":{"1min":0.46400000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.88100000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.58999999999999997,"15min":0.56200000000000006},"min":{"1min":0.42199999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90200000000000002,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.50600000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":0.96999999999999997,"5min":0.999,"15min":0.999},"last":0.78900000000000003}]},{"osd":7,"last update":"Sun Oct 8 09:14:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.58499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":1.0129999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.70799999999999996,"15min":0.67100000000000004},"min":{"1min":0.55000000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.026,"5min":1.149,"15min":1.149},"last":0.94899999999999995}]},{"osd":9,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.77300000000000002,"15min":0.746},"min":{"1min":0.622,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0680000000000001,"5min":1.179,"15min":1.179},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.64200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0429999999999999,"5min":1.071,"15min":1.071},"last":0.878}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.754,"15min":0.72899999999999998},"min":{"1min":0.76200000000000001,"5min":0.501,"15min":0.501},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.69499999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.91500000000000004}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":1},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.93999999999999995}]}]},{"osd":7,"up_from":49,"seq":210453397557,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":1100332,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42599999999999999,"15min":0.41199999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.54700000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.45500000000000002,"15min":0.436},"min":{"1min":0.33500000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.60099999999999998,"5min":0.745,"15min":0.745},"last":0.42499999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83599999999999997,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.518,"15min":1.518},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58499999999999996,"15min":0.56899999999999995},"min":{"1min":0.497,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81599999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.627,"15min":0.60799999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.505,"15min":0.47699999999999998},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52400000000000002,"15min":0.51000000000000001},"min":{"1min":0.36199999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.587,"15min":1.587},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.49399999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89200000000000002,"5min":1.496,"15min":1.496},"last":0.85299999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.71399999999999997,"15min":0.68100000000000005},"min":{"1min":0.54300000000000004,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":3.6680000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.69699999999999995,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.48799999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69199999999999995,"15min":0.66700000000000004},"min":{"1min":0.52100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91700000000000004,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71599999999999997,"15min":0.69999999999999996},"min":{"1min":0.47899999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95799999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76000000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":0.88500000000000001,"15min":0.81299999999999994},"min":{"1min":0.59299999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":11.707000000000001,"5min":11.707000000000001,"15min":11.707000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.079,"5min":0.82699999999999996,"15min":0.78300000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":6.5899999999999999,"5min":6.5899999999999999,"15min":6.5899999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.748,"15min":0.746},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91700000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78100000000000003}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83399999999999996}]}]},{"osd":6,"up_from":44,"seq":188978561081,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":503130,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.54500000000000004,"15min":0.42299999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":16.849,"5min":16.849,"15min":16.849},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41599999999999998,"15min":0.38600000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.67000000000000004,"15min":0.57899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":9.093,"5min":9.093,"15min":9.093},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.65400000000000003,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":5.9980000000000002,"5min":5.9980000000000002,"15min":5.9980000000000002},"last":0.59599999999999997}]},{"osd":2,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61099999999999999,"15min":0.58499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77800000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.68500000000000005,"15min":0.59999999999999998},"min":{"1min":0.38500000000000001,"5min":0.37,"15min":0.37},"max":{"1min":9.0050000000000008,"5min":9.0050000000000008,"15min":9.0050000000000008},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.58399999999999996,"15min":0.47899999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":12.041,"5min":12.041,"15min":12.041},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":6.0270000000000001,"5min":6.0270000000000001,"15min":6.0270000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90500000000000003,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.623},{"interface":"front","average":{"1min":1.0680000000000001,"5min":0.745,"15min":0.63900000000000001},"min":{"1min":0.495,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.1500000000000004,"5min":9.1500000000000004,"15min":9.1500000000000004},"last":0.70099999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.70699999999999996,"15min":0.65700000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":5.8499999999999996,"5min":5.8499999999999996,"15min":5.8499999999999996},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.68999999999999995,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.84699999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":0.81299999999999994,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.1229999999999993,"5min":9.1229999999999993,"15min":9.1229999999999993},"last":0.74299999999999999}]},{"osd":8,"last update":"Sun Oct 8 09:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.83899999999999997,"15min":0.79200000000000004},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":9.0589999999999993,"5min":9.0589999999999993,"15min":9.0589999999999993},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.9160000000000004,"5min":5.9160000000000004,"15min":5.9160000000000004},"last":0.89200000000000002}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97299999999999998,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.998,"5min":0.83699999999999997,"15min":0.82299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":9.4209999999999994,"5min":9.4209999999999994,"15min":9.4209999999999994},"last":0.81299999999999994}]},{"osd":10,"last update":"Sun Oct 8 09:14:00 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.61299999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.25,"5min":0.96999999999999997,"15min":0.92400000000000004},"min":{"1min":0.63,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":9.1769999999999996,"5min":9.1769999999999996,"15min":9.1769999999999996},"last":0.86099999999999999}]},{"osd":11,"last update":"Sun Oct 8 09:14:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":5.9610000000000003,"5min":5.9610000000000003,"15min":5.9610000000000003},"last":0.95899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":567041,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50700000000000001,"15min":0.499},"min":{"1min":0.308,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91500000000000004,"5min":0.999,"15min":0.999},"last":0.44}]},{"osd":2,"last update":"Sun Oct 8 09:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.54300000000000004},"min":{"1min":0.45400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.60699999999999998,"15min":0.54200000000000004},"min":{"1min":0.49099999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.65900000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.50700000000000001},"min":{"1min":0.41299999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82799999999999996,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60199999999999998,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81499999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61199999999999999}]},{"osd":4,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.40000000000000002},"min":{"1min":0.36199999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.62,"5min":0.751,"15min":0.751},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.42099999999999999},"min":{"1min":0.41799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.76000000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Oct 8 09:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.629,"15min":0.60599999999999998},"min":{"1min":0.58499999999999996,"5min":0.377,"15min":0.377},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.66400000000000003,"15min":0.61899999999999999},"min":{"1min":0.59299999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.90900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80000000000000004}]},{"osd":6,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.64700000000000002,"15min":0.63400000000000001},"min":{"1min":0.50900000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.59299999999999997,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.53700000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.63900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.78000000000000003}]},{"osd":9,"last update":"Sun Oct 8 09:14:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.754,"15min":0.73199999999999998},"min":{"1min":0.57499999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.879},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.86199999999999999}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.73599999999999999,"15min":0.71399999999999997},"min":{"1min":0.72899999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.84199999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.752},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.91300000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":504586,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 09:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56399999999999995,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.52400000000000002},"min":{"1min":0.38100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.1819999999999999,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.63800000000000001,"15min":0.54100000000000004},"min":{"1min":0.45000000000000001,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1739999999999999,"5min":3.468,"15min":3.468},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.56399999999999995},"min":{"1min":0.47899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2190000000000001,"5min":3.4409999999999998,"15min":3.4409999999999998},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Oct 8 09:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.622,"15min":0.59299999999999997},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.48899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60799999999999998,"15min":0.57399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.58399999999999996},"min":{"1min":0.40600000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0920000000000001,"5min":1.095,"15min":1.095},"last":0.61099999999999999}]},{"osd":6,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.81499999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47099999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53800000000000003}]},{"osd":8,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.49399999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.55000000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.71499999999999997},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97399999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.621}]},{"osd":10,"last update":"Sun Oct 8 09:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.79400000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.4569999999999999,"5min":3.4569999999999999,"15min":3.4569999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.58499999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90700000000000003,"5min":1.008,"15min":1.008},"last":0.59399999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.86099999999999999,"15min":0.877},"min":{"1min":0.60199999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1459999999999999,"5min":3.427,"15min":3.427},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.63800000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68000000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":567534,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.72799999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52300000000000002,"15min":0.49099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69799999999999995,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.58699999999999997,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0049999999999999,"5min":1.071,"15min":1.071},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Oct 8 09:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.94499999999999995,"5min":1.758,"15min":1.758},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.622,"15min":0.56399999999999995},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":1.772,"15min":1.772},"last":0.72199999999999998}]},{"osd":4,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.68400000000000005}]},{"osd":5,"last update":"Sun Oct 8 09:14:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.48199999999999998,"15min":0.44700000000000001},"min":{"1min":0.314,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.83399999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.51000000000000001},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.65200000000000002,"15min":0.629},"min":{"1min":0.47499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.026,"5min":1.839,"15min":1.839},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.68799999999999994,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":1.492,"15min":1.492},"last":0.84199999999999997}]},{"osd":7,"last update":"Sun Oct 8 09:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.89100000000000001}]},{"osd":8,"last update":"Sun Oct 8 09:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.63500000000000001,"15min":0.59899999999999998},"min":{"1min":0.39600000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.113,"5min":1.127,"15min":1.127},"last":0.875},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.60999999999999999,"15min":0.58999999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91800000000000004,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.748}]},{"osd":9,"last update":"Sun Oct 8 09:14:39 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.96699999999999997,"5min":1.002,"15min":1.002},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.58099999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.0760000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.95199999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:14:04 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.72499999999999998,"15min":0.69699999999999995},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.877,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.96999999999999997}]},{"osd":11,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.92100000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":501976,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40300000000000002,"15min":0.379},"min":{"1min":0.314,"5min":0.217,"15min":0.217},"max":{"1min":0.55900000000000005,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.40999999999999998,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.53600000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.435}]},{"osd":1,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81499999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.45700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73699999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60699999999999998,"15min":0.58899999999999997},"min":{"1min":0.46400000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.77600000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.39900000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82199999999999995,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.64500000000000002,"15min":0.64800000000000002},"min":{"1min":0.49399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72399999999999998,"5min":1.494,"15min":1.494},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 09:14:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.61499999999999999,"15min":0.56100000000000005},"min":{"1min":0.48099999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79100000000000004,"5min":0.87,"15min":0.87},"last":0.60599999999999998}]},{"osd":6,"last update":"Sun Oct 8 09:14:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64100000000000001,"5min":1.01,"15min":1.01},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65900000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Oct 8 09:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70899999999999996,"15min":0.72799999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80200000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.55300000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.81599999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69299999999999995}]},{"osd":8,"last update":"Sun Oct 8 09:12:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001}]},{"osd":9,"last update":"Sun Oct 8 09:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.42599999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.022},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.73299999999999998},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86399999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.64100000000000001}]},{"osd":10,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.82099999999999995,"15min":0.83499999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92500000000000004,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.81799999999999995,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.98099999999999998,"5min":1.125,"15min":1.125},"last":0.81899999999999995}]},{"osd":11,"last update":"Sun Oct 8 09:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73899999999999999}]}]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":506673,"data_compressed":23575,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.499,"15min":0.47499999999999998},"min":{"1min":0.34000000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.36399999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78300000000000003,"5min":1.466,"15min":1.466},"last":0.433}]},{"osd":1,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.36799999999999999},"min":{"1min":0.30199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57999999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.39900000000000002,"15min":0.375},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.29999999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.56999999999999995,"15min":0.53600000000000003},"min":{"1min":0.498,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.52700000000000002},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.89300000000000002,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.52500000000000002},"min":{"1min":0.36899999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77100000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":4.7850000000000001}]},{"osd":5,"last update":"Sun Oct 8 09:14:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.57099999999999995},"min":{"1min":0.47899999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.77700000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.505},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.56000000000000005},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75900000000000001,"5min":1.429,"15min":1.429},"last":0.52500000000000002}]},{"osd":6,"last update":"Sun Oct 8 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63700000000000001,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86299999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83199999999999996,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Oct 8 09:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.78900000000000003,"5min":0.873,"15min":0.873},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.40999999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.82299999999999995,"5min":1.353,"15min":1.353},"last":0.47699999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:14:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69399999999999995,"15min":0.68200000000000005},"min":{"1min":0.51800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97699999999999998,"5min":1.373,"15min":1.373},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.63800000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.628}]},{"osd":9,"last update":"Sun Oct 8 09:14:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93600000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.51700000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.91300000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":4.8259999999999996}]},{"osd":10,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":0.75800000000000001},"min":{"1min":0.48699999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.746,"15min":0.74299999999999999},"min":{"1min":0.50900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.51600000000000001}]},{"osd":11,"last update":"Sun Oct 8 09:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70699999999999996}]}]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":1099161,"data_compressed":59640,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52000000000000002,"15min":0.504},"min":{"1min":0.33500000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.82799999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.52600000000000002,"15min":0.53600000000000003},"min":{"1min":0.32300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.68200000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56200000000000006,"15min":0.53300000000000003},"min":{"1min":0.37,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82899999999999996,"5min":0.998,"15min":0.998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57099999999999995,"15min":0.53400000000000003},"min":{"1min":0.44700000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.41599999999999998,"15min":0.40200000000000002},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.625},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.432,"15min":0.40400000000000003},"min":{"1min":0.32600000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.84199999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79100000000000004}]},{"osd":3,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81000000000000005,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.432,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77200000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.70299999999999996}]},{"osd":4,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.57099999999999995},"min":{"1min":0.51200000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.47799999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.77900000000000003}]},{"osd":6,"last update":"Sun Oct 8 09:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.61199999999999999},"min":{"1min":0.42799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.77900000000000003,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.748},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61399999999999999},"min":{"1min":0.48799999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76100000000000001,"5min":1.175,"15min":1.175},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64900000000000002,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.87},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.52400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90000000000000002,"5min":1.016,"15min":1.016},"last":0.85499999999999998}]},{"osd":8,"last update":"Sun Oct 8 09:14:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97899999999999998,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.83999999999999997}]},{"osd":9,"last update":"Sun Oct 8 09:14:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.56399999999999995,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91300000000000003,"5min":1.071,"15min":1.071},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.60499999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93200000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.93600000000000005}]},{"osd":10,"last update":"Sun Oct 8 09:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73099999999999998,"15min":0.71899999999999997},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93200000000000005,"5min":1.006,"15min":1.006},"last":0.88300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34426,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36065,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T09:15:03.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:15:03.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.221+0000 7f9a81091700 1 -- 172.21.15.49:0/2128078693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c106400 msgr2=0x7f9a7c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.221+0000 7f9a81091700 1 --2- 172.21.15.49:0/2128078693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c106400 0x7f9a7c1087f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400b0a0 tx=0x7f9a7400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 -- 172.21.15.49:0/2128078693 shutdown_connections 2023-10-08T09:15:03.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 --2- 172.21.15.49:0/2128078693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a7c1011a0 0x7f9a7c103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 --2- 172.21.15.49:0/2128078693 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c106400 0x7f9a7c1087f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 --2- 172.21.15.49:0/2128078693 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a7c103ad0 0x7f9a7c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 -- 172.21.15.49:0/2128078693 >> 172.21.15.49:0/2128078693 conn(0x7f9a7c0fab10 msgr2=0x7f9a7c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:03.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 -- 172.21.15.49:0/2128078693 shutdown_connections 2023-10-08T09:15:03.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 -- 172.21.15.49:0/2128078693 wait complete. 2023-10-08T09:15:03.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.222+0000 7f9a81091700 1 Processor -- start 2023-10-08T09:15:03.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a81091700 1 -- start start 2023-10-08T09:15:03.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a81091700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1011a0 0x7f9a7c196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a81091700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a7c103ad0 0x7f9a7c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a81091700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a7c106400 0x7f9a7c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a81091700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c19e160 con 0x7f9a7c1011a0 2023-10-08T09:15:03.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a81091700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c19e2a0 con 0x7f9a7c103ad0 2023-10-08T09:15:03.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a81091700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c19e430 con 0x7f9a7c106400 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1011a0 0x7f9a7c196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1011a0 0x7f9a7c196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50014/0 (socket says 172.21.15.49:50014) 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 -- 172.21.15.49:0/4181189898 learned_addr learned my addr 172.21.15.49:0/4181189898 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7b59e700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a7c106400 0x7f9a7c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7a59c700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a7c103ad0 0x7f9a7c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 -- 172.21.15.49:0/4181189898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a7c106400 msgr2=0x7f9a7c19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a7c106400 0x7f9a7c19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 -- 172.21.15.49:0/4181189898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a7c103ad0 msgr2=0x7f9a7c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a7c103ad0 0x7f9a7c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 -- 172.21.15.49:0/4181189898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74009d00 con 0x7f9a7c1011a0 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7a59c700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a7c103ad0 0x7f9a7c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.223+0000 7f9a7ad9d700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1011a0 0x7f9a7c196680 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c9b0 tx=0x7f9a6c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.224+0000 7f9a63fff700 1 -- 172.21.15.49:0/4181189898 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c007b00 con 0x7f9a7c1011a0 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.224+0000 7f9a63fff700 1 -- 172.21.15.49:0/4181189898 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a6c007c60 con 0x7f9a7c1011a0 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.224+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c19e710 con 0x7f9a7c1011a0 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.224+0000 7f9a63fff700 1 -- 172.21.15.49:0/4181189898 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c018660 con 0x7f9a7c1011a0 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.224+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c19ec70 con 0x7f9a7c1011a0 2023-10-08T09:15:03.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.225+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c0fc690 con 0x7f9a7c1011a0 2023-10-08T09:15:03.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.226+0000 7f9a63fff700 1 -- 172.21.15.49:0/4181189898 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9a6c018880 con 0x7f9a7c1011a0 2023-10-08T09:15:03.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.226+0000 7f9a63fff700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a640a1500 0x7f9a640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.226+0000 7f9a63fff700 1 -- 172.21.15.49:0/4181189898 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9a6c0cd2f0 con 0x7f9a7c1011a0 2023-10-08T09:15:03.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.226+0000 7f9a7a59c700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a640a1500 0x7f9a640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.227+0000 7f9a7a59c700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a640a1500 0x7f9a640a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005950 tx=0x7f9a700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:03.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.230+0000 7f9a63fff700 1 -- 172.21.15.49:0/4181189898 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9a6c096730 con 0x7f9a7c1011a0 2023-10-08T09:15:03.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.433+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a7c067450 con 0x7f9a7c1011a0 2023-10-08T09:15:03.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.434+0000 7f9a63fff700 1 -- 172.21.15.49:0/4181189898 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f9a6c099d50 con 0x7f9a7c1011a0 2023-10-08T09:15:03.435 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:03.435 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:15:03.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a640a1500 msgr2=0x7f9a640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a640a1500 0x7f9a640a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005950 tx=0x7f9a700058e0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1011a0 msgr2=0x7f9a7c196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1011a0 0x7f9a7c196680 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c9b0 tx=0x7f9a6c00cd70 comp rx=0 tx=0).stop 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 shutdown_connections 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9a640a1500 0x7f9a640a39c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a7c106400 0x7f9a7c19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9a7c1011a0 0x7f9a7c196680 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 --2- 172.21.15.49:0/4181189898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a7c103ad0 0x7f9a7c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 >> 172.21.15.49:0/4181189898 conn(0x7f9a7c0fab10 msgr2=0x7f9a7c101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 shutdown_connections 2023-10-08T09:15:03.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.437+0000 7f9a81091700 1 -- 172.21.15.49:0/4181189898 wait complete. 2023-10-08T09:15:03.450 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:15:03.451 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:15:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 -- 172.21.15.49:0/3153834883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 msgr2=0x7f66980ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 --2- 172.21.15.49:0/3153834883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66980ffac0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f668c0099c0 tx=0x7f668c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 -- 172.21.15.49:0/3153834883 shutdown_connections 2023-10-08T09:15:03.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 --2- 172.21.15.49:0/3153834883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6698101230 0x7f6698107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 --2- 172.21.15.49:0/3153834883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66980ffac0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 --2- 172.21.15.49:0/3153834883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698100890 0x7f6698100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 -- 172.21.15.49:0/3153834883 >> 172.21.15.49:0/3153834883 conn(0x7f66980fac10 msgr2=0x7f66980fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 -- 172.21.15.49:0/3153834883 shutdown_connections 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 -- 172.21.15.49:0/3153834883 wait complete. 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 Processor -- start 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 -- start start 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66981966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698100890 0x7f6698196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.581+0000 7f669eabe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6698101230 0x7f669819dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669eabe700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819e220 con 0x7f66980ff6a0 2023-10-08T09:15:03.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669eabe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819e360 con 0x7f6698100890 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669eabe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819e4f0 con 0x7f6698101230 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66981966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66981966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50028/0 (socket says 172.21.15.49:50028) 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 -- 172.21.15.49:0/3500210808 learned_addr learned my addr 172.21.15.49:0/3500210808 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f6697fff700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698100890 0x7f6698196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669d05b700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6698101230 0x7f669819dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 -- 172.21.15.49:0/3500210808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6698101230 msgr2=0x7f669819dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6698101230 0x7f669819dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 -- 172.21.15.49:0/3500210808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698100890 msgr2=0x7f6698196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698100890 0x7f6698196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 -- 172.21.15.49:0/3500210808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0095f0 con 0x7f66980ff6a0 2023-10-08T09:15:03.585 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669c85a700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66981966f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f668c006010 tx=0x7f668c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:03.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f6695ffb700 1 -- 172.21.15.49:0/3500210808 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c01d070 con 0x7f66980ff6a0 2023-10-08T09:15:03.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f6695ffb700 1 -- 172.21.15.49:0/3500210808 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668c004010 con 0x7f66980ff6a0 2023-10-08T09:15:03.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669819e770 con 0x7f66980ff6a0 2023-10-08T09:15:03.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.582+0000 7f6695ffb700 1 -- 172.21.15.49:0/3500210808 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c022cd0 con 0x7f66980ff6a0 2023-10-08T09:15:03.586 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.583+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669819ec20 con 0x7f66980ff6a0 2023-10-08T09:15:03.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.584+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6698067530 con 0x7f66980ff6a0 2023-10-08T09:15:03.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.585+0000 7f6695ffb700 1 -- 172.21.15.49:0/3500210808 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f668c022470 con 0x7f66980ff6a0 2023-10-08T09:15:03.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.585+0000 7f6695ffb700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66880a1480 0x7f66880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.585+0000 7f6695ffb700 1 -- 172.21.15.49:0/3500210808 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f668c0ceb30 con 0x7f66980ff6a0 2023-10-08T09:15:03.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.585+0000 7f6697fff700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66880a1480 0x7f66880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.587 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.586+0000 7f6697fff700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66880a1480 0x7f66880a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6684005950 tx=0x7f668400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:03.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.589+0000 7f6695ffb700 1 -- 172.21.15.49:0/3500210808 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f668c097ef0 con 0x7f66980ff6a0 2023-10-08T09:15:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:03 smithi184 ceph-mon[103675]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2023-10-08T09:15:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:03 smithi184 ceph-mon[103675]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:03 smithi184 ceph-mon[103675]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:03 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4181189898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:03.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.789+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f669810b050 con 0x7f66980ff6a0 2023-10-08T09:15:03.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.789+0000 7f6695ffb700 1 -- 172.21.15.49:0/3500210808 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f668c027020 con 0x7f66980ff6a0 2023-10-08T09:15:03.790 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:15:03.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66880a1480 msgr2=0x7f66880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66880a1480 0x7f66880a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6684005950 tx=0x7f668400b410 comp rx=0 tx=0).stop 2023-10-08T09:15:03.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 msgr2=0x7f66981966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66981966f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f668c006010 tx=0x7f668c005740 comp rx=0 tx=0).stop 2023-10-08T09:15:03.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 shutdown_connections 2023-10-08T09:15:03.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66880a1480 0x7f66880a3940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6698101230 0x7f669819dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66980ff6a0 0x7f66981966f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 --2- 172.21.15.49:0/3500210808 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6698100890 0x7f6698196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 >> 172.21.15.49:0/3500210808 conn(0x7f66980fac10 msgr2=0x7f66980fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:03.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 shutdown_connections 2023-10-08T09:15:03.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.791+0000 7f669eabe700 1 -- 172.21.15.49:0/3500210808 wait complete. 2023-10-08T09:15:03.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:03 smithi132 ceph-mon[103389]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2023-10-08T09:15:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:03 smithi132 ceph-mon[103389]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:03 smithi132 ceph-mon[103389]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:03.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:03 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4181189898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:03.803 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:03 smithi049 ceph-mon[99485]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2023-10-08T09:15:03.803 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:03 smithi049 ceph-mon[99485]: from='client.15696 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:03.803 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:03 smithi049 ceph-mon[99485]: from='client.15702 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:03.804 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:03 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4181189898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:03.805 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:15:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.933+0000 7ff19d053700 1 -- 172.21.15.49:0/3354860992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 msgr2=0x7ff198107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.933+0000 7ff19d053700 1 --2- 172.21.15.49:0/3354860992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff198107af0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff19000b0d0 tx=0x7ff19000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.933+0000 7ff19d053700 1 -- 172.21.15.49:0/3354860992 shutdown_connections 2023-10-08T09:15:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.933+0000 7ff19d053700 1 --2- 172.21.15.49:0/3354860992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1980ff6b0 0x7ff1980ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.934+0000 7ff19d053700 1 --2- 172.21.15.49:0/3354860992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff198107af0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.934+0000 7ff19d053700 1 --2- 172.21.15.49:0/3354860992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1981008a0 0x7ff198100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.934+0000 7ff19d053700 1 -- 172.21.15.49:0/3354860992 >> 172.21.15.49:0/3354860992 conn(0x7ff1980fac40 msgr2=0x7ff1980fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.934+0000 7ff19d053700 1 -- 172.21.15.49:0/3354860992 shutdown_connections 2023-10-08T09:15:03.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.934+0000 7ff19d053700 1 -- 172.21.15.49:0/3354860992 wait complete. 2023-10-08T09:15:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.934+0000 7ff19d053700 1 Processor -- start 2023-10-08T09:15:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.934+0000 7ff19d053700 1 -- start start 2023-10-08T09:15:03.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19d053700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1980ff6b0 0x7ff198196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19d053700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1981008a0 0x7ff198196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19d053700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff19819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19d053700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19819e370 con 0x7ff198101240 2023-10-08T09:15:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19d053700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19819e4b0 con 0x7ff1981008a0 2023-10-08T09:15:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19d053700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19819e640 con 0x7ff1980ff6b0 2023-10-08T09:15:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff19819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff19819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50046/0 (socket says 172.21.15.49:50046) 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 -- 172.21.15.49:0/1529963837 learned_addr learned my addr 172.21.15.49:0/1529963837 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19659c700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1981008a0 0x7ff198196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff196d9d700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1980ff6b0 0x7ff198196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 -- 172.21.15.49:0/1529963837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1980ff6b0 msgr2=0x7ff198196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1980ff6b0 0x7ff198196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 -- 172.21.15.49:0/1529963837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1981008a0 msgr2=0x7ff198196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1981008a0 0x7ff198196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 -- 172.21.15.49:0/1529963837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009d00 con 0x7ff198101240 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff196d9d700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1980ff6b0 0x7ff198196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19759e700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff19819dda0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff190007bd0 tx=0x7ff190009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:03.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff17ffff700 1 -- 172.21.15.49:0/1529963837 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000e040 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19819e8c0 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.935+0000 7ff17ffff700 1 -- 172.21.15.49:0/1529963837 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff190004e60 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.936+0000 7ff17ffff700 1 -- 172.21.15.49:0/1529963837 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190005640 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.936+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19819ece0 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.937+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff198067980 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.938+0000 7ff17ffff700 1 -- 172.21.15.49:0/1529963837 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff1900057a0 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.938+0000 7ff17ffff700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff1800a1480 0x7ff1800a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.938+0000 7ff17ffff700 1 -- 172.21.15.49:0/1529963837 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff1900ce6a0 con 0x7ff198101240 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.938+0000 7ff196d9d700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff1800a1480 0x7ff1800a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:03.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.939+0000 7ff196d9d700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff1800a1480 0x7ff1800a3940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff198100700 tx=0x7ff188008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:03.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:03.943+0000 7ff17ffff700 1 -- 172.21.15.49:0/1529963837 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff190097a60 con 0x7ff198101240 2023-10-08T09:15:04.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.145+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff19819f040 con 0x7ff198101240 2023-10-08T09:15:04.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.146+0000 7ff17ffff700 1 -- 172.21.15.49:0/1529963837 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff19009b080 con 0x7ff198101240 2023-10-08T09:15:04.147 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:15:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff1800a1480 msgr2=0x7ff1800a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff1800a1480 0x7ff1800a3940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff198100700 tx=0x7ff188008040 comp rx=0 tx=0).stop 2023-10-08T09:15:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 msgr2=0x7ff19819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff19819dda0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff190007bd0 tx=0x7ff190009570 comp rx=0 tx=0).stop 2023-10-08T09:15:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 shutdown_connections 2023-10-08T09:15:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff1800a1480 0x7ff1800a3940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1980ff6b0 0x7ff198196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff198101240 0x7ff19819dda0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 --2- 172.21.15.49:0/1529963837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff1981008a0 0x7ff198196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 >> 172.21.15.49:0/1529963837 conn(0x7ff1980fac40 msgr2=0x7ff198068c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 shutdown_connections 2023-10-08T09:15:04.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.148+0000 7ff19d053700 1 -- 172.21.15.49:0/1529963837 wait complete. 2023-10-08T09:15:04.161 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:15:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2600801401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd4101230 msgr2=0x7f4cd4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2600801401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd4101230 0x7f4cd4107ae0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00b0d0 tx=0x7f4ccc00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2600801401 shutdown_connections 2023-10-08T09:15:04.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2600801401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd4101230 0x7f4cd4107ae0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2600801401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd40ff6a0 0x7f4cd40ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2600801401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd4100890 0x7f4cd4100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2600801401 >> 172.21.15.49:0/2600801401 conn(0x7f4cd40fac10 msgr2=0x7f4cd40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2600801401 shutdown_connections 2023-10-08T09:15:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.290+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2600801401 wait complete. 2023-10-08T09:15:04.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 Processor -- start 2023-10-08T09:15:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 -- start start 2023-10-08T09:15:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd40ff6a0 0x7f4cd4196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd4100890 0x7f4cd4196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd4101230 0x7f4cd419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd419e350 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd419e490 con 0x7f4cd4100890 2023-10-08T09:15:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd89a1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd419e620 con 0x7f4cd4101230 2023-10-08T09:15:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd40ff6a0 0x7f4cd4196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd40ff6a0 0x7f4cd4196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50056/0 (socket says 172.21.15.49:50056) 2023-10-08T09:15:04.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd259c700 1 -- 172.21.15.49:0/2697755448 learned_addr learned my addr 172.21.15.49:0/2697755448 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.291+0000 7f4cd2d9d700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd4101230 0x7f4cd419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd1d9b700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd4100890 0x7f4cd4196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd259c700 1 -- 172.21.15.49:0/2697755448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd4101230 msgr2=0x7f4cd419dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd259c700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd4101230 0x7f4cd419dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd259c700 1 -- 172.21.15.49:0/2697755448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd4100890 msgr2=0x7f4cd4196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd259c700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd4100890 0x7f4cd4196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd259c700 1 -- 172.21.15.49:0/2697755448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc009d00 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd259c700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd40ff6a0 0x7f4cd4196820 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800d980 tx=0x7f4cc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cc37fe700 1 -- 172.21.15.49:0/2697755448 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8004020 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd419e900 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cc37fe700 1 -- 172.21.15.49:0/2697755448 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc800f690 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cc37fe700 1 -- 172.21.15.49:0/2697755448 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8003a60 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.292+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd419ee60 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.293+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd4067530 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.294+0000 7f4cc37fe700 1 -- 172.21.15.49:0/2697755448 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4cc800fb00 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.295+0000 7f4cc37fe700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cbc0a1480 0x7f4cbc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.295+0000 7f4cd1d9b700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cbc0a1480 0x7f4cbc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.295+0000 7f4cc37fe700 1 -- 172.21.15.49:0/2697755448 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4cc80ccf60 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.295+0000 7f4cd1d9b700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cbc0a1480 0x7f4cbc0a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4005950 tx=0x7f4cc400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:04.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.299+0000 7f4cc37fe700 1 -- 172.21.15.49:0/2697755448 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4cc8096290 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.504+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cd410aec0 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.504+0000 7f4cc37fe700 1 -- 172.21.15.49:0/2697755448 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cc80998b0 con 0x7f4cd40ff6a0 2023-10-08T09:15:04.505 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:15:04.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cbc0a1480 msgr2=0x7f4cbc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cbc0a1480 0x7f4cbc0a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4005950 tx=0x7f4cc400b410 comp rx=0 tx=0).stop 2023-10-08T09:15:04.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd40ff6a0 msgr2=0x7f4cd4196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd40ff6a0 0x7f4cd4196820 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800d980 tx=0x7f4cc800dc90 comp rx=0 tx=0).stop 2023-10-08T09:15:04.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 shutdown_connections 2023-10-08T09:15:04.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4cbc0a1480 0x7f4cbc0a3940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4cd4101230 0x7f4cd419dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4cd40ff6a0 0x7f4cd4196820 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 --2- 172.21.15.49:0/2697755448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cd4100890 0x7f4cd4196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.506+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 >> 172.21.15.49:0/2697755448 conn(0x7f4cd40fac10 msgr2=0x7f4cd40fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:04.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.507+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 shutdown_connections 2023-10-08T09:15:04.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.507+0000 7f4cd89a1700 1 -- 172.21.15.49:0/2697755448 wait complete. 2023-10-08T09:15:04.519 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:15:04.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 -- 172.21.15.49:0/2733264118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058101230 msgr2=0x7f0058103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 --2- 172.21.15.49:0/2733264118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058101230 0x7f0058103620 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0048009990 tx=0x7f0048009ca0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 -- 172.21.15.49:0/2733264118 shutdown_connections 2023-10-08T09:15:04.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 --2- 172.21.15.49:0/2733264118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0058106490 0x7f0058108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 --2- 172.21.15.49:0/2733264118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058101230 0x7f0058103620 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 --2- 172.21.15.49:0/2733264118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0058103b60 0x7f0058105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 -- 172.21.15.49:0/2733264118 >> 172.21.15.49:0/2733264118 conn(0x7f00580fac40 msgr2=0x7f00580fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 -- 172.21.15.49:0/2733264118 shutdown_connections 2023-10-08T09:15:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.648+0000 7f005d867700 1 -- 172.21.15.49:0/2733264118 wait complete. 2023-10-08T09:15:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 Processor -- start 2023-10-08T09:15:04.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 -- start start 2023-10-08T09:15:04.652 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0058101230 0x7f0058100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058103b60 0x7f00580ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0058106490 0x7f00580ff720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00580ffec0 con 0x7f0058103b60 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058100030 con 0x7f0058106490 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f005d867700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00581a0710 con 0x7f0058101230 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f00567fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058103b60 0x7f00580ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f00567fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058103b60 0x7f00580ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50072/0 (socket says 172.21.15.49:50072) 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.649+0000 7f00567fc700 1 -- 172.21.15.49:0/145554286 learned_addr learned my addr 172.21.15.49:0/145554286 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f00577fe700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0058106490 0x7f00580ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f0056ffd700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0058101230 0x7f0058100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f00567fc700 1 -- 172.21.15.49:0/145554286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0058101230 msgr2=0x7f0058100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f00567fc700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0058101230 0x7f0058100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f00567fc700 1 -- 172.21.15.49:0/145554286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0058106490 msgr2=0x7f00580ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f00567fc700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0058106490 0x7f00580ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f00567fc700 1 -- 172.21.15.49:0/145554286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480095f0 con 0x7f0058103b60 2023-10-08T09:15:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f0056ffd700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0058101230 0x7f0058100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f00567fc700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058103b60 0x7f00580ff1e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f004000d950 tx=0x7f004000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f005c865700 1 -- 172.21.15.49:0/145554286 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400098e0 con 0x7f0058103b60 2023-10-08T09:15:04.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f005c865700 1 -- 172.21.15.49:0/145554286 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0040010460 con 0x7f0058103b60 2023-10-08T09:15:04.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00581a09a0 con 0x7f0058103b60 2023-10-08T09:15:04.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f005c865700 1 -- 172.21.15.49:0/145554286 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004000f620 con 0x7f0058103b60 2023-10-08T09:15:04.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.650+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00581a0eb0 con 0x7f0058103b60 2023-10-08T09:15:04.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.651+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00580fc7c0 con 0x7f0058103b60 2023-10-08T09:15:04.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.652+0000 7f005c865700 1 -- 172.21.15.49:0/145554286 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f00400108d0 con 0x7f0058103b60 2023-10-08T09:15:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.653+0000 7f005c865700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00440a1480 0x7f00440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.653+0000 7f005c865700 1 -- 172.21.15.49:0/145554286 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f00400ccea0 con 0x7f0058103b60 2023-10-08T09:15:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.653+0000 7f0056ffd700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00440a1480 0x7f00440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.653+0000 7f0056ffd700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00440a1480 0x7f00440a3940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f004800b5c0 tx=0x7f0048005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.656+0000 7f005c865700 1 -- 172.21.15.49:0/145554286 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0040096170 con 0x7f0058103b60 2023-10-08T09:15:04.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:04 smithi184 ceph-mon[103675]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:04.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:04 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3500210808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:15:04.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:04 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1529963837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:15:04.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:04 smithi132 ceph-mon[103389]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:04.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:04 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3500210808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:15:04.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:04 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1529963837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:15:04.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.857+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f005810bff0 con 0x7f0058103b60 2023-10-08T09:15:04.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.857+0000 7f005c865700 1 -- 172.21.15.49:0/145554286 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f00400160a0 con 0x7f0058103b60 2023-10-08T09:15:04.858 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:15:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00440a1480 msgr2=0x7f00440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00440a1480 0x7f00440a3940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f004800b5c0 tx=0x7f0048005fb0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058103b60 msgr2=0x7f00580ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058103b60 0x7f00580ff1e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f004000d950 tx=0x7f004000dc60 comp rx=0 tx=0).stop 2023-10-08T09:15:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 shutdown_connections 2023-10-08T09:15:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f00440a1480 0x7f00440a3940 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0058101230 0x7f0058100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0058106490 0x7f00580ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 --2- 172.21.15.49:0/145554286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0058103b60 0x7f00580ff1e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 >> 172.21.15.49:0/145554286 conn(0x7f00580fac40 msgr2=0x7f0058101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 shutdown_connections 2023-10-08T09:15:04.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.859+0000 7f005d867700 1 -- 172.21.15.49:0/145554286 wait complete. 2023-10-08T09:15:04.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:04 smithi049 ceph-mon[99485]: from='client.15708 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-08T09:15:04.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:04 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3500210808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:15:04.871 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:04 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1529963837' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:15:04.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:15:04.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 -- 172.21.15.49:0/2226890959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 msgr2=0x7f9bf8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:04.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2226890959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8101d10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0099c0 tx=0x7f9bec009cd0 comp rx=0 tx=0).stop 2023-10-08T09:15:04.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 -- 172.21.15.49:0/2226890959 shutdown_connections 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2226890959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf8102250 0x7f9bf8108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2226890959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf81006c0 0x7f9bf8100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2226890959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8101d10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 -- 172.21.15.49:0/2226890959 >> 172.21.15.49:0/2226890959 conn(0x7f9bf80fbc50 msgr2=0x7f9bf80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 -- 172.21.15.49:0/2226890959 shutdown_connections 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.998+0000 7f9bfe635700 1 -- 172.21.15.49:0/2226890959 wait complete. 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 Processor -- start 2023-10-08T09:15:05.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 -- start start 2023-10-08T09:15:05.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf81006c0 0x7f9bf8196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf8102250 0x7f9bf819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819e140 con 0x7f9bf81018b0 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819e280 con 0x7f9bf8102250 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bfe635700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819e410 con 0x7f9bf81006c0 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:04.999+0000 7f9bf77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50086/0 (socket says 172.21.15.49:50086) 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 -- 172.21.15.49:0/2270300137 learned_addr learned my addr 172.21.15.49:0/2270300137 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bfcbd2700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf8102250 0x7f9bf819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf7fff700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf81006c0 0x7f9bf8196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 -- 172.21.15.49:0/2270300137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf81006c0 msgr2=0x7f9bf8196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf81006c0 0x7f9bf8196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 -- 172.21.15.49:0/2270300137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf8102250 msgr2=0x7f9bf819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf8102250 0x7f9bf819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 -- 172.21.15.49:0/2270300137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0095f0 con 0x7f9bf81018b0 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf77fe700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8196b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009fd0 tx=0x7f9bec005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf57fa700 1 -- 172.21.15.49:0/2270300137 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec01d070 con 0x7f9bf81018b0 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf819e690 con 0x7f9bf81018b0 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf57fa700 1 -- 172.21.15.49:0/2270300137 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec0059a0 con 0x7f9bf81018b0 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bf57fa700 1 -- 172.21.15.49:0/2270300137 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0173f0 con 0x7f9bf81018b0 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.000+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf819eb40 con 0x7f9bf81018b0 2023-10-08T09:15:05.004 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.001+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8107ae0 con 0x7f9bf81018b0 2023-10-08T09:15:05.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.002+0000 7f9bf57fa700 1 -- 172.21.15.49:0/2270300137 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9bec0226e0 con 0x7f9bf81018b0 2023-10-08T09:15:05.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.003+0000 7f9bf57fa700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9be00a1480 0x7f9be00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.003+0000 7f9bf7fff700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9be00a1480 0x7f9be00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.003+0000 7f9bf57fa700 1 -- 172.21.15.49:0/2270300137 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9bec0ce0e0 con 0x7f9bf81018b0 2023-10-08T09:15:05.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.003+0000 7f9bf7fff700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9be00a1480 0x7f9be00a3940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9be8005950 tx=0x7f9be80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.007+0000 7f9bf57fa700 1 -- 172.21.15.49:0/2270300137 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9bec05d100 con 0x7f9bf81018b0 2023-10-08T09:15:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.255+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bf8066bc0 con 0x7f9bf81018b0 2023-10-08T09:15:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.256+0000 7f9bf57fa700 1 -- 172.21.15.49:0/2270300137 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11378 (secure 0 0 0) 0x7f9bec097440 con 0x7f9bf81018b0 2023-10-08T09:15:05.257 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:05.257 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:13:33.418895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:15:05.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.257+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9be00a1480 msgr2=0x7f9be00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.257+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9be00a1480 0x7f9be00a3940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9be8005950 tx=0x7f9be80058e0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.257+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 msgr2=0x7f9bf8196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.257+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8196b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009fd0 tx=0x7f9bec005f10 comp rx=0 tx=0).stop 2023-10-08T09:15:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 shutdown_connections 2023-10-08T09:15:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9be00a1480 0x7f9be00a3940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bf81006c0 0x7f9bf8196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf8102250 0x7f9bf819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 --2- 172.21.15.49:0/2270300137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9bf81018b0 0x7f9bf8196b50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 >> 172.21.15.49:0/2270300137 conn(0x7f9bf80fbc50 msgr2=0x7f9bf8104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 shutdown_connections 2023-10-08T09:15:05.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.258+0000 7f9bfe635700 1 -- 172.21.15.49:0/2270300137 wait complete. 2023-10-08T09:15:05.261 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 28 2023-10-08T09:15:05.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:15:05.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 -- 172.21.15.49:0/2617931006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1184102250 msgr2=0x7f1184108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 --2- 172.21.15.49:0/2617931006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1184102250 0x7f1184108b00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b0d0 tx=0x7f117c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 -- 172.21.15.49:0/2617931006 shutdown_connections 2023-10-08T09:15:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 --2- 172.21.15.49:0/2617931006 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11841006c0 0x7f1184100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 --2- 172.21.15.49:0/2617931006 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1184102250 0x7f1184108b00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 --2- 172.21.15.49:0/2617931006 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11841018b0 0x7f1184101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 -- 172.21.15.49:0/2617931006 >> 172.21.15.49:0/2617931006 conn(0x7f11840fbc50 msgr2=0x7f11840fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 -- 172.21.15.49:0/2617931006 shutdown_connections 2023-10-08T09:15:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.396+0000 7f118a773700 1 -- 172.21.15.49:0/2617931006 wait complete. 2023-10-08T09:15:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 Processor -- start 2023-10-08T09:15:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 -- start start 2023-10-08T09:15:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11841006c0 0x7f1184074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11841018b0 0x7f1184072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1184102250 0x7f1184073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840738c0 con 0x7f11841006c0 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1184073a30 con 0x7f1184102250 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f118a773700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11841a0780 con 0x7f11841018b0 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f1183fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11841006c0 0x7f1184074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f1183fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11841006c0 0x7f1184074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50098/0 (socket says 172.21.15.49:50098) 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f1183fff700 1 -- 172.21.15.49:0/3166697930 learned_addr learned my addr 172.21.15.49:0/3166697930 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f1188d10700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1184102250 0x7f1184073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.397+0000 7f11837fe700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11841018b0 0x7f1184072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f1183fff700 1 -- 172.21.15.49:0/3166697930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11841018b0 msgr2=0x7f1184072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f1183fff700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11841018b0 0x7f1184072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f1183fff700 1 -- 172.21.15.49:0/3166697930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1184102250 msgr2=0x7f1184073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f1183fff700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1184102250 0x7f1184073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f1183fff700 1 -- 172.21.15.49:0/3166697930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009d00 con 0x7f11841006c0 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f1183fff700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11841006c0 0x7f1184074550 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f117400d980 tx=0x7f117400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f11817fa700 1 -- 172.21.15.49:0/3166697930 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400b8f0 con 0x7f11841006c0 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11841a0a10 con 0x7f11841006c0 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f11817fa700 1 -- 172.21.15.49:0/3166697930 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1174007620 con 0x7f11841006c0 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f11817fa700 1 -- 172.21.15.49:0/3166697930 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400f3c0 con 0x7f11841006c0 2023-10-08T09:15:05.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.398+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11841a0f20 con 0x7f11841006c0 2023-10-08T09:15:05.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.399+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1184107af0 con 0x7f11841006c0 2023-10-08T09:15:05.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.400+0000 7f11817fa700 1 -- 172.21.15.49:0/3166697930 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f11740105f0 con 0x7f11841006c0 2023-10-08T09:15:05.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.401+0000 7f11817fa700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f116c0a1480 0x7f116c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.401+0000 7f11817fa700 1 -- 172.21.15.49:0/3166697930 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f11740cd7e0 con 0x7f11841006c0 2023-10-08T09:15:05.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.401+0000 7f11837fe700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f116c0a1480 0x7f116c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.401+0000 7f11837fe700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f116c0a1480 0x7f116c0a3940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1178005950 tx=0x7f11780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.404+0000 7f11817fa700 1 -- 172.21.15.49:0/3166697930 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1174096b10 con 0x7f11841006c0 2023-10-08T09:15:05.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.613+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f118410bee0 con 0x7f11841006c0 2023-10-08T09:15:05.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.614+0000 7f11817fa700 1 -- 172.21.15.49:0/3166697930 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f117409a130 con 0x7f11841006c0 2023-10-08T09:15:05.615 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:05.615 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f116c0a1480 msgr2=0x7f116c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f116c0a1480 0x7f116c0a3940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1178005950 tx=0x7f11780058e0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11841006c0 msgr2=0x7f1184074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11841006c0 0x7f1184074550 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f117400d980 tx=0x7f117400dc90 comp rx=0 tx=0).stop 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 shutdown_connections 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f116c0a1480 0x7f116c0a3940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1184102250 0x7f1184073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11841006c0 0x7f1184074550 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 --2- 172.21.15.49:0/3166697930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f11841018b0 0x7f1184072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 >> 172.21.15.49:0/3166697930 conn(0x7f11840fbc50 msgr2=0x7f1184104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 shutdown_connections 2023-10-08T09:15:05.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.617+0000 7f118a773700 1 -- 172.21.15.49:0/3166697930 wait complete. 2023-10-08T09:15:05.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:05 smithi049 ceph-mon[99485]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-10-08T09:15:05.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:05 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2697755448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:15:05.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:05 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/145554286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:05.629 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:05 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2270300137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:05.630 DEBUG:tasks.fwd_scrub:Yielding 2023-10-08T09:15:05.631 INFO:teuthology.run_tasks:Running task background_exec... 2023-10-08T09:15:05.641 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:15:05.641 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-10-08T09:15:05.642 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-08T09:15:05.642 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T09:15:05.642 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T09:15:05.645 INFO:teuthology.run_tasks:Running task exec... 2023-10-08T09:15:05.655 INFO:teuthology.task.exec:Executing custom commands... 2023-10-08T09:15:05.655 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:15:05.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-10-08T09:15:05.695 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:05.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:05 smithi184 ceph-mon[103675]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-10-08T09:15:05.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:05 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2697755448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:15:05.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:05 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/145554286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:05.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:05 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2270300137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:05.756 INFO:teuthology.orchestra.run.smithi049.stdout:Filesystem Size Used Avail Use% Mounted on 2023-10-08T09:15:05.756 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:3300,172.21.15.132:3300,172.21.15.184:3300:/volumes/_nogroup/sv_1/485a435c-3760-47ed-ad82-bcf59bb1ae2b 1.1T 332M 1.1T 1% /tmp/cephfs 2023-10-08T09:15:05.758 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-10-08T09:15:05.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 -- 172.21.15.49:0/2929218778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0204101260 msgr2=0x7f0204107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 --2- 172.21.15.49:0/2929218778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0204101260 0x7f0204107b10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00b0d0 tx=0x7f01fc00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 -- 172.21.15.49:0/2929218778 shutdown_connections 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 --2- 172.21.15.49:0/2929218778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02040ff6d0 0x7f02040ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 --2- 172.21.15.49:0/2929218778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0204101260 0x7f0204107b10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 --2- 172.21.15.49:0/2929218778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02041008c0 0x7f0204100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 -- 172.21.15.49:0/2929218778 >> 172.21.15.49:0/2929218778 conn(0x7f02040fac40 msgr2=0x7f02040fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 -- 172.21.15.49:0/2929218778 shutdown_connections 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 -- 172.21.15.49:0/2929218778 wait complete. 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.790+0000 7f020b5c7700 1 Processor -- start 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f020b5c7700 1 -- start start 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f020b5c7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02040ff6d0 0x7f0204195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f020b5c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02041008c0 0x7f0204195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f020b5c7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0204101260 0x7f020419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f020b5c7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020419d160 con 0x7f02040ff6d0 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f020b5c7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020419d2a0 con 0x7f02041008c0 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f020b5c7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020419d430 con 0x7f0204101260 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02040ff6d0 0x7f0204195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02040ff6d0 0x7f0204195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50116/0 (socket says 172.21.15.49:50116) 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 -- 172.21.15.49:0/566110073 learned_addr learned my addr 172.21.15.49:0/566110073 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0208b62700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02041008c0 0x7f0204195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209b64700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0204101260 0x7f020419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 -- 172.21.15.49:0/566110073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0204101260 msgr2=0x7f020419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0204101260 0x7f020419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 -- 172.21.15.49:0/566110073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02041008c0 msgr2=0x7f0204195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02041008c0 0x7f0204195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 -- 172.21.15.49:0/566110073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc009d00 con 0x7f02040ff6d0 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0208b62700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02041008c0 0x7f0204195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209b64700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0204101260 0x7f020419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.791+0000 7f0209363700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02040ff6d0 0x7f0204195630 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f02000098f0 tx=0x7f0200009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.792+0000 7f01fa7fc700 1 -- 172.21.15.49:0/566110073 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200012070 con 0x7f02040ff6d0 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.792+0000 7f01fa7fc700 1 -- 172.21.15.49:0/566110073 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02000044e0 con 0x7f02040ff6d0 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.792+0000 7f01fa7fc700 1 -- 172.21.15.49:0/566110073 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200007d10 con 0x7f02040ff6d0 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.792+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0204068a50 con 0x7f02040ff6d0 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.792+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0204068f60 con 0x7f02040ff6d0 2023-10-08T09:15:05.797 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.794+0000 7f01fa7fc700 1 -- 172.21.15.49:0/566110073 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0200016030 con 0x7f02040ff6d0 2023-10-08T09:15:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:05 smithi132 ceph-mon[103389]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-10-08T09:15:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:05 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2697755448' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:15:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:05 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/145554286' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:15:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:05 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2270300137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.795+0000 7f01fa7fc700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01f00a15d0 0x7f01f00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.795+0000 7f0208b62700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01f00a15d0 0x7f01f00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.795+0000 7f01fa7fc700 1 -- 172.21.15.49:0/566110073 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f02000ce1f0 con 0x7f02040ff6d0 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f0208b62700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01f00a15d0 0x7f01f00a3a90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005fd0 tx=0x7f01f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 Processor -- start 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 -- start start 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940217500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a9402182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940218d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94017f4e0 con 0x55a940217a40 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940170ca0 con 0x55a940218880 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f02177b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94016e5c0 con 0x55a9402170e0 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f0208b62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a9402182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f0208b62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a9402182f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50126/0 (socket says 172.21.15.49:50126) 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.796+0000 7f0208b62700 1 -- 172.21.15.49:0/761496828 learned_addr learned my addr 172.21.15.49:0/761496828 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0209b64700 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940218d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0209363700 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940217500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0208b62700 1 -- 172.21.15.49:0/761496828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 msgr2=0x55a940217500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0208b62700 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940217500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0208b62700 1 -- 172.21.15.49:0/761496828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 msgr2=0x55a940218d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0208b62700 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940218d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0208b62700 1 -- 172.21.15.49:0/761496828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a93ff94e60 con 0x55a940217a40 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f0208b62700 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a9402182f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f01f400a5c0 tx=0x7f01f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaee8ea3fbf625a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f01ef7fe700 1 -- 172.21.15.49:0/761496828 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f4010da0 con 0x55a940217a40 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f01ef7fe700 1 -- 172.21.15.49:0/761496828 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01f400eec0 con 0x55a940217a40 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.797+0000 7f01ef7fe700 1 -- 172.21.15.49:0/761496828 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400f720 con 0x55a940217a40 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 -- 172.21.15.49:0/761496828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 msgr2=0x55a9402182f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a9402182f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f01f400a5c0 tx=0x7f01f4011000 comp rx=0 tx=0).stop 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 -- 172.21.15.49:0/761496828 shutdown_connections 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940217500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940218d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 --2- 172.21.15.49:0/761496828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a9402182f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 -- 172.21.15.49:0/761496828 >> 172.21.15.49:0/761496828 conn(0x55a94018b2e0 msgr2=0x55a940211e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 -- 172.21.15.49:0/761496828 shutdown_connections 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 -- 172.21.15.49:0/761496828 wait complete. 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.798+0000 7f02177b0b80 1 -- start start 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 1 Processor -- start 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940220e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a940221370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a9402283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940169eb0 con 0x55a940217a40 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940106790 con 0x55a940218880 2023-10-08T09:15:05.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940106430 con 0x55a9402170e0 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f02177b0b80 10 client.0 _ll_register_callbacks cb 0x55a93ff63880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f0208b62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a940221370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f0208b62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a940221370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50132/0 (socket says 172.21.15.49:50132) 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f0208b62700 1 -- 172.21.15.49:0/333045440 learned_addr learned my addr 172.21.15.49:0/333045440 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f0209363700 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940220e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.799+0000 7f0209b64700 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a9402283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f0208b62700 1 -- 172.21.15.49:0/333045440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 msgr2=0x55a940220e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f0208b62700 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940220e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f0208b62700 1 -- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 msgr2=0x55a9402283b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f0208b62700 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a9402283b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f0208b62700 1 -- 172.21.15.49:0/333045440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f400eba0 con 0x55a940217a40 2023-10-08T09:15:05.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f0208b62700 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a940221370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f01f401d980 tx=0x7f01f401d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b3b46ec3490c3b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f01ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f01ecff9700 1 -- 172.21.15.49:0/333045440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f402c070 con 0x55a940217a40 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f01ecff9700 1 -- 172.21.15.49:0/333045440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f01f400f6b0 con 0x55a940217a40 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a940162e90 con 0x55a940217a40 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f01ecff9700 1 -- 172.21.15.49:0/333045440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01f400eec0 con 0x55a940217a40 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f02177b0b80 10 client.15762 fetch_fsmap learned FSMap version 28 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a93ff7dce0 con 0x55a940217a40 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.800+0000 7f01ecff9700 1 -- 172.21.15.49:0/333045440 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a940162e90 con 0x55a940217a40 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f01ecff9700 1 -- 172.21.15.49:0/333045440 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f01f4026790 con 0x55a940217a40 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f02177b0b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:15:05.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f02177b0b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f02177b0b80 20 client.15762 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f02177b0b80 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a94022cc80 0x55a94022f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f02177b0b80 4 client.15762 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a940162bb0 con 0x55a94022cc80 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f0209363700 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a94022cc80 0x55a94022f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f0209363700 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a94022cc80 0x55a94022f0a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d11d73488608d2 server_cookie=ae2661bafb1db0e1 in_seq=0 out_seq=0 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.801+0000 7f01ecff9700 10 client.15762 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f01ecff9700 1 -- 172.21.15.49:0/333045440 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a940162bb0 con 0x55a94022cc80 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f01ecff9700 10 client.15762 handle_command_reply: tid=0 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 client.15762 shutdown 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 msgr2=0x55a940221370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a940221370 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f01f401d980 tx=0x7f01f401d9b0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 shutdown_connections 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a94022cc80 0x55a94022f0a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9402170e0 0x55a940220e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a9402283b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 --2- 172.21.15.49:0/333045440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a940217a40 0x55a940221370 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 >> 172.21.15.49:0/333045440 conn(0x55a94018b2e0 msgr2=0x55a940211550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f01ecff9700 0 client.15762 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 shutdown_connections 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.802+0000 7f02177b0b80 1 -- 172.21.15.49:0/333045440 wait complete. 2023-10-08T09:15:05.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.803+0000 7f02177b0b80 20 client.15762 trim_cache size 0 max 16384 2023-10-08T09:15:05.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-10-08T09:15:05.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.825+0000 7f02177b0b80 1 Processor -- start 2023-10-08T09:15:05.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.825+0000 7f02177b0b80 1 -- start start 2023-10-08T09:15:05.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a94021ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940228370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a94021b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f02177b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a93ffea9a0 con 0x55a9402170e0 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f02177b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94020eb90 con 0x55a940218880 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f02177b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a94021ea70 con 0x55a940217a40 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a94021ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a94021ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50138/0 (socket says 172.21.15.49:50138) 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 -- 172.21.15.49:0/3142100751 learned_addr learned my addr 172.21.15.49:0/3142100751 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:05.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0208b62700 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940228370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209b64700 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a94021b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 -- 172.21.15.49:0/3142100751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 msgr2=0x55a940228370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940228370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 -- 172.21.15.49:0/3142100751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 msgr2=0x55a94021b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a94021b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 -- 172.21.15.49:0/3142100751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a94020e320 con 0x55a9402170e0 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0208b62700 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940228370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209b64700 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a94021b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:05.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f0209363700 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a94021ed70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f02000074a0 tx=0x7f0200000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98f72ec2b495b75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f01effff700 1 -- 172.21.15.49:0/3142100751 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020009f120 con 0x55a9402170e0 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f01effff700 1 -- 172.21.15.49:0/3142100751 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f020000d1c0 con 0x55a9402170e0 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.826+0000 7f01effff700 1 -- 172.21.15.49:0/3142100751 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200021e30 con 0x55a9402170e0 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 -- 172.21.15.49:0/3142100751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 msgr2=0x55a94021ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a94021ed70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f02000074a0 tx=0x7f0200000bc0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 -- 172.21.15.49:0/3142100751 shutdown_connections 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a94021b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a94021ed70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 --2- 172.21.15.49:0/3142100751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940228370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 -- 172.21.15.49:0/3142100751 >> 172.21.15.49:0/3142100751 conn(0x55a94018b2e0 msgr2=0x55a940220340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 -- 172.21.15.49:0/3142100751 shutdown_connections 2023-10-08T09:15:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.827+0000 7f02177b0b80 1 -- 172.21.15.49:0/3142100751 wait complete. 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 -- start start 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 Processor -- start 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a940224f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940225480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940280a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9402284a0 con 0x55a9402170e0 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a940228610 con 0x55a940218880 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9402287d0 con 0x55a940217a40 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f02177b0b80 10 client.0 _ll_register_callbacks cb 0x7f01fc00b450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f0209363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a940224f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f0209363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a940224f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50144/0 (socket says 172.21.15.49:50144) 2023-10-08T09:15:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f0209363700 1 -- 172.21.15.49:0/513089072 learned_addr learned my addr 172.21.15.49:0/513089072 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f0209b64700 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940280a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f0208b62700 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940225480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f0209363700 1 -- 172.21.15.49:0/513089072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 msgr2=0x55a940225480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.828+0000 7f0209363700 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940225480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f0209363700 1 -- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 msgr2=0x55a940280a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f0209363700 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940280a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f0209363700 1 -- 172.21.15.49:0/513089072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0204068f60 con 0x55a9402170e0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f0209363700 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a940224f40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f02000cba10 tx=0x7f02000a36e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bb32a6bb097bea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f01ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f01ecff9700 1 -- 172.21.15.49:0/513089072 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020009f330 con 0x55a9402170e0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f01ecff9700 1 -- 172.21.15.49:0/513089072 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f0200099320 con 0x55a9402170e0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a940131f30 con 0x55a9402170e0 2023-10-08T09:15:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f01ecff9700 1 -- 172.21.15.49:0/513089072 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0200099580 con 0x55a9402170e0 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f02177b0b80 10 client.15768 fetch_fsmap learned FSMap version 28 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a93ffbb2d0 con 0x55a9402170e0 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f01ecff9700 1 -- 172.21.15.49:0/513089072 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02000a9080 con 0x55a9402170e0 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f01ecff9700 1 -- 172.21.15.49:0/513089072 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f02000a31e0 con 0x55a9402170e0 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f02177b0b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f02177b0b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.829+0000 7f02177b0b80 20 client.15768 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.830+0000 7f02177b0b80 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a940282530 0x55a940284950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.830+0000 7f02177b0b80 4 client.15768 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.830+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a93ffb5ff0 con 0x55a940282530 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.830+0000 7f0208b62700 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a940282530 0x55a940284950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.830+0000 7f01ecff9700 10 client.15768 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.830+0000 7f0208b62700 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a940282530 0x55a940284950 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c1aecd266fa9804 server_cookie=ab54de3c7443efec in_seq=0 out_seq=0 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.831+0000 7f01ecff9700 1 -- 172.21.15.49:0/513089072 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a93ffb5ff0 con 0x55a940282530 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.831+0000 7f01ecff9700 10 client.15768 handle_command_reply: tid=0 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.831+0000 7f02177b0b80 1 client.15768 shutdown 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 msgr2=0x55a940224f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a940224f40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f02000cba10 tx=0x7f02000a36e0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 shutdown_connections 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a940282530 0x55a940284950 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a940218880 0x55a940280a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9402170e0 0x55a940224f40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f01ecff9700 0 client.15768 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 --2- 172.21.15.49:0/513089072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a940217a40 0x55a940225480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 >> 172.21.15.49:0/513089072 conn(0x55a94018b2e0 msgr2=0x55a94021eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 shutdown_connections 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 1 -- 172.21.15.49:0/513089072 wait complete. 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f02177b0b80 20 client.15768 trim_cache size 0 max 16384 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "3b2f5057-f48c-4beb-9b35-5b69a2af46f6", 2023-10-08T09:15:05.839 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2023-10-08T09:15:05.840 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:15:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01f00a15d0 msgr2=0x7f01f00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01f00a15d0 0x7f01f00a3a90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005fd0 tx=0x7f01f4005e30 comp rx=0 tx=0).stop 2023-10-08T09:15:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02040ff6d0 msgr2=0x7f0204195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:05.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02040ff6d0 0x7f0204195630 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f02000098f0 tx=0x7f0200009cb0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 shutdown_connections 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01f00a15d0 0x7f01f00a3a90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0204101260 0x7f020419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f02040ff6d0 0x7f0204195630 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.832+0000 7f020b5c7700 1 --2- 172.21.15.49:0/566110073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02041008c0 0x7f0204195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.833+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 >> 172.21.15.49:0/566110073 conn(0x7f02040fac40 msgr2=0x7f0204103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.833+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 shutdown_connections 2023-10-08T09:15:05.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:05.833+0000 7f020b5c7700 1 -- 172.21.15.49:0/566110073 wait complete. 2023-10-08T09:15:05.846 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:15:05.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-10-08T09:15:06.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 -- 172.21.15.49:0/2181015474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60200ff050 msgr2=0x7f60200ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 --2- 172.21.15.49:0/2181015474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60200ff050 0x7f60200ff470 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6010009990 tx=0x7f6010009ca0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 -- 172.21.15.49:0/2181015474 shutdown_connections 2023-10-08T09:15:06.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 --2- 172.21.15.49:0/2181015474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6020100370 0x7f60201007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 --2- 172.21.15.49:0/2181015474 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60200ff050 0x7f60200ff470 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 --2- 172.21.15.49:0/2181015474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200ff9b0 0x7f60200ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 -- 172.21.15.49:0/2181015474 >> 172.21.15.49:0/2181015474 conn(0x7f60200fac10 msgr2=0x7f60200fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 -- 172.21.15.49:0/2181015474 shutdown_connections 2023-10-08T09:15:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.039+0000 7f60264a9700 1 -- 172.21.15.49:0/2181015474 wait complete. 2023-10-08T09:15:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 Processor -- start 2023-10-08T09:15:06.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 -- start start 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200ff050 0x7f6020198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60200ff9b0 0x7f6020198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6020100370 0x7f602019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60201a0450 con 0x7f6020100370 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60201a0590 con 0x7f60200ff050 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.040+0000 7f60264a9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60201a0720 con 0x7f60200ff9b0 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6020100370 0x7f602019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6020100370 0x7f602019fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50160/0 (socket says 172.21.15.49:50160) 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 -- 172.21.15.49:0/881523101 learned_addr learned my addr 172.21.15.49:0/881523101 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f601ffff700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200ff050 0x7f6020198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f601f7fe700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60200ff9b0 0x7f6020198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 -- 172.21.15.49:0/881523101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60200ff9b0 msgr2=0x7f6020198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60200ff9b0 0x7f6020198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 -- 172.21.15.49:0/881523101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200ff050 msgr2=0x7f6020198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200ff050 0x7f6020198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 -- 172.21.15.49:0/881523101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100095f0 con 0x7f6020100370 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f6024a46700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6020100370 0x7f602019fe80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f601800ada0 tx=0x7f601800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f601d7fa700 1 -- 172.21.15.49:0/881523101 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601800f070 con 0x7f6020100370 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f601d7fa700 1 -- 172.21.15.49:0/881523101 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6018007b30 con 0x7f6020100370 2023-10-08T09:15:06.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60201a0a00 con 0x7f6020100370 2023-10-08T09:15:06.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f601d7fa700 1 -- 172.21.15.49:0/881523101 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60180053e0 con 0x7f6020100370 2023-10-08T09:15:06.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.041+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60201a0f10 con 0x7f6020100370 2023-10-08T09:15:06.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.043+0000 7f601d7fa700 1 -- 172.21.15.49:0/881523101 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f601800c980 con 0x7f6020100370 2023-10-08T09:15:06.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.044+0000 7f601d7fa700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60080a1480 0x7f60080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.044+0000 7f601ffff700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60080a1480 0x7f60080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.045+0000 7f601d7fa700 1 -- 172.21.15.49:0/881523101 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f60180cc5c0 con 0x7f6020100370 2023-10-08T09:15:06.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.045+0000 7f601ffff700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60080a1480 0x7f60080a3940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f601000b5c0 tx=0x7f6010003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:06.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.045+0000 7f6032692b80 1 Processor -- start 2023-10-08T09:15:06.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.045+0000 7f6032692b80 1 -- start start 2023-10-08T09:15:06.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f6032692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x56458082c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f6032692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x56458082d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f6032692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458082dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f6032692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580794460 con 0x56458082c9c0 2023-10-08T09:15:06.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f6032692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580785c20 con 0x56458082d800 2023-10-08T09:15:06.051 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f6032692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580783540 con 0x56458082c060 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x56458082d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x56458082d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50168/0 (socket says 172.21.15.49:50168) 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 -- 172.21.15.49:0/1812076201 learned_addr learned my addr 172.21.15.49:0/1812076201 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601ffff700 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x56458082c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f6024a46700 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458082dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 -- 172.21.15.49:0/1812076201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 msgr2=0x56458082c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x56458082c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 -- 172.21.15.49:0/1812076201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 msgr2=0x56458082dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458082dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 -- 172.21.15.49:0/1812076201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645805a9de0 con 0x56458082c9c0 2023-10-08T09:15:06.052 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.046+0000 7f601f7fe700 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x56458082d270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f601400bd70 tx=0x7f601400f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2286a318606113a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f60067fc700 1 -- 172.21.15.49:0/1812076201 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601400fda0 con 0x56458082c9c0 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f60067fc700 1 -- 172.21.15.49:0/1812076201 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f601401a920 con 0x56458082c9c0 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f60067fc700 1 -- 172.21.15.49:0/1812076201 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014011730 con 0x56458082c9c0 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 -- 172.21.15.49:0/1812076201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 msgr2=0x56458082d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x56458082d270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f601400bd70 tx=0x7f601400f640 comp rx=0 tx=0).stop 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 -- 172.21.15.49:0/1812076201 shutdown_connections 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x56458082c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458082dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 --2- 172.21.15.49:0/1812076201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x56458082d270 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 -- 172.21.15.49:0/1812076201 >> 172.21.15.49:0/1812076201 conn(0x5645807a0260 msgr2=0x564580826e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.047+0000 7f6032692b80 1 -- 172.21.15.49:0/1812076201 shutdown_connections 2023-10-08T09:15:06.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 -- 172.21.15.49:0/1812076201 wait complete. 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 -- start start 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 Processor -- start 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580835e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580836340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458083d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458077ee30 con 0x56458082c9c0 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458071b710 con 0x56458082d800 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458071b3b0 con 0x56458082c060 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f6032692b80 10 client.0 _ll_register_callbacks cb 0x564580578800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.048+0000 7f601f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580836340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580836340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50174/0 (socket says 172.21.15.49:50174) 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 -- 172.21.15.49:0/2501657099 learned_addr learned my addr 172.21.15.49:0/2501657099 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601ffff700 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580835e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f6024a46700 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458083d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 -- 172.21.15.49:0/2501657099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 msgr2=0x564580835e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580835e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 -- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 msgr2=0x56458083d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458083d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 -- 172.21.15.49:0/2501657099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601400b9a0 con 0x56458082c9c0 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f601f7fe700 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580836340 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f601400bf00 tx=0x7f601400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601ef0f19e3ee526 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f5fff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2501657099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014020070 con 0x56458082c9c0 2023-10-08T09:15:06.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2501657099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f601401aa90 con 0x56458082c9c0 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564580746eb0 con 0x56458082c9c0 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2501657099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f601401ac50 con 0x56458082c9c0 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f6032692b80 10 client.15780 fetch_fsmap learned FSMap version 28 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.049+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564580592c60 con 0x56458082c9c0 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2501657099 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564580746eb0 con 0x56458082c9c0 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2501657099 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6014004980 con 0x56458082c9c0 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f6032692b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f6032692b80 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f6032692b80 20 client.15780 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f6032692b80 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580841b60 0x564580843f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f6032692b80 4 client.15780 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:06.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564580777e10 con 0x564580841b60 2023-10-08T09:15:06.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.050+0000 7f601ffff700 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580841b60 0x564580843f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f5fff7fe700 10 client.15780 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:06.057 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f601ffff700 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580841b60 0x564580843f80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8670629a5f2628c server_cookie=83bf30f7eb5520b9 in_seq=0 out_seq=0 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2501657099 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564580777e10 con 0x564580841b60 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f5fff7fe700 10 client.15780 handle_command_reply: tid=0 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 client.15780 shutdown 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 msgr2=0x564580836340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580836340 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f601400bf00 tx=0x7f601400b910 comp rx=0 tx=0).stop 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 shutdown_connections 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580841b60 0x564580843f80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580835e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x56458083d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f5fff7fe700 0 client.15780 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:06.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 --2- 172.21.15.49:0/2501657099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580836340 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.051+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 >> 172.21.15.49:0/2501657099 conn(0x5645807a0260 msgr2=0x5645808264d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:06.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.052+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 shutdown_connections 2023-10-08T09:15:06.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.052+0000 7f6032692b80 1 -- 172.21.15.49:0/2501657099 wait complete. 2023-10-08T09:15:06.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.052+0000 7f6032692b80 20 client.15780 trim_cache size 0 max 16384 2023-10-08T09:15:06.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 Processor -- start 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 -- start start 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c060 0x56458083da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082c9c0 0x564580835310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082d800 0x564580848600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645805ff920 con 0x56458082c060 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645808339c0 con 0x56458082c9c0 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.068+0000 7f6032692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458083d700 con 0x56458082d800 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c060 0x56458083da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c060 0x56458083da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50180/0 (socket says 172.21.15.49:50180) 2023-10-08T09:15:06.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 -- 172.21.15.49:0/134208553 learned_addr learned my addr 172.21.15.49:0/134208553 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601f7fe700 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082c9c0 0x564580835310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f6024a46700 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082d800 0x564580848600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 -- 172.21.15.49:0/134208553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082d800 msgr2=0x564580848600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082d800 0x564580848600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 -- 172.21.15.49:0/134208553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082c9c0 msgr2=0x564580835310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082c9c0 0x564580835310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 -- 172.21.15.49:0/134208553 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645805d0250 con 0x56458082c060 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601f7fe700 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082c9c0 0x564580835310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f6024a46700 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082d800 0x564580848600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f601ffff700 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c060 0x56458083da30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f601000b940 tx=0x7f6010005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b8b2062c7f4c3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f6006ffd700 1 -- 172.21.15.49:0/134208553 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601002a040 con 0x56458082c060 2023-10-08T09:15:06.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f6006ffd700 1 -- 172.21.15.49:0/134208553 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f601002b040 con 0x56458082c060 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.069+0000 7f6006ffd700 1 -- 172.21.15.49:0/134208553 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000f460 con 0x56458082c060 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 -- 172.21.15.49:0/134208553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c060 msgr2=0x56458083da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c060 0x56458083da30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f601000b940 tx=0x7f6010005000 comp rx=0 tx=0).stop 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 -- 172.21.15.49:0/134208553 shutdown_connections 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082d800 0x564580848600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c060 0x56458083da30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 --2- 172.21.15.49:0/134208553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082c9c0 0x564580835310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 -- 172.21.15.49:0/134208553 >> 172.21.15.49:0/134208553 conn(0x5645807a0260 msgr2=0x564580822f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 -- 172.21.15.49:0/134208553 shutdown_connections 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 -- 172.21.15.49:0/134208553 wait complete. 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.070+0000 7f6032692b80 1 -- start start 2023-10-08T09:15:06.073 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 1 Processor -- start 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580848440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580848980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x564580846c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580847360 con 0x56458082c9c0 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645808474d0 con 0x56458082d800 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564580847690 con 0x56458082c060 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6032692b80 10 client.0 _ll_register_callbacks cb 0x7f6010005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580848980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580848980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50188/0 (socket says 172.21.15.49:50188) 2023-10-08T09:15:06.077 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 -- 172.21.15.49:0/2904299158 learned_addr learned my addr 172.21.15.49:0/2904299158 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601ffff700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580848440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6024a46700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x564580846c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 -- 172.21.15.49:0/2904299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 msgr2=0x564580848440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580848440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 -- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 msgr2=0x564580846c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x564580846c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 -- 172.21.15.49:0/2904299158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602010b720 con 0x56458082c9c0 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f6024a46700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x564580846c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f601f7fe700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580848980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6014011480 tx=0x7f601401a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a3846e1a0a0f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.071+0000 7f5fff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2904299158 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6014020070 con 0x56458082c9c0 2023-10-08T09:15:06.078 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2904299158 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6014000e40 con 0x56458082c9c0 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f601401a920 con 0x56458082c9c0 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2904299158 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6014005c20 con 0x56458082c9c0 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 10 client.15786 fetch_fsmap learned FSMap version 28 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5645805a82b0 con 0x56458082c9c0 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2904299158 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f601401a920 con 0x56458082c9c0 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2904299158 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f60140042c0 con 0x56458082c9c0 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 20 client.15786 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580896ad0 0x564580898ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 4 client.15786 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:15:06.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.072+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564580777b30 con 0x564580896ad0 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f601ffff700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580896ad0 0x564580898ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f5fff7fe700 10 client.15786 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f601ffff700 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580896ad0 0x564580898ef0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=934594e02d4e0b5b server_cookie=28f29abefb512583 in_seq=0 out_seq=0 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f5fff7fe700 1 -- 172.21.15.49:0/2904299158 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564580777b30 con 0x564580896ad0 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f5fff7fe700 10 client.15786 handle_command_reply: tid=0 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 client.15786 shutdown 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 msgr2=0x564580848980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580848980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6014011480 tx=0x7f601401a8b0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 shutdown_connections 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564580896ad0 0x564580898ef0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56458082c060 0x564580848440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f5fff7fe700 0 client.15786 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56458082d800 0x564580846c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 --2- 172.21.15.49:0/2904299158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56458082c9c0 0x564580848980 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 >> 172.21.15.49:0/2904299158 conn(0x5645807a0260 msgr2=0x564580830720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.073+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 shutdown_connections 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f6032692b80 1 -- 172.21.15.49:0/2904299158 wait complete. 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f6032692b80 20 client.15786 trim_cache size 0 max 16384 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:15:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:15:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "3b2f5057-f48c-4beb-9b35-5b69a2af46f6": { 2023-10-08T09:15:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:15:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "3b2f5057-f48c-4beb-9b35-5b69a2af46f6", 2023-10-08T09:15:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:15:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:15:06.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:15:06.082 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60080a1480 msgr2=0x7f60080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60080a1480 0x7f60080a3940 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f601000b5c0 tx=0x7f6010003790 comp rx=0 tx=0).stop 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6020100370 msgr2=0x7f602019fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6020100370 0x7f602019fe80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f601800ada0 tx=0x7f601800c330 comp rx=0 tx=0).stop 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 shutdown_connections 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60080a1480 0x7f60080a3940 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6020100370 0x7f602019fe80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60200ff050 0x7f6020198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 --2- 172.21.15.49:0/881523101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60200ff9b0 0x7f6020198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:06.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 >> 172.21.15.49:0/881523101 conn(0x7f60200fac10 msgr2=0x7f60200fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:06.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 shutdown_connections 2023-10-08T09:15:06.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:06.074+0000 7f60264a9700 1 -- 172.21.15.49:0/881523101 wait complete. 2023-10-08T09:15:06.087 INFO:tasks.cephfs.filesystem:scrub status for tag:3b2f5057-f48c-4beb-9b35-5b69a2af46f6 - {'path': '/', 'tag': '3b2f5057-f48c-4beb-9b35-5b69a2af46f6', 'options': 'recursive,force'} 2023-10-08T09:15:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:06 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3166697930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:06 smithi132 ceph-mon[103389]: scrub queued for path: / 2023-10-08T09:15:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:06 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:06 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:06.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:06 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:06.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:06 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3166697930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:06.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:06 smithi049 ceph-mon[99485]: scrub queued for path: / 2023-10-08T09:15:06.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:06 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:06.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:06 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:06.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:06 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:06 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3166697930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:15:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:06 smithi184 ceph-mon[103675]: scrub queued for path: / 2023-10-08T09:15:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:06 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:06 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:06.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:06 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:07.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:07 smithi132 ceph-mon[103389]: pgmap v335: 129 pgs: 129 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 12 op/s 2023-10-08T09:15:07.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:07 smithi049 ceph-mon[99485]: pgmap v335: 129 pgs: 129 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 12 op/s 2023-10-08T09:15:07.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:07 smithi184 ceph-mon[103675]: pgmap v335: 129 pgs: 129 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 12 op/s 2023-10-08T09:15:08.254 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-10-08T09:15:08.300 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-10-08T09:15:08.303 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-10-08T09:15:08.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-08T09:15:08.468 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2023-10-08T09:15:08.468 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:08.468 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:15:08.468 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 .. 2023-10-08T09:15:08.469 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 data 2023-10-08T09:15:08.469 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:08.469 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2023-10-08T09:15:08.469 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:08.469 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:08.469 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 .. 2023-10-08T09:15:08.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-10-08T09:15:08.562 INFO:teuthology.orchestra.run.smithi049.stdout:[Service] 2023-10-08T09:15:08.562 INFO:teuthology.orchestra.run.smithi049.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-10-08T09:15:08.562 INFO:teuthology.orchestra.run.smithi049.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-10-08T09:15:08.564 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-10-08T09:15:08.660 INFO:teuthology.orchestra.run.smithi049.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-10-08T09:15:09.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:09 smithi132 ceph-mon[103389]: pgmap v336: 129 pgs: 129 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-08T09:15:09.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:09 smithi049 ceph-mon[99485]: pgmap v336: 129 pgs: 129 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-08T09:15:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:09 smithi184 ceph-mon[103675]: pgmap v336: 129 pgs: 129 active+clean; 647 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-08T09:15:11.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:11 smithi132 ceph-mon[103389]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-10-08T09:15:11.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:11 smithi049 ceph-mon[99485]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-10-08T09:15:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:11 smithi184 ceph-mon[103675]: pgmap v337: 129 pgs: 129 active+clean; 1.4 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 68 KiB/s wr, 13 op/s 2023-10-08T09:15:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:13 smithi132 ceph-mon[103389]: pgmap v338: 129 pgs: 129 active+clean; 5.9 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-10-08T09:15:13.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:13 smithi049 ceph-mon[99485]: pgmap v338: 129 pgs: 129 active+clean; 5.9 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-10-08T09:15:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:13 smithi184 ceph-mon[103675]: pgmap v338: 129 pgs: 129 active+clean; 5.9 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 454 KiB/s wr, 15 op/s 2023-10-08T09:15:15.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:15 smithi132 ceph-mon[103389]: pgmap v339: 129 pgs: 129 active+clean; 14 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 MiB/s wr, 31 op/s 2023-10-08T09:15:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:15 smithi049 ceph-mon[99485]: pgmap v339: 129 pgs: 129 active+clean; 14 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 MiB/s wr, 31 op/s 2023-10-08T09:15:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:15 smithi184 ceph-mon[103675]: pgmap v339: 129 pgs: 129 active+clean; 14 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.1 MiB/s wr, 31 op/s 2023-10-08T09:15:16.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:16 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:15:16.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:16 smithi132 ceph-mon[103389]: scrub summary: idle 2023-10-08T09:15:16.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:16 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:15:16.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:16 smithi049 ceph-mon[99485]: scrub summary: idle 2023-10-08T09:15:16.965 INFO:teuthology.orchestra.run.smithi049.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-10-08T09:15:16.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-08T09:15:16.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:16 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:15:16.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:16 smithi184 ceph-mon[103675]: scrub summary: idle 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 .. 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 data 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2023-10-08T09:15:17.037 INFO:teuthology.orchestra.run.smithi049.stdout:total 30 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 . 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 .. 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 base 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 8 09:15 global 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 log 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_commit_ts 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_dynshmem 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 8 09:15 pg_hba.conf 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 8 09:15 pg_ident.conf 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 pg_logical 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 pg_multixact 2023-10-08T09:15:17.038 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 pg_notify 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_replslot 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_serial 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_snapshots 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_stat 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_stat_tmp 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 pg_subtrans 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_tblspc 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_twophase 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 pg_wal 2023-10-08T09:15:17.039 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 pg_xact 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 8 09:15 postgresql.auto.conf 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 8 09:15 postgresql.conf 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base: 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 . 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 1 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 12346 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 12347 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.040 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:total 7185 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 . 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 .. 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 112 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 113 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 8 09:15 12182 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12182_fsm 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12182_vm 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12184 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12186 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187 2023-10-08T09:15:17.041 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12187_fsm 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187_vm 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12189 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12191 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12192_fsm 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192_vm 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12194 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12196 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12197_fsm 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197_vm 2023-10-08T09:15:17.042 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12199 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12201 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12202_fsm 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202_vm 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12204 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12206 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12207_fsm 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207_vm 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12209 2023-10-08T09:15:17.043 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12211 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12212 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12214 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12216 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 8 09:15 1247 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1247_fsm 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1247_vm 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 8 09:15 1249 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1249_fsm 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1249_vm 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 8 09:15 1255 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1255_fsm 2023-10-08T09:15:17.044 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1255_vm 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 8 09:15 1259 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1259_fsm 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1259_vm 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417_vm 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418_vm 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 174 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 175 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2187 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224 2023-10-08T09:15:17.045 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224_vm 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328_vm 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336_vm 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2337 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2600 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2600_fsm 2023-10-08T09:15:17.046 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2600_vm 2023-10-08T09:15:17.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601 2023-10-08T09:15:17.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2601_fsm 2023-10-08T09:15:17.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601_vm 2023-10-08T09:15:17.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2602 2023-10-08T09:15:17.047 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2602_fsm 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2602_vm 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2603 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2603_fsm 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2603_vm 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604_vm 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2605 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2605_fsm 2023-10-08T09:15:17.048 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2605_vm 2023-10-08T09:15:17.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606 2023-10-08T09:15:17.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2606_fsm 2023-10-08T09:15:17.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606_vm 2023-10-08T09:15:17.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607 2023-10-08T09:15:17.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607_fsm 2023-10-08T09:15:17.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2607_vm 2023-10-08T09:15:17.049 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 8 09:15 2608 2023-10-08T09:15:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2608_fsm 2023-10-08T09:15:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2608_vm 2023-10-08T09:15:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 8 09:15 2609 2023-10-08T09:15:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2609_fsm 2023-10-08T09:15:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2609_vm 2023-10-08T09:15:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610 2023-10-08T09:15:17.050 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610_fsm 2023-10-08T09:15:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2610_vm 2023-10-08T09:15:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611 2023-10-08T09:15:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611_vm 2023-10-08T09:15:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612 2023-10-08T09:15:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2612_fsm 2023-10-08T09:15:17.051 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612_vm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613_vm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2615_fsm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615_vm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616_fsm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2616_vm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 8 09:15 2617 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2617_fsm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2617_vm 2023-10-08T09:15:17.052 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 8 09:15 2618 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2618_fsm 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2618_vm 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 8 09:15 2619 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2619_fsm 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2619_vm 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620_vm 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2650 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2651 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2652 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2653 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2654 2023-10-08T09:15:17.053 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2655 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2656 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2657 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 8 09:15 2658 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2659 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2660 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2661 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2662 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2663 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2664 2023-10-08T09:15:17.054 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2665 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2666 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2667 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2668 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2669 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2670 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 8 09:15 2673 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 8 09:15 2674 2023-10-08T09:15:17.055 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 8 09:15 2675 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2678 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2679 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2680 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2681 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2682 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2683 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2684 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2685 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2686 2023-10-08T09:15:17.056 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2687 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2688 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2689 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2690 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 8 09:15 2691 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2692 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2693 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2696 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2699 2023-10-08T09:15:17.057 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2701 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2702 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2703 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2704 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2753 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2753_fsm 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2753_vm 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2754 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2755 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2756 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2757 2023-10-08T09:15:17.058 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830_vm 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2831 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832_vm 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2833 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834_vm 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2835 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836_vm 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2837 2023-10-08T09:15:17.059 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 8 09:15 2838 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2838_fsm 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2838_vm 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2839 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840_fsm 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2840_vm 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2841 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995_vm 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2996 2023-10-08T09:15:17.060 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3079_fsm 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079_vm 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3080 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3081 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3085 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118_vm 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3119 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3164 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256_vm 2023-10-08T09:15:17.061 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3257 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3258 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350_vm 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3351 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3379 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3380 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381_vm 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3394 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3394_fsm 2023-10-08T09:15:17.062 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3394_vm 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3395 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439_vm 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3440 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 3455 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3456 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3456_fsm 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3456_vm 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466_vm 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3467 2023-10-08T09:15:17.063 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3468 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501_vm 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3502 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3503 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3534 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3541_fsm 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541_vm 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3542 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3574 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3575 2023-10-08T09:15:17.064 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576_vm 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596_vm 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3597 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598_vm 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3599 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3600_fsm 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600_vm 2023-10-08T09:15:17.065 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3601_fsm 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601_vm 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3602_fsm 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602_vm 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3603 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3603_fsm 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3603_vm 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3604 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3605 2023-10-08T09:15:17.066 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3606 2023-10-08T09:15:17.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3607 2023-10-08T09:15:17.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3608 2023-10-08T09:15:17.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 3609 2023-10-08T09:15:17.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3712 2023-10-08T09:15:17.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764 2023-10-08T09:15:17.071 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3764_fsm 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764_vm 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3766 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3767 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3997 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 5002 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 548 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 549 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102_vm 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104_vm 2023-10-08T09:15:17.072 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106_vm 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6110 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6111 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6112 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6113 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6117 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826_vm 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 827 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 828 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:15:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:total 7185 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 . 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 .. 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 112 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 113 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 8 09:15 12182 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12182_fsm 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12182_vm 2023-10-08T09:15:17.074 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12184 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12186 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12187_fsm 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187_vm 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12189 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12191 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12192_fsm 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192_vm 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12194 2023-10-08T09:15:17.075 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12196 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12197_fsm 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197_vm 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12199 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12201 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12202_fsm 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202_vm 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12204 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12206 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207 2023-10-08T09:15:17.076 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12207_fsm 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207_vm 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12209 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12211 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12212 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12214 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12216 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 8 09:15 1247 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1247_fsm 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1247_vm 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 8 09:15 1249 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1249_fsm 2023-10-08T09:15:17.077 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1249_vm 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 8 09:15 1255 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1255_fsm 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1255_vm 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 8 09:15 1259 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1259_fsm 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1259_vm 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417_vm 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418_vm 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 174 2023-10-08T09:15:17.078 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 175 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2187 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224_vm 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328_vm 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336_vm 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2337 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2600 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2600_fsm 2023-10-08T09:15:17.079 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2600_vm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2601_fsm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601_vm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2602 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2602_fsm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2602_vm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2603 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2603_fsm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2603_vm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604_vm 2023-10-08T09:15:17.080 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2605 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2605_fsm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2605_vm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2606_fsm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606_vm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607_fsm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2607_vm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 8 09:15 2608 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2608_fsm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2608_vm 2023-10-08T09:15:17.081 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 8 09:15 2609 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2609_fsm 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2609_vm 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610_fsm 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2610_vm 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611_vm 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2612_fsm 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612_vm 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613 2023-10-08T09:15:17.082 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613_vm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2615_fsm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615_vm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616_fsm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2616_vm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 8 09:15 2617 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2617_fsm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2617_vm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 8 09:15 2618 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2618_fsm 2023-10-08T09:15:17.083 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2618_vm 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 8 09:15 2619 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2619_fsm 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2619_vm 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620_vm 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2650 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2651 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2652 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2653 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2654 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2655 2023-10-08T09:15:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2656 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2657 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 8 09:15 2658 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2659 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2660 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2661 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2662 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2663 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2664 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2665 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2666 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2667 2023-10-08T09:15:17.085 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2668 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2669 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2670 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 8 09:15 2673 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 8 09:15 2674 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 8 09:15 2675 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2678 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2679 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2680 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2681 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2682 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2683 2023-10-08T09:15:17.086 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2684 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2685 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2686 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2687 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2688 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2689 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2690 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 8 09:15 2691 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2692 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2693 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2696 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2699 2023-10-08T09:15:17.087 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2701 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2702 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2703 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2704 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2753 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2753_fsm 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2753_vm 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2754 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2755 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2756 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2757 2023-10-08T09:15:17.088 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830_vm 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2831 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832_vm 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2833 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834_vm 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2835 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836_vm 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2837 2023-10-08T09:15:17.089 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 8 09:15 2838 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2838_fsm 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2838_vm 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2839 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840_fsm 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2840_vm 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2841 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995_vm 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2996 2023-10-08T09:15:17.090 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3079_fsm 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079_vm 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3080 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3081 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3085 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118_vm 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3119 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3164 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256_vm 2023-10-08T09:15:17.091 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3257 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3258 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350_vm 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3351 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3379 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3380 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381_vm 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3394 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3394_fsm 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3394_vm 2023-10-08T09:15:17.092 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3395 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439_vm 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3440 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 3455 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3456 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3456_fsm 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3456_vm 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466_vm 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3467 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3468 2023-10-08T09:15:17.093 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501_vm 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3502 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3503 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3534 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3541_fsm 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541_vm 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3542 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3574 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3575 2023-10-08T09:15:17.094 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576_vm 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596_vm 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3597 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598_vm 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3599 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3600_fsm 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600_vm 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601 2023-10-08T09:15:17.095 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3601_fsm 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601_vm 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3602_fsm 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602_vm 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3603 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3603_fsm 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3603_vm 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3604 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3605 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3606 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3607 2023-10-08T09:15:17.096 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3608 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 3609 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3712 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3764_fsm 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764_vm 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3766 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3767 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3997 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 5002 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 548 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 549 2023-10-08T09:15:17.097 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102_vm 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104_vm 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106_vm 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6110 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6111 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6112 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6113 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6117 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826 2023-10-08T09:15:17.098 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826_vm 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 827 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 828 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:total 7185 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 . 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 .. 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 112 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 113 2023-10-08T09:15:17.099 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 8 09:15 12182 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12182_fsm 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12182_vm 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12184 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12186 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12187_fsm 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187_vm 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12189 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12191 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192 2023-10-08T09:15:17.100 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12192_fsm 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192_vm 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12194 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12196 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12197_fsm 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197_vm 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12199 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12201 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12202_fsm 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202_vm 2023-10-08T09:15:17.101 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12204 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12206 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12207_fsm 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207_vm 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12209 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12211 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12212 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12214 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12216 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 8 09:15 1247 2023-10-08T09:15:17.102 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1247_fsm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1247_vm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 8 09:15 1249 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1249_fsm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1249_vm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 8 09:15 1255 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1255_fsm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1255_vm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 8 09:15 1259 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1259_fsm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1259_vm 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417 2023-10-08T09:15:17.103 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417_vm 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418_vm 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 174 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 175 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2187 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224_vm 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328_vm 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336_vm 2023-10-08T09:15:17.104 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2337 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2600 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2600_fsm 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2600_vm 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2601_fsm 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601_vm 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2602 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2602_fsm 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2602_vm 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2603 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2603_fsm 2023-10-08T09:15:17.105 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2603_vm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604_vm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2605 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2605_fsm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2605_vm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2606_fsm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606_vm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607_fsm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2607_vm 2023-10-08T09:15:17.106 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 8 09:15 2608 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2608_fsm 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2608_vm 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 8 09:15 2609 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2609_fsm 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2609_vm 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610_fsm 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2610_vm 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611_vm 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612 2023-10-08T09:15:17.107 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2612_fsm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612_vm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613_vm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2615_fsm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615_vm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616_fsm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2616_vm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 8 09:15 2617 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2617_fsm 2023-10-08T09:15:17.108 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2617_vm 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 8 09:15 2618 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2618_fsm 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2618_vm 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 8 09:15 2619 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2619_fsm 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2619_vm 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620_vm 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2650 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2651 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2652 2023-10-08T09:15:17.109 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2653 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2654 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2655 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2656 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2657 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 8 09:15 2658 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2659 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2660 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2661 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2662 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2663 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2664 2023-10-08T09:15:17.110 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2665 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2666 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2667 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2668 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2669 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2670 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 8 09:15 2673 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 8 09:15 2674 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 8 09:15 2675 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2678 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2679 2023-10-08T09:15:17.111 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2680 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2681 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2682 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2683 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2684 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2685 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2686 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2687 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2688 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2689 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2690 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 8 09:15 2691 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2692 2023-10-08T09:15:17.112 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2693 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2696 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2699 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2701 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2702 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2703 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2704 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2753 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2753_fsm 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2753_vm 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2754 2023-10-08T09:15:17.113 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2755 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2756 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2757 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830_vm 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2831 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832_vm 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2833 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834_vm 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2835 2023-10-08T09:15:17.114 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836_vm 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2837 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 8 09:15 2838 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2838_fsm 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2838_vm 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2839 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840_fsm 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2840_vm 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2841 2023-10-08T09:15:17.115 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995_vm 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2996 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3079_fsm 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079_vm 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3080 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3081 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3085 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118_vm 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3119 2023-10-08T09:15:17.116 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3164 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256_vm 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3257 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3258 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350_vm 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3351 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3379 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3380 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381_vm 2023-10-08T09:15:17.117 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3394 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3394_fsm 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3394_vm 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3395 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439_vm 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3440 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 3455 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3456 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3456_fsm 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3456_vm 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466 2023-10-08T09:15:17.118 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466_vm 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3467 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3468 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501_vm 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3502 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3503 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3534 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3541_fsm 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541_vm 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3542 2023-10-08T09:15:17.119 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3574 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3575 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576_vm 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596_vm 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3597 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598_vm 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3599 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3600_fsm 2023-10-08T09:15:17.120 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600_vm 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3601_fsm 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601_vm 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3602_fsm 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602_vm 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3603 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3603_fsm 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3603_vm 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3604 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3605 2023-10-08T09:15:17.121 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3606 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3607 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3608 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 3609 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3712 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3764_fsm 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764_vm 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3766 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3767 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3997 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 5002 2023-10-08T09:15:17.122 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 548 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 549 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102_vm 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104_vm 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106_vm 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6110 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6111 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6112 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6113 2023-10-08T09:15:17.123 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6117 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826_vm 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 827 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 828 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/global: 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:total 553 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 8 09:15 . 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.124 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1136 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1136_fsm 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1136_vm 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 1137 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1213 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1213_fsm 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1213_vm 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1214 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1214_fsm 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1214_vm 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 1232 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 1233 2023-10-08T09:15:17.125 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1260 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1260_fsm 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1260_vm 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1261 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1261_fsm 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1261_vm 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1262 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1262_fsm 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1262_vm 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2396 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2396_fsm 2023-10-08T09:15:17.126 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2396_vm 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2397 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2671 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2672 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2676 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2677 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2694 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2695 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2697 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2698 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2846 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2846_vm 2023-10-08T09:15:17.127 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2847 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2964 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2964_vm 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2965 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2966 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2966_vm 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2967 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3592 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3592_vm 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3593 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 4060 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 4060_vm 2023-10-08T09:15:17.128 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 4061 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6000 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6000_vm 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6001 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6002 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6100 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6100_vm 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6114 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6115 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 pg_control 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/log: 2023-10-08T09:15:17.129 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.130 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:total 1 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 . 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 mappings 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 8 09:15 replorigin_checkpoint 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 snapshots 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-08T09:15:17.131 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 .. 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 .. 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 . 2023-10-08T09:15:17.132 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 members 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 offsets 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 .. 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:15:17.133 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 .. 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.134 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-08T09:15:17.136 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.137 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:total 16384 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 . 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:15 000000010000000000000001 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 archive_status 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-08T09:15:17.138 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 .. 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 8 09:15 .. 2023-10-08T09:15:17.139 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:15:17.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-10-08T09:15:17.288 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2023-10-08T09:15:17.344 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-10-08T09:15:17.350 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-10-08T09:15:17.353 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-10-08T09:15:17.357 INFO:teuthology.orchestra.run.smithi049.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-10-08T09:15:17.376 INFO:teuthology.orchestra.run.smithi049.stderr:creating tables... 2023-10-08T09:15:17.440 INFO:teuthology.orchestra.run.smithi049.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.90 s) 2023-10-08T09:15:17.543 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:17 smithi049 ceph-mon[99485]: pgmap v340: 129 pgs: 129 active+clean; 29 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.4 MiB/s wr, 51 op/s 2023-10-08T09:15:17.543 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:17 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:17.626 INFO:teuthology.orchestra.run.smithi049.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.25 s, remaining 3.71 s) 2023-10-08T09:15:17.740 INFO:teuthology.orchestra.run.smithi049.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.36 s, remaining 3.50 s) 2023-10-08T09:15:17.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:17 smithi132 ceph-mon[103389]: pgmap v340: 129 pgs: 129 active+clean; 29 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.4 MiB/s wr, 51 op/s 2023-10-08T09:15:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:17 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:17.896 INFO:teuthology.orchestra.run.smithi049.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.52 s, remaining 3.63 s) 2023-10-08T09:15:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:17 smithi184 ceph-mon[103675]: pgmap v340: 129 pgs: 129 active+clean; 29 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.4 MiB/s wr, 51 op/s 2023-10-08T09:15:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:17 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:18.066 INFO:teuthology.orchestra.run.smithi049.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.69 s, remaining 3.71 s) 2023-10-08T09:15:18.184 INFO:teuthology.orchestra.run.smithi049.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.81 s, remaining 3.49 s) 2023-10-08T09:15:18.337 INFO:teuthology.orchestra.run.smithi049.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.96 s, remaining 3.42 s) 2023-10-08T09:15:18.516 INFO:teuthology.orchestra.run.smithi049.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.14 s, remaining 3.41 s) 2023-10-08T09:15:18.621 INFO:teuthology.orchestra.run.smithi049.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.24 s, remaining 3.18 s) 2023-10-08T09:15:18.774 INFO:teuthology.orchestra.run.smithi049.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.40 s, remaining 3.07 s) 2023-10-08T09:15:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.812+0000 7f47099e9700 1 -- 172.21.15.49:0/3247854784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47040ff660 msgr2=0x7f47040ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.812+0000 7f47099e9700 1 --2- 172.21.15.49:0/3247854784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47040ff660 0x7f47040ffa80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b0d0 tx=0x7f46fc00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.812+0000 7f47099e9700 1 -- 172.21.15.49:0/3247854784 shutdown_connections 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.812+0000 7f47099e9700 1 --2- 172.21.15.49:0/3247854784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47041011f0 0x7f4704107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.812+0000 7f47099e9700 1 --2- 172.21.15.49:0/3247854784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47040ff660 0x7f47040ffa80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.812+0000 7f47099e9700 1 --2- 172.21.15.49:0/3247854784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4704100850 0x7f4704100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.812+0000 7f47099e9700 1 -- 172.21.15.49:0/3247854784 >> 172.21.15.49:0/3247854784 conn(0x7f47040fabd0 msgr2=0x7f47040fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.813+0000 7f47099e9700 1 -- 172.21.15.49:0/3247854784 shutdown_connections 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.813+0000 7f47099e9700 1 -- 172.21.15.49:0/3247854784 wait complete. 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.813+0000 7f47099e9700 1 Processor -- start 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.813+0000 7f47099e9700 1 -- start start 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.813+0000 7f47099e9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47040ff660 0x7f4704069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.813+0000 7f47099e9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704100850 0x7f470419b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47099e9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47041011f0 0x7f470419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47099e9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470419e360 con 0x7f4704100850 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47099e9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470419e4d0 con 0x7f47040ff660 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47099e9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470419e690 con 0x7f47041011f0 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47027fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704100850 0x7f470419b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47037fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47041011f0 0x7f470419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47027fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704100850 0x7f470419b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50222/0 (socket says 172.21.15.49:50222) 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47027fc700 1 -- 172.21.15.49:0/2553950490 learned_addr learned my addr 172.21.15.49:0/2553950490 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f4702ffd700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47040ff660 0x7f4704069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47037fe700 1 -- 172.21.15.49:0/2553950490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47040ff660 msgr2=0x7f4704069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47037fe700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47040ff660 0x7f4704069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47037fe700 1 -- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704100850 msgr2=0x7f470419b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47037fe700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704100850 0x7f470419b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47037fe700 1 -- 172.21.15.49:0/2553950490 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f4009520 con 0x7f47041011f0 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47037fe700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47041011f0 0x7f470419dbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f46f400e900 tx=0x7f46f400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47089e7700 1 -- 172.21.15.49:0/2553950490 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4004220 con 0x7f47041011f0 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc009d00 con 0x7f47041011f0 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.814+0000 7f47089e7700 1 -- 172.21.15.49:0/2553950490 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f4004380 con 0x7f47041011f0 2023-10-08T09:15:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.815+0000 7f47089e7700 1 -- 172.21.15.49:0/2553950490 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4003d20 con 0x7f47041011f0 2023-10-08T09:15:18.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.815+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470419ed40 con 0x7f47041011f0 2023-10-08T09:15:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.817+0000 7f47089e7700 1 -- 172.21.15.49:0/2553950490 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f46f4018430 con 0x7f47041011f0 2023-10-08T09:15:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f47089e7700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46f00a1480 0x7f46f00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f47089e7700 1 -- 172.21.15.49:0/2553950490 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f46f4014070 con 0x7f47041011f0 2023-10-08T09:15:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 Processor -- start 2023-10-08T09:15:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4702ffd700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46f00a1480 0x7f46f00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 -- start start 2023-10-08T09:15:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e0fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 0x55ab5e0fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e0fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e05e400 con 0x55ab5e0fa250 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e04fbc0 con 0x55ab5e0fabb0 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e04d4e0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4702ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e0fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4702ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e0fa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50228/0 (socket says 172.21.15.49:50228) 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.818+0000 7f4702ffd700 1 -- 172.21.15.49:0/2655670533 learned_addr learned my addr 172.21.15.49:0/2655670533 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f47037fe700 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e0fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f47027fc700 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 0x55ab5e0fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f4702ffd700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46f00a1480 0x7f46f00a3940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009fd0 tx=0x7f46fc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f4702ffd700 1 -- 172.21.15.49:0/2655670533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 msgr2=0x55ab5e0fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f4702ffd700 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e0fbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f4702ffd700 1 -- 172.21.15.49:0/2655670533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 msgr2=0x55ab5e0fb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f4702ffd700 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 0x55ab5e0fb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f4702ffd700 1 -- 172.21.15.49:0/2655670533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab5de73d80 con 0x55ab5e0fa250 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f4702ffd700 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e0fa670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f46fc015000 tx=0x7f46fc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868b5df2355e5381 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f46f9ffb700 1 -- 172.21.15.49:0/2655670533 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00e040 con 0x55ab5e0fa250 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f46f9ffb700 1 -- 172.21.15.49:0/2655670533 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46fc00bca0 con 0x55ab5e0fa250 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.819+0000 7f46f9ffb700 1 -- 172.21.15.49:0/2655670533 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc01c670 con 0x55ab5e0fa250 2023-10-08T09:15:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 -- 172.21.15.49:0/2655670533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 msgr2=0x55ab5e0fa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e0fa670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f46fc015000 tx=0x7f46fc01d000 comp rx=0 tx=0).stop 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 -- 172.21.15.49:0/2655670533 shutdown_connections 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e0fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e0fa670 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/2655670533 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 0x55ab5e0fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 -- 172.21.15.49:0/2655670533 >> 172.21.15.49:0/2655670533 conn(0x55ab5e06a200 msgr2=0x55ab5e0f08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 -- 172.21.15.49:0/2655670533 shutdown_connections 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 -- 172.21.15.49:0/2655670533 wait complete. 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.820+0000 7f4715bd2b80 1 -- start start 2023-10-08T09:15:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 1 Processor -- start 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 0x55ab5e104110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e10b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e048dd0 con 0x55ab5e0fa250 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5dfe56b0 con 0x55ab5e0fabb0 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5dfe5350 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4715bd2b80 10 client.0 _ll_register_callbacks cb 0x55ab5de427a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f4702ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f47037fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e10b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f47037fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e10b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37270/0 (socket says 172.21.15.49:37270) 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.821+0000 7f47037fe700 1 -- 172.21.15.49:0/376383875 learned_addr learned my addr 172.21.15.49:0/376383875 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f47037fe700 1 -- 172.21.15.49:0/376383875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 msgr2=0x55ab5e104110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f47037fe700 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 0x55ab5e104110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f47037fe700 1 -- 172.21.15.49:0/376383875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 msgr2=0x55ab5e103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f47037fe700 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f47037fe700 1 -- 172.21.15.49:0/376383875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47041083f0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f47037fe700 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e10b150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46f40967a0 tx=0x7f46f40044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b83b05af774221 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f46e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f46e2ffd700 1 -- 172.21.15.49:0/376383875 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f409a2a0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f46e2ffd700 1 -- 172.21.15.49:0/376383875 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f46f409e3f0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab5e010e50 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f46e2ffd700 1 -- 172.21.15.49:0/376383875 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f40a2630 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f4715bd2b80 10 client.25459 fetch_fsmap learned FSMap version 29 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ab5de5cc00 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.822+0000 7f46e2ffd700 1 -- 172.21.15.49:0/376383875 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab5e010e50 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f46e2ffd700 1 -- 172.21.15.49:0/376383875 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f46f4099790 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f4715bd2b80 10 client.25459 fetch_fsmap finished waiting for FSMap version 29 2023-10-08T09:15:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f4715bd2b80 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f4715bd2b80 20 client.25459 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e10fa50 0x55ab5e111e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f4715bd2b80 4 client.25459 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab5e041db0 con 0x55ab5e10fa50 2023-10-08T09:15:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.823+0000 7f4702ffd700 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e10fa50 0x55ab5e111e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.824+0000 7f46e2ffd700 10 client.25459 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.824+0000 7f4702ffd700 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e10fa50 0x55ab5e111e70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61b27d2893e08ed server_cookie=c47303f9244aeaf9 in_seq=0 out_seq=0 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.824+0000 7f46e2ffd700 1 -- 172.21.15.49:0/376383875 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab5e041db0 con 0x55ab5e10fa50 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.824+0000 7f46e2ffd700 10 client.25459 handle_command_reply: tid=0 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.824+0000 7f4715bd2b80 1 client.25459 shutdown 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.825+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 msgr2=0x55ab5e10b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.825+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e10b150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46f40967a0 tx=0x7f46f40044f0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 shutdown_connections 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e10fa50 0x55ab5e111e70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e10b150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fa250 0x55ab5e103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/376383875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fabb0 0x55ab5e104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 >> 172.21.15.49:0/376383875 conn(0x55ab5e06a200 msgr2=0x55ab5e0f0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f46e2ffd700 0 client.25459 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 shutdown_connections 2023-10-08T09:15:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 1 -- 172.21.15.49:0/376383875 wait complete. 2023-10-08T09:15:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.826+0000 7f4715bd2b80 20 client.25459 trim_cache size 0 max 16384 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 Processor -- start 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 -- start start 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e10b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fabb0 0x55ab5e10b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e102940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5dec98c0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e041de0 con 0x55ab5e0fa250 2023-10-08T09:15:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e0fb2f0 con 0x55ab5e0fabb0 2023-10-08T09:15:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f47037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e102940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f47037fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e102940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50244/0 (socket says 172.21.15.49:50244) 2023-10-08T09:15:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f47037fe700 1 -- 172.21.15.49:0/4019174886 learned_addr learned my addr 172.21.15.49:0/4019174886 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f4702ffd700 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e10b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.852+0000 7f47027fc700 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fabb0 0x55ab5e10b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f47037fe700 1 -- 172.21.15.49:0/4019174886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fabb0 msgr2=0x55ab5e10b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f47037fe700 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fabb0 0x55ab5e10b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f47037fe700 1 -- 172.21.15.49:0/4019174886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 msgr2=0x55ab5e10b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f47037fe700 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e10b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f47037fe700 1 -- 172.21.15.49:0/4019174886 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab5de9a1f0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f47037fe700 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e102940 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f46f40985f0 tx=0x7f46f409aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c089f155be61d933 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f46fa7fc700 1 -- 172.21.15.49:0/4019174886 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f409e3f0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f46fa7fc700 1 -- 172.21.15.49:0/4019174886 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f409a2a0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.853+0000 7f46fa7fc700 1 -- 172.21.15.49:0/4019174886 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f40a21f0 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 -- 172.21.15.49:0/4019174886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fb6c0 msgr2=0x55ab5e102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e102940 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f46f40985f0 tx=0x7f46f409aae0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 -- 172.21.15.49:0/4019174886 shutdown_connections 2023-10-08T09:15:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e102940 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/4019174886 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fabb0 0x55ab5e10b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 -- 172.21.15.49:0/4019174886 >> 172.21.15.49:0/4019174886 conn(0x55ab5e06a200 msgr2=0x55ab5e0f0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 -- 172.21.15.49:0/4019174886 shutdown_connections 2023-10-08T09:15:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 -- 172.21.15.49:0/4019174886 wait complete. 2023-10-08T09:15:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.854+0000 7f4715bd2b80 1 -- start start 2023-10-08T09:15:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 1 Processor -- start 2023-10-08T09:15:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e109350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fabb0 0x55ab5e109890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e10a420 con 0x55ab5e0fabb0 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e10a590 con 0x55ab5e0fa250 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab5e10a750 con 0x55ab5e0fb6c0 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4715bd2b80 10 client.0 _ll_register_callbacks cb 0x55ab5de427a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f47027fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fabb0 0x55ab5e109890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4702ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f47027fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fabb0 0x55ab5e109890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50248/0 (socket says 172.21.15.49:50248) 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f47027fc700 1 -- 172.21.15.49:0/773386412 learned_addr learned my addr 172.21.15.49:0/773386412 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4702ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e109350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:39860/0 (socket says 172.21.15.49:39860) 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f47037fe700 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4702ffd700 1 -- 172.21.15.49:0/773386412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 msgr2=0x55ab5e109dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4702ffd700 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e109dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4702ffd700 1 -- 172.21.15.49:0/773386412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fabb0 msgr2=0x55ab5e109890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4702ffd700 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fabb0 0x55ab5e109890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.855+0000 7f4702ffd700 1 -- 172.21.15.49:0/773386412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab5de5cc00 con 0x55ab5e0fa250 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4702ffd700 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e109350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00d010 tx=0x7f46fc0095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b9bf8f96b874e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f46e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f46e2ffd700 1 -- 172.21.15.49:0/773386412 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc029ea0 con 0x55ab5e0fa250 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46fc00bca0 con 0x55ab5e0fa250 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f46e2ffd700 1 -- 172.21.15.49:0/773386412 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f46fc036040 con 0x55ab5e0fa250 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f46e2ffd700 1 -- 172.21.15.49:0/773386412 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46fc00e040 con 0x55ab5e0fa250 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 10 client.25442 fetch_fsmap learned FSMap version 29 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ab5de72250 con 0x55ab5e0fa250 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f46e2ffd700 1 -- 172.21.15.49:0/773386412 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46fc00bca0 con 0x55ab5e0fa250 2023-10-08T09:15:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f46e2ffd700 1 -- 172.21.15.49:0/773386412 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 29) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f46fc025050 con 0x55ab5e0fa250 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 10 client.25442 fetch_fsmap finished waiting for FSMap version 29 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 20 client.25442 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e164120 0x55ab5e166540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 4 client.25442 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.856+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab5e041ad0 con 0x55ab5e164120 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.857+0000 7f47027fc700 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e164120 0x55ab5e166540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.857+0000 7f47027fc700 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e164120 0x55ab5e166540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3600ed5815b95e21 server_cookie=7b2f330142c04989 in_seq=0 out_seq=0 2023-10-08T09:15:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:18.857+0000 7f46e2ffd700 10 client.25442 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f46e2ffd700 1 -- 172.21.15.49:0/773386412 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab5e041ad0 con 0x55ab5e164120 2023-10-08T09:15:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f46e2ffd700 10 client.25442 handle_command_reply: tid=0 2023-10-08T09:15:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 client.25442 shutdown 2023-10-08T09:15:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 msgr2=0x55ab5e109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e109350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00d010 tx=0x7f46fc0095a0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 shutdown_connections 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ab5e164120 0x55ab5e166540 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ab5e0fb6c0 0x55ab5e109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f46e2ffd700 0 client.25442 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ab5e0fa250 0x55ab5e109350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 --2- 172.21.15.49:0/773386412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ab5e0fabb0 0x55ab5e109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 >> 172.21.15.49:0/773386412 conn(0x55ab5e06a200 msgr2=0x55ab5e103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 shutdown_connections 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.244+0000 7f4715bd2b80 1 -- 172.21.15.49:0/773386412 wait complete. 2023-10-08T09:15:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f4715bd2b80 20 client.25442 trim_cache size 0 max 16384 2023-10-08T09:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:15:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46f00a1480 msgr2=0x7f46f00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46f00a1480 0x7f46f00a3940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009fd0 tx=0x7f46fc00b040 comp rx=0 tx=0).stop 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47041011f0 msgr2=0x7f470419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47041011f0 0x7f470419dbc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f46f400e900 tx=0x7f46f400ec10 comp rx=0 tx=0).stop 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 shutdown_connections 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46f00a1480 0x7f46f00a3940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47041011f0 0x7f470419dbc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47040ff660 0x7f4704069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 --2- 172.21.15.49:0/2553950490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4704100850 0x7f470419b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 >> 172.21.15.49:0/2553950490 conn(0x7f47040fabd0 msgr2=0x7f4704103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 shutdown_connections 2023-10-08T09:15:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:19.245+0000 7f47099e9700 1 -- 172.21.15.49:0/2553950490 wait complete. 2023-10-08T09:15:19.250 INFO:teuthology.orchestra.run.smithi049.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.87 s, remaining 3.57 s) 2023-10-08T09:15:19.367 INFO:teuthology.orchestra.run.smithi049.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.99 s, remaining 3.31 s) 2023-10-08T09:15:19.537 INFO:teuthology.orchestra.run.smithi049.stderr:1300000 of 3200000 tuples (40%) done (elapsed 2.16 s, remaining 3.16 s) 2023-10-08T09:15:19.693 INFO:teuthology.orchestra.run.smithi049.stderr:1400000 of 3200000 tuples (43%) done (elapsed 2.31 s, remaining 2.98 s) 2023-10-08T09:15:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: pgmap v341: 129 pgs: 129 active+clean; 38 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.2 MiB/s wr, 62 op/s 2023-10-08T09:15:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:15:19.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:15:19.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.769 INFO:teuthology.orchestra.run.smithi049.stderr:1500000 of 3200000 tuples (46%) done (elapsed 2.39 s, remaining 2.71 s) 2023-10-08T09:15:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: pgmap v341: 129 pgs: 129 active+clean; 38 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.2 MiB/s wr, 62 op/s 2023-10-08T09:15:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:15:19.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:15:19.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: pgmap v341: 129 pgs: 129 active+clean; 38 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.2 MiB/s wr, 62 op/s 2023-10-08T09:15:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:15:19.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:15:19.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:15:19.940 INFO:teuthology.orchestra.run.smithi049.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.56 s, remaining 2.56 s) 2023-10-08T09:15:20.116 INFO:teuthology.orchestra.run.smithi049.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.74 s, remaining 2.42 s) 2023-10-08T09:15:20.183 INFO:teuthology.orchestra.run.smithi049.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.81 s, remaining 2.18 s) 2023-10-08T09:15:20.359 INFO:teuthology.orchestra.run.smithi049.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.98 s, remaining 2.04 s) 2023-10-08T09:15:20.518 INFO:teuthology.orchestra.run.smithi049.stderr:2000000 of 3200000 tuples (62%) done (elapsed 3.14 s, remaining 1.88 s) 2023-10-08T09:15:20.614 INFO:teuthology.orchestra.run.smithi049.stderr:2100000 of 3200000 tuples (65%) done (elapsed 3.24 s, remaining 1.70 s) 2023-10-08T09:15:20.785 INFO:teuthology.orchestra.run.smithi049.stderr:2200000 of 3200000 tuples (68%) done (elapsed 3.41 s, remaining 1.55 s) 2023-10-08T09:15:20.852 INFO:teuthology.orchestra.run.smithi049.stderr:2300000 of 3200000 tuples (71%) done (elapsed 3.47 s, remaining 1.36 s) 2023-10-08T09:15:21.028 INFO:teuthology.orchestra.run.smithi049.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.65 s, remaining 1.22 s) 2023-10-08T09:15:21.177 INFO:teuthology.orchestra.run.smithi049.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.80 s, remaining 1.06 s) 2023-10-08T09:15:21.263 INFO:teuthology.orchestra.run.smithi049.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.89 s, remaining 0.90 s) 2023-10-08T09:15:21.425 INFO:teuthology.orchestra.run.smithi049.stderr:2700000 of 3200000 tuples (84%) done (elapsed 4.05 s, remaining 0.75 s) 2023-10-08T09:15:21.585 INFO:teuthology.orchestra.run.smithi049.stderr:2800000 of 3200000 tuples (87%) done (elapsed 4.21 s, remaining 0.60 s) 2023-10-08T09:15:21.677 INFO:teuthology.orchestra.run.smithi049.stderr:2900000 of 3200000 tuples (90%) done (elapsed 4.30 s, remaining 0.44 s) 2023-10-08T09:15:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:21 smithi184 ceph-mon[103675]: pgmap v342: 129 pgs: 129 active+clean; 87 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 11 MiB/s wr, 159 op/s 2023-10-08T09:15:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:21 smithi132 ceph-mon[103389]: pgmap v342: 129 pgs: 129 active+clean; 87 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 11 MiB/s wr, 159 op/s 2023-10-08T09:15:21.832 INFO:teuthology.orchestra.run.smithi049.stderr:3000000 of 3200000 tuples (93%) done (elapsed 4.45 s, remaining 0.30 s) 2023-10-08T09:15:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:21 smithi049 ceph-mon[99485]: pgmap v342: 129 pgs: 129 active+clean; 87 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 11 MiB/s wr, 159 op/s 2023-10-08T09:15:21.932 INFO:teuthology.orchestra.run.smithi049.stderr:3100000 of 3200000 tuples (96%) done (elapsed 4.55 s, remaining 0.15 s) 2023-10-08T09:15:22.119 INFO:teuthology.orchestra.run.smithi049.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.74 s, remaining 0.00 s) 2023-10-08T09:15:22.147 INFO:teuthology.orchestra.run.smithi049.stderr:vacuum... 2023-10-08T09:15:22.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:22 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:22.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:22 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:22.800 INFO:teuthology.orchestra.run.smithi049.stderr:set primary keys... 2023-10-08T09:15:22.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:22 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:23.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:23 smithi184 ceph-mon[103675]: pgmap v343: 129 pgs: 129 active+clean; 174 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 25 MiB/s wr, 272 op/s 2023-10-08T09:15:23.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:23 smithi132 ceph-mon[103389]: pgmap v343: 129 pgs: 129 active+clean; 174 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 25 MiB/s wr, 272 op/s 2023-10-08T09:15:23.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:23 smithi049 ceph-mon[99485]: pgmap v343: 129 pgs: 129 active+clean; 174 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 25 MiB/s wr, 272 op/s 2023-10-08T09:15:24.242 INFO:teuthology.orchestra.run.smithi049.stderr:done. 2023-10-08T09:15:24.245 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-10-08T09:15:24.309 INFO:teuthology.orchestra.run.smithi049.stderr:starting vacuum...end. 2023-10-08T09:15:25.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:25 smithi184 ceph-mon[103675]: pgmap v344: 129 pgs: 129 active+clean; 291 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 45 MiB/s wr, 382 op/s 2023-10-08T09:15:25.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:25 smithi132 ceph-mon[103389]: pgmap v344: 129 pgs: 129 active+clean; 291 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 45 MiB/s wr, 382 op/s 2023-10-08T09:15:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:25 smithi049 ceph-mon[99485]: pgmap v344: 129 pgs: 129 active+clean; 291 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 45 MiB/s wr, 382 op/s 2023-10-08T09:15:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:27 smithi184 ceph-mon[103675]: pgmap v345: 129 pgs: 129 active+clean; 451 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 62 MiB/s wr, 393 op/s 2023-10-08T09:15:27.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:27 smithi132 ceph-mon[103389]: pgmap v345: 129 pgs: 129 active+clean; 451 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 62 MiB/s wr, 393 op/s 2023-10-08T09:15:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:27 smithi049 ceph-mon[99485]: pgmap v345: 129 pgs: 129 active+clean; 451 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 62 MiB/s wr, 393 op/s 2023-10-08T09:15:28.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:28 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:28.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:28 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:28.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:28 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:29.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 5.0 s, 2807.8 tps, lat 34.551 ms stddev 40.621 2023-10-08T09:15:29.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:29 smithi184 ceph-mon[103675]: pgmap v346: 129 pgs: 129 active+clean; 744 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 80 MiB/s wr, 454 op/s 2023-10-08T09:15:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:29 smithi132 ceph-mon[103389]: pgmap v346: 129 pgs: 129 active+clean; 744 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 80 MiB/s wr, 454 op/s 2023-10-08T09:15:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:29 smithi049 ceph-mon[99485]: pgmap v346: 129 pgs: 129 active+clean; 744 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 80 MiB/s wr, 454 op/s 2023-10-08T09:15:30.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:30 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:30.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:30 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:30 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:31.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:31 smithi184 ceph-mon[103675]: pgmap v347: 129 pgs: 129 active+clean; 930 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 92 MiB/s wr, 534 op/s 2023-10-08T09:15:31.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:31 smithi132 ceph-mon[103389]: pgmap v347: 129 pgs: 129 active+clean; 930 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 92 MiB/s wr, 534 op/s 2023-10-08T09:15:31.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:31 smithi049 ceph-mon[99485]: pgmap v347: 129 pgs: 129 active+clean; 930 MiB data, 771 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 92 MiB/s wr, 534 op/s 2023-10-08T09:15:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.466+0000 7f9c0d38f700 1 -- 172.21.15.49:0/1046330108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c0810cb30 msgr2=0x7f9c0810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.466+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/1046330108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c0810cb30 0x7f9c0810cfb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bf801a9a0 tx=0x7f9bf801acb0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.466+0000 7f9c0d38f700 1 -- 172.21.15.49:0/1046330108 shutdown_connections 2023-10-08T09:15:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.466+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/1046330108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c0810cb30 0x7f9c0810cfb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.466+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/1046330108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c081077f0 0x7f9c08109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.466+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/1046330108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c0810a160 0x7f9c0810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.466+0000 7f9c0d38f700 1 -- 172.21.15.49:0/1046330108 >> 172.21.15.49:0/1046330108 conn(0x7f9c0806d770 msgr2=0x7f9c0806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.476+0000 7f9c0d38f700 1 -- 172.21.15.49:0/1046330108 shutdown_connections 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.479+0000 7f9c0d38f700 1 -- 172.21.15.49:0/1046330108 wait complete. 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.481+0000 7f9c0d38f700 1 Processor -- start 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.482+0000 7f9c0d38f700 1 -- start start 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.482+0000 7f9c0d38f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c0810a160 0x7f9c0819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.482+0000 7f9c0d38f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0810cb30 0x7f9c0819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.482+0000 7f9c0d38f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c08076a40 0x7f9c08078ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.482+0000 7f9c0d38f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08079410 con 0x7f9c0810cb30 2023-10-08T09:15:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.482+0000 7f9c0d38f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08079580 con 0x7f9c0810a160 2023-10-08T09:15:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.482+0000 7f9c0d38f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08079740 con 0x7f9c08076a40 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0810cb30 0x7f9c0819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0810cb30 0x7f9c0819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50260/0 (socket says 172.21.15.49:50260) 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 -- 172.21.15.49:0/238130055 learned_addr learned my addr 172.21.15.49:0/238130055 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c077fe700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c08076a40 0x7f9c08078ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 -- 172.21.15.49:0/238130055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c08076a40 msgr2=0x7f9c08078ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c08076a40 0x7f9c08078ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 -- 172.21.15.49:0/238130055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c0810a160 msgr2=0x7f9c0819ceb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c0810a160 0x7f9c0819ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.483+0000 7f9c067fc700 1 -- 172.21.15.49:0/238130055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf801a600 con 0x7f9c0810cb30 2023-10-08T09:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.484+0000 7f9c067fc700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0810cb30 0x7f9c0819d3f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b840 tx=0x7f9c0000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.484+0000 7f9beffff700 1 -- 172.21.15.49:0/238130055 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000f840 con 0x7f9c0810cb30 2023-10-08T09:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.484+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c08079a20 con 0x7f9c0810cb30 2023-10-08T09:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.484+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08079f80 con 0x7f9c0810cb30 2023-10-08T09:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.489+0000 7f9c19578b80 1 Processor -- start 2023-10-08T09:15:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.489+0000 7f9c19578b80 1 -- start start 2023-10-08T09:15:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.490+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.490+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 0x5640753c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.490+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c32b0 0x5640753c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.490+0000 7f9c19578b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407532a400 con 0x5640753c1e40 2023-10-08T09:15:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.490+0000 7f9c19578b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407531bbc0 con 0x5640753c32b0 2023-10-08T09:15:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.490+0000 7f9c19578b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640753194e0 con 0x5640753c27a0 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9c077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9c077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50264/0 (socket says 172.21.15.49:50264) 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9c077fe700 1 -- 172.21.15.49:0/3405030233 learned_addr learned my addr 172.21.15.49:0/3405030233 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9beffff700 1 -- 172.21.15.49:0/238130055 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c0000fe60 con 0x7f9c0810cb30 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9beffff700 1 -- 172.21.15.49:0/238130055 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000d3d0 con 0x7f9c0810cb30 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9c077fe700 1 -- 172.21.15.49:0/3405030233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 msgr2=0x5640753c2cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9c077fe700 1 --2- 172.21.15.49:0/3405030233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 0x5640753c2cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9c077fe700 1 -- 172.21.15.49:0/3405030233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c32b0 msgr2=0x5640753c3730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.491+0000 7f9c077fe700 1 --2- 172.21.15.49:0/3405030233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c32b0 0x5640753c3730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.492+0000 7f9c077fe700 1 -- 172.21.15.49:0/3405030233 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56407513fd80 con 0x5640753c1e40 2023-10-08T09:15:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.503+0000 7f9beffff700 1 -- 172.21.15.49:0/238130055 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9c0000f9a0 con 0x7f9c0810cb30 2023-10-08T09:15:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.504+0000 7f9beffff700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bf00a1550 0x7f9bf00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.504+0000 7f9beffff700 1 -- 172.21.15.49:0/238130055 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9c000ccd40 con 0x7f9c0810cb30 2023-10-08T09:15:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.512+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bf00a1550 0x7f9bf00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.513+0000 7f9c077fe700 1 --2- 172.21.15.49:0/3405030233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753c2260 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9bf801a9a0 tx=0x7f9bf80038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d178b684d24e38cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.513+0000 7f9bed7fa700 1 -- 172.21.15.49:0/3405030233 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf802d040 con 0x5640753c1e40 2023-10-08T09:15:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.513+0000 7f9bed7fa700 1 -- 172.21.15.49:0/3405030233 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bf8005500 con 0x5640753c1e40 2023-10-08T09:15:32.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.513+0000 7f9bed7fa700 1 -- 172.21.15.49:0/3405030233 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8004a20 con 0x5640753c1e40 2023-10-08T09:15:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.514+0000 7f9c19578b80 1 -- 172.21.15.49:0/3405030233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 msgr2=0x5640753c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.514+0000 7f9c19578b80 1 --2- 172.21.15.49:0/3405030233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753c2260 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9bf801a9a0 tx=0x7f9bf80038f0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.518+0000 7f9c19578b80 1 -- 172.21.15.49:0/3405030233 shutdown_connections 2023-10-08T09:15:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.518+0000 7f9c19578b80 1 --2- 172.21.15.49:0/3405030233 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c32b0 0x5640753c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.518+0000 7f9c19578b80 1 --2- 172.21.15.49:0/3405030233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753c2260 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.518+0000 7f9c19578b80 1 --2- 172.21.15.49:0/3405030233 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 0x5640753c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.518+0000 7f9c19578b80 1 -- 172.21.15.49:0/3405030233 >> 172.21.15.49:0/3405030233 conn(0x564075336200 msgr2=0x5640753bc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.518+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bf00a1550 0x7f9bf00a3a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9c08071a90 tx=0x7f9bfc006c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.519+0000 7f9c19578b80 1 -- 172.21.15.49:0/3405030233 shutdown_connections 2023-10-08T09:15:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.523+0000 7f9c19578b80 1 -- 172.21.15.49:0/3405030233 wait complete. 2023-10-08T09:15:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 -- start start 2023-10-08T09:15:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 Processor -- start 2023-10-08T09:15:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 0x5640753cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c27a0 0x5640753cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564075314dd0 con 0x5640753c27a0 2023-10-08T09:15:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640752b16b0 con 0x5640753c1e40 2023-10-08T09:15:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640752b1350 con 0x5640753c32b0 2023-10-08T09:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.524+0000 7f9c19578b80 10 client.0 _ll_register_callbacks cb 0x56407510e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.525+0000 7f9c067fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c27a0 0x5640753cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.525+0000 7f9c067fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c27a0 0x5640753cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50272/0 (socket says 172.21.15.49:50272) 2023-10-08T09:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.525+0000 7f9c067fc700 1 -- 172.21.15.49:0/2148679392 learned_addr learned my addr 172.21.15.49:0/2148679392 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.525+0000 7f9c077fe700 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9c067fc700 1 -- 172.21.15.49:0/2148679392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 msgr2=0x5640753d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9c067fc700 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9c067fc700 1 -- 172.21.15.49:0/2148679392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 msgr2=0x5640753cb810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9c067fc700 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 0x5640753cb810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9c067fc700 1 -- 172.21.15.49:0/2148679392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0810da70 con 0x5640753c27a0 2023-10-08T09:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9c067fc700 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c27a0 0x5640753cbd50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c000961d0 tx=0x7f9c00099e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af28fbad9aa54983 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9beaff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9beaff5700 1 -- 172.21.15.49:0/2148679392 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c000ca5c0 con 0x5640753c27a0 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640752dce50 con 0x5640753c27a0 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.526+0000 7f9beaff5700 1 -- 172.21.15.49:0/2148679392 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9c000991c0 con 0x5640753c27a0 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.527+0000 7f9beaff5700 1 -- 172.21.15.49:0/2148679392 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c00099380 con 0x5640753c27a0 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.527+0000 7f9c19578b80 10 client.15813 fetch_fsmap learned FSMap version 32 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.527+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564075128c00 con 0x5640753c27a0 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.528+0000 7f9beaff5700 1 -- 172.21.15.49:0/2148679392 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640752dce50 con 0x5640753c27a0 2023-10-08T09:15:32.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.528+0000 7f9beaff5700 1 -- 172.21.15.49:0/2148679392 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9c00095cd0 con 0x5640753c27a0 2023-10-08T09:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.528+0000 7f9c19578b80 10 client.15813 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T09:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.528+0000 7f9c19578b80 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.528+0000 7f9c19578b80 20 client.15813 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.529+0000 7f9c19578b80 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5640753d77a0 0x5640753d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.529+0000 7f9c19578b80 4 client.15813 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.529+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56407530ddb0 con 0x5640753d77a0 2023-10-08T09:15:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.529+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5640753d77a0 0x5640753d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.536+0000 7f9beaff5700 10 client.15813 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.536+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5640753d77a0 0x5640753d9bc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339e55fd71c64c06 server_cookie=b63918756407fa92 in_seq=0 out_seq=0 2023-10-08T09:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.537+0000 7f9beaff5700 1 -- 172.21.15.49:0/2148679392 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56407530ddb0 con 0x5640753d77a0 2023-10-08T09:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.537+0000 7f9beaff5700 10 client.15813 handle_command_reply: tid=0 2023-10-08T09:15:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.537+0000 7f9c19578b80 1 client.15813 shutdown 2023-10-08T09:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.537+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c27a0 msgr2=0x5640753cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.537+0000 7f9c19578b80 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c27a0 0x5640753cbd50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9c000961d0 tx=0x7f9c00099e10 comp rx=0 tx=0).stop 2023-10-08T09:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.538+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 shutdown_connections 2023-10-08T09:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.538+0000 7f9c19578b80 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5640753d77a0 0x5640753d9bc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.538+0000 7f9c19578b80 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.538+0000 7f9c19578b80 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 0x5640753cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.538+0000 7f9c19578b80 1 --2- 172.21.15.49:0/2148679392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c27a0 0x5640753cbd50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.538+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 >> 172.21.15.49:0/2148679392 conn(0x564075336200 msgr2=0x5640753bc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.541+0000 7f9beaff5700 0 client.15813 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.541+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 shutdown_connections 2023-10-08T09:15:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.541+0000 7f9c19578b80 1 -- 172.21.15.49:0/2148679392 wait complete. 2023-10-08T09:15:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.545+0000 7f9c19578b80 20 client.15813 trim_cache size 0 max 16384 2023-10-08T09:15:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 Processor -- start 2023-10-08T09:15:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 -- start start 2023-10-08T09:15:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 0x5640753c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 0x5640753c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c32b0 0x5640753d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640751958c0 con 0x5640753c32b0 2023-10-08T09:15:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407530ddb0 con 0x5640753c1e40 2023-10-08T09:15:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.570+0000 7f9c19578b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640752dce50 con 0x5640753c27a0 2023-10-08T09:15:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c32b0 0x5640753d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c077fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c32b0 0x5640753d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50284/0 (socket says 172.21.15.49:50284) 2023-10-08T09:15:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c077fe700 1 -- 172.21.15.49:0/1519373874 learned_addr learned my addr 172.21.15.49:0/1519373874 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c067fc700 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 0x5640753c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 0x5640753c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c077fe700 1 -- 172.21.15.49:0/1519373874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 msgr2=0x5640753c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c077fe700 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 0x5640753c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c077fe700 1 -- 172.21.15.49:0/1519373874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 msgr2=0x5640753c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.572+0000 7f9c077fe700 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 0x5640753c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.573+0000 7f9c077fe700 1 -- 172.21.15.49:0/1519373874 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640751661f0 con 0x5640753c32b0 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.575+0000 7f9c077fe700 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c32b0 0x5640753d2cb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005b40 tx=0x7f9bf8005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b642f454f26a8c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.576+0000 7f9bedffb700 1 -- 172.21.15.49:0/1519373874 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80058e0 con 0x5640753c32b0 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.576+0000 7f9bedffb700 1 -- 172.21.15.49:0/1519373874 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bf8005670 con 0x5640753c32b0 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.576+0000 7f9bedffb700 1 -- 172.21.15.49:0/1519373874 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf803c430 con 0x5640753c32b0 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.577+0000 7f9c19578b80 1 -- 172.21.15.49:0/1519373874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c32b0 msgr2=0x5640753d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.577+0000 7f9c19578b80 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c32b0 0x5640753d2cb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8005b40 tx=0x7f9bf8005ad0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.580+0000 7f9c19578b80 1 -- 172.21.15.49:0/1519373874 shutdown_connections 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.580+0000 7f9c19578b80 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c32b0 0x5640753d2cb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.580+0000 7f9c19578b80 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c1e40 0x5640753c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.580+0000 7f9c19578b80 1 --2- 172.21.15.49:0/1519373874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c27a0 0x5640753c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.580+0000 7f9c19578b80 1 -- 172.21.15.49:0/1519373874 >> 172.21.15.49:0/1519373874 conn(0x564075336200 msgr2=0x5640753c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.586+0000 7f9c19578b80 1 -- 172.21.15.49:0/1519373874 shutdown_connections 2023-10-08T09:15:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.587+0000 7f9c19578b80 1 -- 172.21.15.49:0/1519373874 wait complete. 2023-10-08T09:15:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.587+0000 7f9c19578b80 1 -- start start 2023-10-08T09:15:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.587+0000 7f9c19578b80 1 Processor -- start 2023-10-08T09:15:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.588+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.588+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c27a0 0x5640753d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.588+0000 7f9c19578b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.588+0000 7f9c19578b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640753d1b40 con 0x5640753c1e40 2023-10-08T09:15:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.588+0000 7f9c19578b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640753d1cb0 con 0x5640753c27a0 2023-10-08T09:15:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.588+0000 7f9c19578b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640753d1e70 con 0x5640753c32b0 2023-10-08T09:15:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.588+0000 7f9c19578b80 10 client.0 _ll_register_callbacks cb 0x7f9c0009a3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.589+0000 7f9c06ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.589+0000 7f9c06ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50288/0 (socket says 172.21.15.49:50288) 2023-10-08T09:15:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.589+0000 7f9c06ffd700 1 -- 172.21.15.49:0/615956882 learned_addr learned my addr 172.21.15.49:0/615956882 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.589+0000 7f9c077fe700 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.591+0000 7f9c06ffd700 1 -- 172.21.15.49:0/615956882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 msgr2=0x5640753d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.591+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.591+0000 7f9c06ffd700 1 -- 172.21.15.49:0/615956882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c27a0 msgr2=0x5640753d05f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.591+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c27a0 0x5640753d05f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.591+0000 7f9c06ffd700 1 -- 172.21.15.49:0/615956882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56407513fd80 con 0x5640753c1e40 2023-10-08T09:15:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.591+0000 7f9c06ffd700 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753d00b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5640753da490 tx=0x7f9bfc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56de2e864e09a615 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.591+0000 7f9beaff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.592+0000 7f9beaff5700 1 -- 172.21.15.49:0/615956882 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc013e30 con 0x5640753c1e40 2023-10-08T09:15:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.592+0000 7f9beaff5700 1 -- 172.21.15.49:0/615956882 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9bfc0283e0 con 0x5640753c1e40 2023-10-08T09:15:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.592+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bf8005500 con 0x5640753c1e40 2023-10-08T09:15:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.594+0000 7f9beaff5700 1 -- 172.21.15.49:0/615956882 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bfc0285a0 con 0x5640753c1e40 2023-10-08T09:15:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.594+0000 7f9c19578b80 10 client.15819 fetch_fsmap learned FSMap version 32 2023-10-08T09:15:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.594+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56407513e250 con 0x5640753c1e40 2023-10-08T09:15:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9beaff5700 1 -- 172.21.15.49:0/615956882 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bf8005500 con 0x5640753c1e40 2023-10-08T09:15:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9beaff5700 1 -- 172.21.15.49:0/615956882 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 32) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9bfc0338b0 con 0x5640753c1e40 2023-10-08T09:15:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9c19578b80 10 client.15819 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T09:15:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9c19578b80 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9c19578b80 20 client.15819 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9c19578b80 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56407542bcb0 0x56407542e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9c19578b80 4 client.15819 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:15:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.596+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56407530dad0 con 0x56407542bcb0 2023-10-08T09:15:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.599+0000 7f9c067fc700 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56407542bcb0 0x56407542e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.599+0000 7f9c067fc700 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56407542bcb0 0x56407542e110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=315315b3c621a1e server_cookie=ab622f074874ea02 in_seq=0 out_seq=0 2023-10-08T09:15:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.600+0000 7f9beaff5700 10 client.15819 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.641+0000 7f9beaff5700 1 -- 172.21.15.49:0/615956882 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56407530dad0 con 0x56407542bcb0 2023-10-08T09:15:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.641+0000 7f9beaff5700 10 client.15819 handle_command_reply: tid=0 2023-10-08T09:15:32.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.641+0000 7f9c19578b80 1 client.15819 shutdown 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.642+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 msgr2=0x5640753d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.642+0000 7f9c19578b80 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753d00b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x5640753da490 tx=0x7f9bfc0047b0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 shutdown_connections 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9c19578b80 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56407542bcb0 0x56407542e110 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9c19578b80 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5640753c32b0 0x5640753d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9c19578b80 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5640753c1e40 0x5640753d00b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9c19578b80 1 --2- 172.21.15.49:0/615956882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5640753c27a0 0x5640753d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 >> 172.21.15.49:0/615956882 conn(0x564075336200 msgr2=0x5640753c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9beaff5700 0 client.15819 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.643+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 shutdown_connections 2023-10-08T09:15:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.644+0000 7f9c19578b80 1 -- 172.21.15.49:0/615956882 wait complete. 2023-10-08T09:15:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:15:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:15:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:15:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:15:32.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.644+0000 7f9c19578b80 20 client.15819 trim_cache size 0 max 16384 2023-10-08T09:15:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bf00a1550 msgr2=0x7f9bf00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bf00a1550 0x7f9bf00a3a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9c08071a90 tx=0x7f9bfc006c10 comp rx=0 tx=0).stop 2023-10-08T09:15:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0810cb30 msgr2=0x7f9c0819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:32.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0810cb30 0x7f9c0819d3f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000b840 tx=0x7f9c0000bb50 comp rx=0 tx=0).stop 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 shutdown_connections 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bf00a1550 0x7f9bf00a3a10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c08076a40 0x7f9c08078ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c0810a160 0x7f9c0819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.646+0000 7f9c0d38f700 1 --2- 172.21.15.49:0/238130055 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c0810cb30 0x7f9c0819d3f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.647+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 >> 172.21.15.49:0/238130055 conn(0x7f9c0806d770 msgr2=0x7f9c0810a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.647+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 shutdown_connections 2023-10-08T09:15:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:32.647+0000 7f9c0d38f700 1 -- 172.21.15.49:0/238130055 wait complete. 2023-10-08T09:15:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:33 smithi184 ceph-mon[103675]: pgmap v348: 129 pgs: 129 active+clean; 1.0 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 91 MiB/s wr, 575 op/s 2023-10-08T09:15:33.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:33 smithi132 ceph-mon[103389]: pgmap v348: 129 pgs: 129 active+clean; 1.0 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 91 MiB/s wr, 575 op/s 2023-10-08T09:15:33.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:33 smithi049 ceph-mon[99485]: pgmap v348: 129 pgs: 129 active+clean; 1.0 GiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 91 MiB/s wr, 575 op/s 2023-10-08T09:15:34.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 10.0 s, 2711.3 tps, lat 37.403 ms stddev 62.577 2023-10-08T09:15:35.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:35 smithi132 ceph-mon[103389]: pgmap v349: 129 pgs: 129 active+clean; 1.1 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 100 MiB/s wr, 760 op/s 2023-10-08T09:15:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:35 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:35 smithi049 ceph-mon[99485]: pgmap v349: 129 pgs: 129 active+clean; 1.1 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 100 MiB/s wr, 760 op/s 2023-10-08T09:15:35.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:35 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:35.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:35 smithi184 ceph-mon[103675]: pgmap v349: 129 pgs: 129 active+clean; 1.1 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 100 MiB/s wr, 760 op/s 2023-10-08T09:15:35.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:35 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:15:36.088 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:15:36.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 -- 172.21.15.49:0/3033963194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 msgr2=0x7f684c10fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 --2- 172.21.15.49:0/3033963194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 0x7f684c10fff0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f68400098f0 tx=0x7f6840009c00 comp rx=0 tx=0).stop 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 -- 172.21.15.49:0/3033963194 shutdown_connections 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 --2- 172.21.15.49:0/3033963194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c108900 0x7f684c10acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 --2- 172.21.15.49:0/3033963194 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 0x7f684c10fff0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 --2- 172.21.15.49:0/3033963194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c10b230 0x7f684c10d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 -- 172.21.15.49:0/3033963194 >> 172.21.15.49:0/3033963194 conn(0x7f684c06d790 msgr2=0x7f684c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 -- 172.21.15.49:0/3033963194 shutdown_connections 2023-10-08T09:15:36.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.298+0000 7f68510c7700 1 -- 172.21.15.49:0/3033963194 wait complete. 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 Processor -- start 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 -- start start 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c108900 0x7f684c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c10b230 0x7f684c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 0x7f684c1a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1a6a20 con 0x7f684c10db60 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1a6b60 con 0x7f684c108900 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.299+0000 7f68510c7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1a6cf0 con 0x7f684c10b230 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c10b230 0x7f684c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c108900 0x7f684c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c10b230 0x7f684c19f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37336/0 (socket says 172.21.15.49:37336) 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c108900 0x7f684c19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:39914/0 (socket says 172.21.15.49:39914) 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 -- 172.21.15.49:0/1423734919 learned_addr learned my addr 172.21.15.49:0/1423734919 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684b59e700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 0x7f684c1a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 -- 172.21.15.49:0/1423734919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c108900 msgr2=0x7f684c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c108900 0x7f684c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 -- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 msgr2=0x7f684c1a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 0x7f684c1a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 -- 172.21.15.49:0/1423734919 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6840009520 con 0x7f684c10b230 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684b59e700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 0x7f684c1a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f684a59c700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c10b230 0x7f684c19f480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f683c00e9a0 tx=0x7f683c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f6833fff700 1 -- 172.21.15.49:0/1423734919 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c009790 con 0x7f684c10b230 2023-10-08T09:15:36.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c1a6fd0 con 0x7f684c10b230 2023-10-08T09:15:36.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f6833fff700 1 -- 172.21.15.49:0/1423734919 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683c00cbd0 con 0x7f684c10b230 2023-10-08T09:15:36.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f6833fff700 1 -- 172.21.15.49:0/1423734919 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c018680 con 0x7f684c10b230 2023-10-08T09:15:36.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.300+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1a7450 con 0x7f684c10b230 2023-10-08T09:15:36.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.306+0000 7f685d2b0b80 1 Processor -- start 2023-10-08T09:15:36.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.306+0000 7f6833fff700 1 -- 172.21.15.49:0/1423734919 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f683c010430 con 0x7f684c10b230 2023-10-08T09:15:36.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f685d2b0b80 1 -- start start 2023-10-08T09:15:36.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f6833fff700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f68340a1500 0x7f68340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f79a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6ed8460 con 0x55f3d6f7a010 2023-10-08T09:15:36.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6ec9c20 con 0x55f3d6f700a0 2023-10-08T09:15:36.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6ec7540 con 0x55f3d6f70a00 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f79a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f79a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37342/0 (socket says 172.21.15.49:37342) 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f684a59c700 1 -- 172.21.15.49:0/4074832097 learned_addr learned my addr 172.21.15.49:0/4074832097 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.307+0000 7f684ad9d700 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f684ad9d700 1 -- 172.21.15.49:0/4074832097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 msgr2=0x55f3d6f79a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f684ad9d700 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f79a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f684ad9d700 1 -- 172.21.15.49:0/4074832097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 msgr2=0x55f3d6f7a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f684ad9d700 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f684ad9d700 1 -- 172.21.15.49:0/4074832097 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3d6cedde0 con 0x55f3d6f700a0 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f6833fff700 1 -- 172.21.15.49:0/1423734919 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f683c014070 con 0x7f684c10b230 2023-10-08T09:15:36.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f684ad9d700 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f704c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6844011580 tx=0x7f68440099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d318b35aa8ff3be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f684b59e700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f68340a1500 0x7f68340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f68317fa700 1 -- 172.21.15.49:0/4074832097 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400e040 con 0x55f3d6f700a0 2023-10-08T09:15:36.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f68317fa700 1 -- 172.21.15.49:0/4074832097 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f684400f040 con 0x55f3d6f700a0 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.308+0000 7f68317fa700 1 -- 172.21.15.49:0/4074832097 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844007e80 con 0x55f3d6f700a0 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4074832097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 msgr2=0x55f3d6f704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f704c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6844011580 tx=0x7f68440099e0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f684b59e700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f68340a1500 0x7f68340a39c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f684000fa40 tx=0x7f6840019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4074832097 shutdown_connections 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f7a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f704c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4074832097 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f79a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.309+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4074832097 >> 172.21.15.49:0/4074832097 conn(0x55f3d6ee4260 msgr2=0x55f3d6f6ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:36.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.310+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4074832097 shutdown_connections 2023-10-08T09:15:36.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.310+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4074832097 wait complete. 2023-10-08T09:15:36.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 -- start start 2023-10-08T09:15:36.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 Processor -- start 2023-10-08T09:15:36.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f700a0 0x55f3d6f825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f70a00 0x55f3d6f82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f89b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6ec2e30 con 0x55f3d6f70a00 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6e5f710 con 0x55f3d6f7a010 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6e5f3b0 con 0x55f3d6f700a0 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.311+0000 7f685d2b0b80 10 client.0 _ll_register_callbacks cb 0x55f3d6cbc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f70a00 0x55f3d6f82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f70a00 0x55f3d6f82b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50314/0 (socket says 172.21.15.49:50314) 2023-10-08T09:15:36.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684a59c700 1 -- 172.21.15.49:0/3123864443 learned_addr learned my addr 172.21.15.49:0/3123864443 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684b59e700 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f89b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684b59e700 1 -- 172.21.15.49:0/3123864443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f700a0 msgr2=0x55f3d6f825e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684b59e700 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f700a0 0x55f3d6f825e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684b59e700 1 -- 172.21.15.49:0/3123864443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f70a00 msgr2=0x55f3d6f82b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:36.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684b59e700 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f70a00 0x55f3d6f82b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.312+0000 7f684b59e700 1 -- 172.21.15.49:0/3123864443 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6844011230 con 0x55f3d6f7a010 2023-10-08T09:15:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.313+0000 7f684b59e700 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f89b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f684000fb70 tx=0x7f6840009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=837f1f7c8fde21ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.313+0000 7f682eff5700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.313+0000 7f682eff5700 1 -- 172.21.15.49:0/3123864443 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840029070 con 0x55f3d6f7a010 2023-10-08T09:15:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.313+0000 7f682eff5700 1 -- 172.21.15.49:0/3123864443 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6840005250 con 0x55f3d6f7a010 2023-10-08T09:15:36.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.313+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3d6e8aeb0 con 0x55f3d6f7a010 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.314+0000 7f682eff5700 1 -- 172.21.15.49:0/3123864443 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6840005410 con 0x55f3d6f7a010 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.314+0000 7f685d2b0b80 10 client.25457 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.314+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f3d6cd6c60 con 0x55f3d6f7a010 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.314+0000 7f682eff5700 1 -- 172.21.15.49:0/3123864443 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3d6e8aeb0 con 0x55f3d6f7a010 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.315+0000 7f682eff5700 1 -- 172.21.15.49:0/3123864443 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6840035070 con 0x55f3d6f7a010 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.315+0000 7f685d2b0b80 10 client.25457 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.315+0000 7f685d2b0b80 10 client.25457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.315+0000 7f685d2b0b80 20 client.25457 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:36.316 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.315+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f8e460 0x55f3d6f90880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.315+0000 7f685d2b0b80 4 client.25457 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.315+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3d6ebbe10 con 0x55f3d6f8e460 2023-10-08T09:15:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.316+0000 7f684ad9d700 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f8e460 0x55f3d6f90880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.316+0000 7f684ad9d700 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f8e460 0x55f3d6f90880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788b987a476410c server_cookie=c031d5638e9ceee5 in_seq=0 out_seq=0 2023-10-08T09:15:36.317 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.317+0000 7f682eff5700 10 client.25457 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.317+0000 7f682eff5700 1 -- 172.21.15.49:0/3123864443 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f3d6ebbe10 con 0x55f3d6f8e460 2023-10-08T09:15:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.317+0000 7f682eff5700 10 client.25457 handle_command_reply: tid=0 2023-10-08T09:15:36.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.317+0000 7f685d2b0b80 1 client.25457 shutdown 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 msgr2=0x55f3d6f89b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f89b60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f684000fb70 tx=0x7f6840009260 comp rx=0 tx=0).stop 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 shutdown_connections 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f8e460 0x55f3d6f90880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f700a0 0x55f3d6f825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f89b60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f682eff5700 0 client.25457 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/3123864443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f70a00 0x55f3d6f82b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 >> 172.21.15.49:0/3123864443 conn(0x55f3d6ee4260 msgr2=0x55f3d6f6a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 shutdown_connections 2023-10-08T09:15:36.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 1 -- 172.21.15.49:0/3123864443 wait complete. 2023-10-08T09:15:36.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.318+0000 7f685d2b0b80 20 client.25457 trim_cache size 0 max 16384 2023-10-08T09:15:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 Processor -- start 2023-10-08T09:15:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 -- start start 2023-10-08T09:15:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f80540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f89b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f81af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6d43920 con 0x55f3d6f7a010 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6f67ae0 con 0x55f3d6f700a0 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6f71040 con 0x55f3d6f70a00 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684b59e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f81af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f89b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f80540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684b59e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f81af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50324/0 (socket says 172.21.15.49:50324) 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684b59e700 1 -- 172.21.15.49:0/4035893034 learned_addr learned my addr 172.21.15.49:0/4035893034 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f89b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37356/0 (socket says 172.21.15.49:37356) 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 -- 172.21.15.49:0/4035893034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 msgr2=0x55f3d6f80540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 --2- 172.21.15.49:0/4035893034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f80540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 -- 172.21.15.49:0/4035893034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 msgr2=0x55f3d6f81af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 --2- 172.21.15.49:0/4035893034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f81af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 -- 172.21.15.49:0/4035893034 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3d6d14250 con 0x55f3d6f70a00 2023-10-08T09:15:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.345+0000 7f684a59c700 1 --2- 172.21.15.49:0/4035893034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f89b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f683c007f60 tx=0x7f683c08fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443049306b6d184 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.346+0000 7f6831ffb700 1 -- 172.21.15.49:0/4035893034 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0cd120 con 0x55f3d6f70a00 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.346+0000 7f6831ffb700 1 -- 172.21.15.49:0/4035893034 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f683c094e90 con 0x55f3d6f70a00 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.346+0000 7f6831ffb700 1 -- 172.21.15.49:0/4035893034 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c0a2610 con 0x55f3d6f70a00 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4035893034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 msgr2=0x55f3d6f89b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4035893034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f89b20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f683c007f60 tx=0x7f683c08fa40 comp rx=0 tx=0).stop 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4035893034 shutdown_connections 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4035893034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f7a010 0x55f3d6f81af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4035893034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f700a0 0x55f3d6f80540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/4035893034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f89b20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4035893034 >> 172.21.15.49:0/4035893034 conn(0x55f3d6ee4260 msgr2=0x55f3d6f66f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:36.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4035893034 shutdown_connections 2023-10-08T09:15:36.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.347+0000 7f685d2b0b80 1 -- 172.21.15.49:0/4035893034 wait complete. 2023-10-08T09:15:36.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 -- start start 2023-10-08T09:15:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 Processor -- start 2023-10-08T09:15:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f700a0 0x55f3d6f86e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f882c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6f88800 con 0x55f3d6f700a0 2023-10-08T09:15:36.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6f88970 con 0x55f3d6f7a010 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3d6f88ae0 con 0x55f3d6f70a00 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f685d2b0b80 10 client.0 _ll_register_callbacks cb 0x7f684000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.348+0000 7f684a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f87350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37362/0 (socket says 172.21.15.49:37362) 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684a59c700 1 -- 172.21.15.49:0/1043306158 learned_addr learned my addr 172.21.15.49:0/1043306158 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684b59e700 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f882c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684b59e700 1 -- 172.21.15.49:0/1043306158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 msgr2=0x55f3d6f87350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684b59e700 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f87350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684b59e700 1 -- 172.21.15.49:0/1043306158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f700a0 msgr2=0x55f3d6f86e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684b59e700 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f700a0 0x55f3d6f86e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684b59e700 1 -- 172.21.15.49:0/1043306158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1a7450 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f684b59e700 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f882c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f684000fb70 tx=0x7f68400057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59c94a49f56f31f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f682eff5700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.349+0000 7f682eff5700 1 -- 172.21.15.49:0/1043306158 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840029040 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f684400f040 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f682eff5700 1 -- 172.21.15.49:0/1043306158 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6840003680 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f682eff5700 1 -- 172.21.15.49:0/1043306158 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6840003840 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 10 client.25463 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f3d6cec2b0 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f682eff5700 1 -- 172.21.15.49:0/1043306158 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f684400f040 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f682eff5700 1 -- 172.21.15.49:0/1043306158 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6840035050 con 0x55f3d6f7a010 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 10 client.25463 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 10 client.25463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:36.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 20 client.25463 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f71300 0x55f3d6f73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 4 client.25463 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.350+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f3d6ebbb30 con 0x55f3d6f71300 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f684ad9d700 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f71300 0x55f3d6f73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f682eff5700 10 client.25463 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f684ad9d700 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f71300 0x55f3d6f73720 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f334c87850940b server_cookie=3a41b28df55d061a in_seq=0 out_seq=0 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f682eff5700 1 -- 172.21.15.49:0/1043306158 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f3d6ebbb30 con 0x55f3d6f71300 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f682eff5700 10 client.25463 handle_command_reply: tid=0 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 client.25463 shutdown 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 msgr2=0x55f3d6f882c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f882c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f684000fb70 tx=0x7f68400057f0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 shutdown_connections 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f3d6f71300 0x55f3d6f73720 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f682eff5700 0 client.25463 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f3d6f7a010 0x55f3d6f882c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f3d6f700a0 0x55f3d6f86e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 --2- 172.21.15.49:0/1043306158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f3d6f70a00 0x55f3d6f87350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.351+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 >> 172.21.15.49:0/1043306158 conn(0x55f3d6ee4260 msgr2=0x55f3d6f81aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.352+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 shutdown_connections 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.352+0000 7f685d2b0b80 1 -- 172.21.15.49:0/1043306158 wait complete. 2023-10-08T09:15:36.357 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:15:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "no active scrubs running", 2023-10-08T09:15:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": {} 2023-10-08T09:15:36.358 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f685d2b0b80 20 client.25463 trim_cache size 0 max 16384 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f68340a1500 msgr2=0x7f68340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f68340a1500 0x7f68340a39c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f684000fa40 tx=0x7f6840019040 comp rx=0 tx=0).stop 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c10b230 msgr2=0x7f684c19f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c10b230 0x7f684c19f480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f683c00e9a0 tx=0x7f683c00ecb0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 shutdown_connections 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f68340a1500 0x7f68340a39c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f684c10db60 0x7f684c1a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.359 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f684c108900 0x7f684c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 --2- 172.21.15.49:0/1423734919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f684c10b230 0x7f684c19f480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.353+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 >> 172.21.15.49:0/1423734919 conn(0x7f684c06d790 msgr2=0x7f684c108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:36.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.354+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 shutdown_connections 2023-10-08T09:15:36.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.354+0000 7f68510c7700 1 -- 172.21.15.49:0/1423734919 wait complete. 2023-10-08T09:15:36.373 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-08T09:15:36.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:15:36.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 -- 172.21.15.49:0/1436069473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34108860 msgr2=0x7fdd3410f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1436069473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34108860 0x7fdd3410f110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00b0a0 tx=0x7fdd2c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 -- 172.21.15.49:0/1436069473 shutdown_connections 2023-10-08T09:15:36.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1436069473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34108860 0x7fdd3410f110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1436069473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34106cd0 0x7fdd341070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1436069473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd34107ec0 0x7fdd34108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.570 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 -- 172.21.15.49:0/1436069473 >> 172.21.15.49:0/1436069473 conn(0x7fdd34075c70 msgr2=0x7fdd340780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:36.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 -- 172.21.15.49:0/1436069473 shutdown_connections 2023-10-08T09:15:36.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.569+0000 7fdd392ff700 1 -- 172.21.15.49:0/1436069473 wait complete. 2023-10-08T09:15:36.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.570+0000 7fdd392ff700 1 Processor -- start 2023-10-08T09:15:36.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.570+0000 7fdd392ff700 1 -- start start 2023-10-08T09:15:36.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.570+0000 7fdd392ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34106cd0 0x7fdd3419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.570+0000 7fdd392ff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd34107ec0 0x7fdd3419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.570+0000 7fdd392ff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34108860 0x7fdd341a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd392ff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341a4840 con 0x7fdd34107ec0 2023-10-08T09:15:36.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd392ff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341a49b0 con 0x7fdd34108860 2023-10-08T09:15:36.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd392ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd341a4b70 con 0x7fdd34106cd0 2023-10-08T09:15:36.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34108860 0x7fdd341a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd327fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd34107ec0 0x7fdd3419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34108860 0x7fdd341a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:39958/0 (socket says 172.21.15.49:39958) 2023-10-08T09:15:36.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 -- 172.21.15.49:0/1087779368 learned_addr learned my addr 172.21.15.49:0/1087779368 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:36.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd32ffd700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34106cd0 0x7fdd3419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 -- 172.21.15.49:0/1087779368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34106cd0 msgr2=0x7fdd3419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34106cd0 0x7fdd3419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 -- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd34107ec0 msgr2=0x7fdd3419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd34107ec0 0x7fdd3419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.571+0000 7fdd337fe700 1 -- 172.21.15.49:0/1087779368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c009d00 con 0x7fdd34108860 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.572+0000 7fdd337fe700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34108860 0x7fdd341a4270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c007bc0 tx=0x7fdd2c00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.572+0000 7fdd1bfff700 1 -- 172.21.15.49:0/1087779368 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c00e040 con 0x7fdd34108860 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.572+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd341a4df0 con 0x7fdd34108860 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.572+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd341a5200 con 0x7fdd34108860 2023-10-08T09:15:36.574 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.572+0000 7fdd1bfff700 1 -- 172.21.15.49:0/1087779368 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd2c00be90 con 0x7fdd34108860 2023-10-08T09:15:36.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.572+0000 7fdd1bfff700 1 -- 172.21.15.49:0/1087779368 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c01ddd0 con 0x7fdd34108860 2023-10-08T09:15:36.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.573+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd20005320 con 0x7fdd34108860 2023-10-08T09:15:36.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.575+0000 7fdd1bfff700 1 -- 172.21.15.49:0/1087779368 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdd2c01d470 con 0x7fdd34108860 2023-10-08T09:15:36.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.575+0000 7fdd1bfff700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd1c0a14d0 0x7fdd1c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:36.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.576+0000 7fdd32ffd700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd1c0a14d0 0x7fdd1c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:36.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.576+0000 7fdd1bfff700 1 -- 172.21.15.49:0/1087779368 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdd2c0cde90 con 0x7fdd34108860 2023-10-08T09:15:36.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.577+0000 7fdd32ffd700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd1c0a14d0 0x7fdd1c0a3990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdd3419e1d0 tx=0x7fdd28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:36.580 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:36.580+0000 7fdd1bfff700 1 -- 172.21.15.49:0/1087779368 <== mon.2 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd2c097280 con 0x7fdd34108860 2023-10-08T09:15:37.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.013+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd200058a0 con 0x7fdd34108860 2023-10-08T09:15:37.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.014+0000 7fdd1bfff700 1 -- 172.21.15.49:0/1087779368 <== mon.2 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11393 (secure 0 0 0) 0x7fdd2c09a8a0 con 0x7fdd34108860 2023-10-08T09:15:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:15:37.015 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:15:34.244348+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:15:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.018+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd1c0a14d0 msgr2=0x7fdd1c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.018+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd1c0a14d0 0x7fdd1c0a3990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdd3419e1d0 tx=0x7fdd28009260 comp rx=0 tx=0).stop 2023-10-08T09:15:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.018+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34108860 msgr2=0x7fdd341a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.018+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34108860 0x7fdd341a4270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c007bc0 tx=0x7fdd2c00bce0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 shutdown_connections 2023-10-08T09:15:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd1c0a14d0 0x7fdd1c0a3990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd34106cd0 0x7fdd3419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd34108860 0x7fdd341a4270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 --2- 172.21.15.49:0/1087779368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd34107ec0 0x7fdd3419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 >> 172.21.15.49:0/1087779368 conn(0x7fdd34075c70 msgr2=0x7fdd3410b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 shutdown_connections 2023-10-08T09:15:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.019+0000 7fdd392ff700 1 -- 172.21.15.49:0/1087779368 wait complete. 2023-10-08T09:15:37.021 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 33 2023-10-08T09:15:37.046 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-08T09:15:37.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.278+0000 7fd91b88c700 1 -- 172.21.15.49:0/1715745919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9140ff080 msgr2=0x7fd9141014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.278+0000 7fd91b88c700 1 --2- 172.21.15.49:0/1715745919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9140ff080 0x7fd9141014b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd904009990 tx=0x7fd904009ca0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.280+0000 7fd91b88c700 1 -- 172.21.15.49:0/1715745919 shutdown_connections 2023-10-08T09:15:37.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.280+0000 7fd91b88c700 1 --2- 172.21.15.49:0/1715745919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9140ff080 0x7fd9141014b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.280+0000 7fd91b88c700 1 --2- 172.21.15.49:0/1715745919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9141043c0 0x7fd914104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.280+0000 7fd91b88c700 1 --2- 172.21.15.49:0/1715745919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9141019f0 0x7fd914103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.280+0000 7fd91b88c700 1 -- 172.21.15.49:0/1715745919 >> 172.21.15.49:0/1715745919 conn(0x7fd9140fac40 msgr2=0x7fd9140fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.280+0000 7fd91b88c700 1 -- 172.21.15.49:0/1715745919 shutdown_connections 2023-10-08T09:15:37.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.280+0000 7fd91b88c700 1 -- 172.21.15.49:0/1715745919 wait complete. 2023-10-08T09:15:37.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 Processor -- start 2023-10-08T09:15:37.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 -- start start 2023-10-08T09:15:37.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff080 0x7fd914070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9141019f0 0x7fd9140708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9141043c0 0x7fd9140718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914071e90 con 0x7fd9141019f0 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914072000 con 0x7fd9140ff080 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.281+0000 7fd91b88c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140721c0 con 0x7fd9141043c0 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd918e27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9141019f0 0x7fd9140708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919e29700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9141043c0 0x7fd9140718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd918e27700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9141019f0 0x7fd9140708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50356/0 (socket says 172.21.15.49:50356) 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919628700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff080 0x7fd914070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd918e27700 1 -- 172.21.15.49:0/3525247164 learned_addr learned my addr 172.21.15.49:0/3525247164 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919628700 1 -- 172.21.15.49:0/3525247164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9141043c0 msgr2=0x7fd9140718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919628700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9141043c0 0x7fd9140718c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919628700 1 -- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9141019f0 msgr2=0x7fd9140708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919628700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9141019f0 0x7fd9140708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919628700 1 -- 172.21.15.49:0/3525247164 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9040095f0 con 0x7fd9140ff080 2023-10-08T09:15:37.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.282+0000 7fd919e29700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9141043c0 0x7fd9140718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:37.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.283+0000 7fd919628700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff080 0x7fd914070390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9040098e0 tx=0x7fd9040057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.284+0000 7fd90a7fc700 1 -- 172.21.15.49:0/3525247164 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90401d070 con 0x7fd9140ff080 2023-10-08T09:15:37.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.284+0000 7fd90a7fc700 1 -- 172.21.15.49:0/3525247164 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd904021410 con 0x7fd9140ff080 2023-10-08T09:15:37.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.284+0000 7fd90a7fc700 1 -- 172.21.15.49:0/3525247164 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904017650 con 0x7fd9140ff080 2023-10-08T09:15:37.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.284+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9140723e0 con 0x7fd9140ff080 2023-10-08T09:15:37.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.284+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd914072840 con 0x7fd9140ff080 2023-10-08T09:15:37.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.289+0000 7fd927a75b80 1 Processor -- start 2023-10-08T09:15:37.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.289+0000 7fd927a75b80 1 -- start start 2023-10-08T09:15:37.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b060 0x56267329b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b9c0 0x56267329c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329c800 0x56267329cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd927a75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562673203460 con 0x56267329b060 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd927a75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626731f4c20 con 0x56267329c800 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd927a75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626731f2540 con 0x56267329b9c0 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b060 0x56267329b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b060 0x56267329b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50362/0 (socket says 172.21.15.49:50362) 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 -- 172.21.15.49:0/3507320744 learned_addr learned my addr 172.21.15.49:0/3507320744 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd918e27700 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329c800 0x56267329cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 -- 172.21.15.49:0/3507320744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b9c0 msgr2=0x56267329c270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b9c0 0x56267329c270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 -- 172.21.15.49:0/3507320744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329c800 msgr2=0x56267329cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329c800 0x56267329cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 -- 172.21.15.49:0/3507320744 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562673018de0 con 0x56267329b060 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd918e27700 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329c800 0x56267329cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.290+0000 7fd919e29700 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b060 0x56267329b480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd910009e20 tx=0x7fd91000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eda6b35b03dce3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.291+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/3507320744 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000ec20 con 0x56267329b060 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.291+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/3507320744 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91000ed80 con 0x56267329b060 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.291+0000 7fd8ff7fe700 1 -- 172.21.15.49:0/3507320744 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910012630 con 0x56267329b060 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 -- 172.21.15.49:0/3507320744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b060 msgr2=0x56267329b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b060 0x56267329b480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd910009e20 tx=0x7fd91000e4b0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 -- 172.21.15.49:0/3507320744 shutdown_connections 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329c800 0x56267329cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b060 0x56267329b480 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 --2- 172.21.15.49:0/3507320744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b9c0 0x56267329c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 -- 172.21.15.49:0/3507320744 >> 172.21.15.49:0/3507320744 conn(0x56267320f260 msgr2=0x562673295e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 -- 172.21.15.49:0/3507320744 shutdown_connections 2023-10-08T09:15:37.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 -- 172.21.15.49:0/3507320744 wait complete. 2023-10-08T09:15:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.292+0000 7fd927a75b80 1 -- start start 2023-10-08T09:15:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 1 Processor -- start 2023-10-08T09:15:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd90a7fc700 1 -- 172.21.15.49:0/3525247164 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd90400fa50 con 0x7fd9140ff080 2023-10-08T09:15:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b060 0x5626732b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b9c0 0x5626732b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329c800 0x5626732bd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626731ede30 con 0x56267329b9c0 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267318a710 con 0x56267329b060 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267318a3b0 con 0x56267329c800 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd927a75b80 10 client.0 _ll_register_callbacks cb 0x562672fe7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd919e29700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b060 0x5626732b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd919e29700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b060 0x5626732b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:39978/0 (socket says 172.21.15.49:39978) 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.293+0000 7fd919e29700 1 -- 172.21.15.49:0/1135496337 learned_addr learned my addr 172.21.15.49:0/1135496337 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd919e29700 1 -- 172.21.15.49:0/1135496337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329c800 msgr2=0x5626732bd400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd919e29700 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329c800 0x5626732bd400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd919e29700 1 -- 172.21.15.49:0/1135496337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b9c0 msgr2=0x5626732b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd919e29700 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b9c0 0x5626732b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd919e29700 1 -- 172.21.15.49:0/1135496337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c00e280 con 0x56267329b060 2023-10-08T09:15:37.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd90a7fc700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9000a1590 0x7fd9000a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd919e29700 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b060 0x5626732b5e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd910006d10 tx=0x7fd910012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb3294e39efeaf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd919628700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9000a1590 0x7fd9000a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd8fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1135496337 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000ec20 con 0x56267329b060 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1135496337 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd910012e10 con 0x56267329b060 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.294+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626731b5eb0 con 0x56267329b060 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.295+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1135496337 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd910006da0 con 0x56267329b060 2023-10-08T09:15:37.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.295+0000 7fd90a7fc700 1 -- 172.21.15.49:0/3525247164 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd9040ce130 con 0x7fd9140ff080 2023-10-08T09:15:37.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.295+0000 7fd927a75b80 10 client.25478 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:37.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.295+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fd910009a50 con 0x56267329b060 2023-10-08T09:15:37.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.295+0000 7fd919628700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9000a1590 0x7fd9000a3a50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd904097bc0 tx=0x7fd904095c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1135496337 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626731b5eb0 con 0x56267329b060 2023-10-08T09:15:37.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1135496337 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd910011070 con 0x56267329b060 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd927a75b80 10 client.25478 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd927a75b80 10 client.25478 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd927a75b80 20 client.25478 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x5626732c1bb0 0x5626732c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd927a75b80 4 client.25478 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 --> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626731e6e10 con 0x5626732c1bb0 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.297+0000 7fd918e27700 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x5626732c1bb0 0x5626732c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.298+0000 7fd918e27700 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x5626732c1bb0 0x5626732c3fd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5d3fcf9e9bdc58e server_cookie=193fd96ff8b985ed in_seq=0 out_seq=0 2023-10-08T09:15:37.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.298+0000 7fd8fcff9700 10 client.25478 ms_handle_connect on v2:172.21.15.132:6832/3058040154 2023-10-08T09:15:37.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.298+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1135496337 <== mds.4 v2:172.21.15.132:6832/3058040154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5626731e6e10 con 0x5626732c1bb0 2023-10-08T09:15:37.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.298+0000 7fd8fcff9700 10 client.25478 handle_command_reply: tid=0 2023-10-08T09:15:37.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.298+0000 7fd927a75b80 1 client.25478 shutdown 2023-10-08T09:15:37.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b060 msgr2=0x5626732b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b060 0x5626732b5e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd910006d10 tx=0x7fd910012730 comp rx=0 tx=0).stop 2023-10-08T09:15:37.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 shutdown_connections 2023-10-08T09:15:37.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329c800 0x5626732bd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x5626732c1bb0 0x5626732c3fd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b060 0x5626732b5e80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1135496337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329b9c0 0x5626732b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 >> 172.21.15.49:0/1135496337 conn(0x56267320f260 msgr2=0x5626732954d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.302+0000 7fd8fcff9700 0 client.25478 ms_handle_reset on v2:172.21.15.132:6832/3058040154 2023-10-08T09:15:37.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.303+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 shutdown_connections 2023-10-08T09:15:37.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.304+0000 7fd927a75b80 1 -- 172.21.15.49:0/1135496337 wait complete. 2023-10-08T09:15:37.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.304+0000 7fd927a75b80 20 client.25478 trim_cache size 0 max 16384 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.327+0000 7fd927a75b80 1 Processor -- start 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.327+0000 7fd927a75b80 1 -- start start 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.327+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 0x5626732c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x562673291f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd927a75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267306e920 con 0x56267329c800 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd927a75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626732c16f0 con 0x56267329b9c0 2023-10-08T09:15:37.328 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd927a75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562673290570 con 0x56267329b060 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:39988/0 (socket says 172.21.15.49:39988) 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 -- 172.21.15.49:0/2225937887 learned_addr learned my addr 172.21.15.49:0/2225937887 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd919e29700 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x562673291f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd919e29700 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 0x5626732c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 -- 172.21.15.49:0/2225937887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 msgr2=0x5626732c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 0x5626732c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 -- 172.21.15.49:0/2225937887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 msgr2=0x562673291f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x562673291f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd918e27700 1 -- 172.21.15.49:0/2225937887 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562673001c60 con 0x56267329b9c0 2023-10-08T09:15:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.328+0000 7fd919e29700 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x562673291f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:15:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.329+0000 7fd918e27700 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bd390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000f80 tx=0x7fd90c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0031a2db74e3f82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.329+0000 7fd8fffff700 1 -- 172.21.15.49:0/2225937887 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01c070 con 0x56267329b9c0 2023-10-08T09:15:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.329+0000 7fd8fffff700 1 -- 172.21.15.49:0/2225937887 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd90c00f040 con 0x56267329b9c0 2023-10-08T09:15:37.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.329+0000 7fd8fffff700 1 -- 172.21.15.49:0/2225937887 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00e040 con 0x56267329b9c0 2023-10-08T09:15:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.330+0000 7fd927a75b80 1 -- 172.21.15.49:0/2225937887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 msgr2=0x5626732bd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.330+0000 7fd927a75b80 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bd390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000f80 tx=0x7fd90c00b040 comp rx=0 tx=0).stop 2023-10-08T09:15:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.332+0000 7fd927a75b80 1 -- 172.21.15.49:0/2225937887 shutdown_connections 2023-10-08T09:15:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.332+0000 7fd927a75b80 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 0x5626732c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.332+0000 7fd927a75b80 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x562673291f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.332+0000 7fd927a75b80 1 --2- 172.21.15.49:0/2225937887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bd390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.332+0000 7fd927a75b80 1 -- 172.21.15.49:0/2225937887 >> 172.21.15.49:0/2225937887 conn(0x56267320f260 msgr2=0x562673295dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.332+0000 7fd927a75b80 1 -- 172.21.15.49:0/2225937887 shutdown_connections 2023-10-08T09:15:37.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.332+0000 7fd927a75b80 1 -- 172.21.15.49:0/2225937887 wait complete. 2023-10-08T09:15:37.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.333+0000 7fd927a75b80 1 -- start start 2023-10-08T09:15:37.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.333+0000 7fd927a75b80 1 Processor -- start 2023-10-08T09:15:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.333+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 0x5626732bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.333+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.333+0000 7fd927a75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x5626732bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.334+0000 7fd927a75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626732bcd20 con 0x56267329c800 2023-10-08T09:15:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.334+0000 7fd927a75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626732bce90 con 0x56267329b9c0 2023-10-08T09:15:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.334+0000 7fd927a75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626732bd050 con 0x56267329b060 2023-10-08T09:15:37.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.334+0000 7fd927a75b80 10 client.0 _ll_register_callbacks cb 0x7fd90c009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:37.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.334+0000 7fd918e27700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.334+0000 7fd918e27700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:39994/0 (socket says 172.21.15.49:39994) 2023-10-08T09:15:37.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.334+0000 7fd918e27700 1 -- 172.21.15.49:0/1533930298 learned_addr learned my addr 172.21.15.49:0/1533930298 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.335+0000 7fd918e27700 1 -- 172.21.15.49:0/1533930298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 msgr2=0x5626732bb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.335+0000 7fd918e27700 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 0x5626732bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd918e27700 1 -- 172.21.15.49:0/1533930298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 msgr2=0x5626732bc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd918e27700 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x5626732bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd918e27700 1 -- 172.21.15.49:0/1533930298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c009050 con 0x56267329b9c0 2023-10-08T09:15:37.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd918e27700 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bb820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009960 tx=0x7fd90c007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19f187ea188830a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd8fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:37.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1533930298 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00e040 con 0x56267329b9c0 2023-10-08T09:15:37.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1533930298 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd90c007c70 con 0x56267329b9c0 2023-10-08T09:15:37.337 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.336+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd91000ed80 con 0x56267329b9c0 2023-10-08T09:15:37.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1533930298 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90c007e30 con 0x56267329b9c0 2023-10-08T09:15:37.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 10 client.25484 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:37.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56267303f250 con 0x56267329b9c0 2023-10-08T09:15:37.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1533930298 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91000ed80 con 0x56267329b9c0 2023-10-08T09:15:37.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1533930298 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd90c0255e0 con 0x56267329b9c0 2023-10-08T09:15:37.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 10 client.25484 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:37.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 10 client.25484 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-10-08T09:15:37.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 20 client.25484 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:37.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x562673316af0 0x562673318f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 4 client.25484 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:15:37.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.339+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 --> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626731e6b30 con 0x562673316af0 2023-10-08T09:15:37.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.340+0000 7fd919e29700 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x562673316af0 0x562673318f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.342+0000 7fd919e29700 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x562673316af0 0x562673318f10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c301ae243aa0b8a server_cookie=3fb684d0d0ba1c11 in_seq=0 out_seq=0 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.342+0000 7fd8fcff9700 10 client.25484 ms_handle_connect on v2:172.21.15.132:6832/3058040154 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd8fcff9700 1 -- 172.21.15.49:0/1533930298 <== mds.4 v2:172.21.15.132:6832/3058040154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626731e6b30 con 0x562673316af0 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd8fcff9700 10 client.25484 handle_command_reply: tid=0 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 client.25484 shutdown 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 msgr2=0x5626732bb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bb820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009960 tx=0x7fd90c007500 comp rx=0 tx=0).stop 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 shutdown_connections 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56267329b060 0x5626732bb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:6832/3058040154,v1:172.21.15.132:6833/3058040154] conn(0x562673316af0 0x562673318f10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56267329c800 0x5626732bc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 --2- 172.21.15.49:0/1533930298 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56267329b9c0 0x5626732bb820 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd8fcff9700 0 client.25484 ms_handle_reset on v2:172.21.15.132:6832/3058040154 2023-10-08T09:15:37.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 >> 172.21.15.49:0/1533930298 conn(0x56267320f260 msgr2=0x562673294a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 shutdown_connections 2023-10-08T09:15:37.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.343+0000 7fd927a75b80 1 -- 172.21.15.49:0/1533930298 wait complete. 2023-10-08T09:15:37.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.345+0000 7fd927a75b80 20 client.25484 trim_cache size 0 max 16384 2023-10-08T09:15:37.346 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2023-10-08T09:15:37.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9000a1590 msgr2=0x7fd9000a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9000a1590 0x7fd9000a3a50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd904097bc0 tx=0x7fd904095c10 comp rx=0 tx=0).stop 2023-10-08T09:15:37.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff080 msgr2=0x7fd914070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff080 0x7fd914070390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd9040098e0 tx=0x7fd9040057b0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 shutdown_connections 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd9000a1590 0x7fd9000a3a50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9141043c0 0x7fd9140718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff080 0x7fd914070390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 --2- 172.21.15.49:0/3525247164 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9141019f0 0x7fd9140708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 >> 172.21.15.49:0/3525247164 conn(0x7fd9140fac40 msgr2=0x7fd9141023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.348 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 shutdown_connections 2023-10-08T09:15:37.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.347+0000 7fd91b88c700 1 -- 172.21.15.49:0/3525247164 wait complete. 2023-10-08T09:15:37.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-08T09:15:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.599+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2964027954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a40109260 msgr2=0x7f2a4010faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.599+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2964027954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a40109260 0x7f2a4010faf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c210 tx=0x7f2a3800c520 comp rx=0 tx=0).stop 2023-10-08T09:15:37.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.599+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2964027954 shutdown_connections 2023-10-08T09:15:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.599+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2964027954 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a40109260 0x7f2a4010faf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.599+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2964027954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a40107b40 0x7f2a40107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.599+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2964027954 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a401088a0 0x7f2a40108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.599+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2964027954 >> 172.21.15.49:0/2964027954 conn(0x7f2a4006e030 msgr2=0x7f2a40070490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.601+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2964027954 shutdown_connections 2023-10-08T09:15:37.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.601+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2964027954 wait complete. 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 Processor -- start 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 -- start start 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a40107b40 0x7f2a401a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a401088a0 0x7f2a401a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a40109260 0x7f2a401a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401a8cb0 con 0x7f2a40107b40 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401a8df0 con 0x7f2a40109260 2023-10-08T09:15:37.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a44c2e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401a8f80 con 0x7f2a401088a0 2023-10-08T09:15:37.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.602+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a40107b40 0x7f2a401a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a40107b40 0x7f2a401a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50394/0 (socket says 172.21.15.49:50394) 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3e59c700 1 -- 172.21.15.49:0/2383688080 learned_addr learned my addr 172.21.15.49:0/2383688080 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a40109260 0x7f2a401a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a401088a0 0x7f2a401a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/2383688080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a40109260 msgr2=0x7f2a401a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a40109260 0x7f2a401a86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a40107b40 msgr2=0x7f2a401a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a40107b40 0x7f2a401a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.603+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/2383688080 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3800ae10 con 0x7f2a401088a0 2023-10-08T09:15:37.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.604+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a401088a0 0x7f2a401a16c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b450 tx=0x7f2a3400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.604+0000 7f2a2f7fe700 1 -- 172.21.15.49:0/2383688080 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34011830 con 0x7f2a401088a0 2023-10-08T09:15:37.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.604+0000 7f2a2f7fe700 1 -- 172.21.15.49:0/2383688080 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a34011e50 con 0x7f2a401088a0 2023-10-08T09:15:37.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.604+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a401a9260 con 0x7f2a401088a0 2023-10-08T09:15:37.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.604+0000 7f2a2f7fe700 1 -- 172.21.15.49:0/2383688080 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400f2e0 con 0x7f2a401088a0 2023-10-08T09:15:37.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.604+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a401a9770 con 0x7f2a401088a0 2023-10-08T09:15:37.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.609+0000 7f2a2f7fe700 1 -- 172.21.15.49:0/2383688080 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2a340105a0 con 0x7f2a401088a0 2023-10-08T09:15:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.610+0000 7f2a50e17b80 1 Processor -- start 2023-10-08T09:15:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.610+0000 7f2a2f7fe700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2a280a1500 0x7f2a280a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.611+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2a280a1500 0x7f2a280a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.611+0000 7f2a2f7fe700 1 -- 172.21.15.49:0/2383688080 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2a340ccb30 con 0x7f2a401088a0 2023-10-08T09:15:37.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.611+0000 7f2a50e17b80 1 -- start start 2023-10-08T09:15:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.611+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2a280a1500 0x7f2a280a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2a380088c0 tx=0x7f2a3800c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.611+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9c4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.611+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.611+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.612+0000 7f2a3dd9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.612 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.612+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.612+0000 7f2a3dd9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9c5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37438/0 (socket says 172.21.15.49:37438) 2023-10-08T09:15:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.612+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/3902447376 learned_addr learned my addr 172.21.15.49:0/3902447376 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.612+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd92c460 con 0x55eecd9c5800 2023-10-08T09:15:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd91dc20 con 0x55eecd9c4060 2023-10-08T09:15:37.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd91b540 con 0x55eecd9c49c0 2023-10-08T09:15:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/3902447376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 msgr2=0x55eecd9c4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9c4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/3902447376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 msgr2=0x55eecd9c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.613+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/3902447376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecd741de0 con 0x55eecd9c49c0 2023-10-08T09:15:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.614+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9c5270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a34095fc0 tx=0x7f2a34004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41bc691b6a407cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.614+0000 7f2a2cff9700 1 -- 172.21.15.49:0/3902447376 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34099ac0 con 0x55eecd9c49c0 2023-10-08T09:15:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.614+0000 7f2a2cff9700 1 -- 172.21.15.49:0/3902447376 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3409dc10 con 0x55eecd9c49c0 2023-10-08T09:15:37.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.614+0000 7f2a2cff9700 1 -- 172.21.15.49:0/3902447376 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34099210 con 0x55eecd9c49c0 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 msgr2=0x55eecd9c5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9c5270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a34095fc0 tx=0x7f2a34004750 comp rx=0 tx=0).stop 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 shutdown_connections 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/3902447376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9c5270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 >> 172.21.15.49:0/3902447376 conn(0x55eecd938260 msgr2=0x55eecd9bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 shutdown_connections 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.615+0000 7f2a50e17b80 1 -- 172.21.15.49:0/3902447376 wait complete. 2023-10-08T09:15:37.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.616+0000 7f2a50e17b80 1 -- start start 2023-10-08T09:15:37.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 1 Processor -- start 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c49c0 0x55eecd9d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c5800 0x55eecd9ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd916e30 con 0x55eecd9c4060 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd8b3710 con 0x55eecd9c49c0 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd8b33b0 con 0x55eecd9c5800 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.617+0000 7f2a50e17b80 10 client.0 _ll_register_callbacks cb 0x55eecd710800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c49c0 0x55eecd9d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c49c0 0x55eecd9d6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40024/0 (socket says 172.21.15.49:40024) 2023-10-08T09:15:37.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/1818487875 learned_addr learned my addr 172.21.15.49:0/1818487875 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9d65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50408/0 (socket says 172.21.15.49:50408) 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/1818487875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c5800 msgr2=0x55eecd9ddb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c5800 0x55eecd9ddb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/1818487875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 msgr2=0x55eecd9d65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9d65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 -- 172.21.15.49:0/1818487875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a401a9770 con 0x55eecd9c49c0 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3e59c700 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9d65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c49c0 0x55eecd9d6b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a340a1920 tx=0x7f2a34095e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc78cd578c0b8e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a25ffb700 1 -- 172.21.15.49:0/1818487875 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3407c070 con 0x55eecd9c49c0 2023-10-08T09:15:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a25ffb700 1 -- 172.21.15.49:0/1818487875 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2a34099ac0 con 0x55eecd9c49c0 2023-10-08T09:15:37.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.618+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eecd8deeb0 con 0x55eecd9c49c0 2023-10-08T09:15:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.620+0000 7f2a25ffb700 1 -- 172.21.15.49:0/1818487875 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a34010380 con 0x55eecd9c49c0 2023-10-08T09:15:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.621+0000 7f2a50e17b80 10 client.25496 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.621+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55eecd72ac60 con 0x55eecd9c49c0 2023-10-08T09:15:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.621+0000 7f2a25ffb700 1 -- 172.21.15.49:0/1818487875 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eecd8deeb0 con 0x55eecd9c49c0 2023-10-08T09:15:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.621+0000 7f2a25ffb700 1 -- 172.21.15.49:0/1818487875 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2a3407f020 con 0x55eecd9c49c0 2023-10-08T09:15:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.621+0000 7f2a50e17b80 10 client.25496 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:37.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.621+0000 7f2a50e17b80 10 client.25496 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2023-10-08T09:15:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.621+0000 7f2a50e17b80 20 client.25496 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.622+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9e2080 0x55eecd9e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.622+0000 7f2a50e17b80 4 client.25496 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.622+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 --> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eecd90fe10 con 0x55eecd9e2080 2023-10-08T09:15:37.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.622+0000 7f2a3e59c700 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9e2080 0x55eecd9e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.622+0000 7f2a25ffb700 10 client.25496 ms_handle_connect on v2:172.21.15.184:6832/3616311663 2023-10-08T09:15:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.622+0000 7f2a3e59c700 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9e2080 0x55eecd9e44a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97b2cecfd9869f46 server_cookie=6ae220809c00dc86 in_seq=0 out_seq=0 2023-10-08T09:15:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.623+0000 7f2a25ffb700 1 -- 172.21.15.49:0/1818487875 <== mds.3 v2:172.21.15.184:6832/3616311663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55eecd90fe10 con 0x55eecd9e2080 2023-10-08T09:15:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.623+0000 7f2a25ffb700 10 client.25496 handle_command_reply: tid=0 2023-10-08T09:15:37.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.623+0000 7f2a50e17b80 1 client.25496 shutdown 2023-10-08T09:15:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c49c0 msgr2=0x55eecd9d6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c49c0 0x55eecd9d6b30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a340a1920 tx=0x7f2a34095e90 comp rx=0 tx=0).stop 2023-10-08T09:15:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 shutdown_connections 2023-10-08T09:15:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c5800 0x55eecd9ddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9e2080 0x55eecd9e44a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9d65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/1818487875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c49c0 0x55eecd9d6b30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 >> 172.21.15.49:0/1818487875 conn(0x55eecd938260 msgr2=0x55eecd9be4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a25ffb700 0 client.25496 ms_handle_reset on v2:172.21.15.184:6832/3616311663 2023-10-08T09:15:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 shutdown_connections 2023-10-08T09:15:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.624+0000 7f2a50e17b80 1 -- 172.21.15.49:0/1818487875 wait complete. 2023-10-08T09:15:37.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.625+0000 7f2a50e17b80 20 client.25496 trim_cache size 0 max 16384 2023-10-08T09:15:37.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:37 smithi049 ceph-mon[99485]: pgmap v350: 129 pgs: 129 active+clean; 1.1 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 99 MiB/s wr, 867 op/s 2023-10-08T09:15:37.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:37 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1087779368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.646+0000 7f2a50e17b80 1 Processor -- start 2023-10-08T09:15:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.646+0000 7f2a50e17b80 1 -- start start 2023-10-08T09:15:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.647+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.647+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.647+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.647+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd797920 con 0x55eecd9c5800 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.647+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd9b95d0 con 0x55eecd9c4060 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.647+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd9bfd10 con 0x55eecd9c49c0 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9e4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50416/0 (socket says 172.21.15.49:50416) 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 -- 172.21.15.49:0/2975290859 learned_addr learned my addr 172.21.15.49:0/2975290859 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3e59c700 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 -- 172.21.15.49:0/2975290859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 msgr2=0x55eecd9ddb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:37.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9ddb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 -- 172.21.15.49:0/2975290859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 msgr2=0x55eecd9cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 -- 172.21.15.49:0/2975290859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eecd768250 con 0x55eecd9c5800 2023-10-08T09:15:37.650 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.648+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9e4090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2a38008740 tx=0x7f2a3801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd1272ff1853ed6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.649+0000 7f2a2d7fa700 1 -- 172.21.15.49:0/2975290859 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800a5d0 con 0x55eecd9c5800 2023-10-08T09:15:37.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.649+0000 7f2a2d7fa700 1 -- 172.21.15.49:0/2975290859 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3802a470 con 0x55eecd9c5800 2023-10-08T09:15:37.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.649+0000 7f2a2d7fa700 1 -- 172.21.15.49:0/2975290859 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38007700 con 0x55eecd9c5800 2023-10-08T09:15:37.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.650+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2975290859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 msgr2=0x55eecd9e4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.651 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.650+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9e4090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2a38008740 tx=0x7f2a3801a000 comp rx=0 tx=0).stop 2023-10-08T09:15:37.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.656+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2975290859 shutdown_connections 2023-10-08T09:15:37.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.656+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c5800 0x55eecd9e4090 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.656+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c4060 0x55eecd9cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.656+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2975290859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9ddb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.659 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.656+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2975290859 >> 172.21.15.49:0/2975290859 conn(0x55eecd938260 msgr2=0x55eecd9baf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.663+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2975290859 shutdown_connections 2023-10-08T09:15:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.664+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2975290859 wait complete. 2023-10-08T09:15:37.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.664+0000 7f2a50e17b80 1 -- start start 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.664+0000 7f2a50e17b80 1 Processor -- start 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.664+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9dd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.664+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.664+0000 7f2a50e17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c5800 0x55eecd9dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd9dc1f0 con 0x55eecd9c4060 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd9dc360 con 0x55eecd9c5800 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a50e17b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eecd9dc520 con 0x55eecd9c49c0 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a50e17b80 10 client.0 _ll_register_callbacks cb 0x7f2a38021040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9dd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a3e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9dd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50422/0 (socket says 172.21.15.49:50422) 2023-10-08T09:15:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a3e59c700 1 -- 172.21.15.49:0/2252596577 learned_addr learned my addr 172.21.15.49:0/2252596577 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c5800 0x55eecd9dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.665+0000 7f2a3dd9b700 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a3ed9d700 1 -- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 msgr2=0x55eecd9dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a3ed9d700 1 -- 172.21.15.49:0/2252596577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 msgr2=0x55eecd9dd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9dd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a3ed9d700 1 -- 172.21.15.49:0/2252596577 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a40110380 con 0x55eecd9c5800 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a3ed9d700 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c5800 0x55eecd9dbbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c210 tx=0x7f2a3800aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b333983b366604 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a25ffb700 1 -- 172.21.15.49:0/2252596577 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38004a10 con 0x55eecd9c5800 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.666+0000 7f2a25ffb700 1 -- 172.21.15.49:0/2252596577 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2a38004b70 con 0x55eecd9c5800 2023-10-08T09:15:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.667+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a3409dc10 con 0x55eecd9c5800 2023-10-08T09:15:37.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.670+0000 7f2a25ffb700 1 -- 172.21.15.49:0/2252596577 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a38004d30 con 0x55eecd9c5800 2023-10-08T09:15:37.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.670+0000 7f2a50e17b80 10 client.25502 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:37.670 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.670+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55eecd7402b0 con 0x55eecd9c5800 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.670+0000 7f2a25ffb700 1 -- 172.21.15.49:0/2252596577 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3409dc10 con 0x55eecd9c5800 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.671+0000 7f2a25ffb700 1 -- 172.21.15.49:0/2252596577 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2a380074a0 con 0x55eecd9c5800 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.671+0000 7f2a50e17b80 10 client.25502 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.671+0000 7f2a50e17b80 10 client.25502 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.671+0000 7f2a50e17b80 20 client.25502 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.671+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9d4690 0x55eecd9d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.671+0000 7f2a50e17b80 4 client.25502 mds_command: new command op to 24553 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:15:37.672 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.671+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 --> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eecd90fb30 con 0x55eecd9d4690 2023-10-08T09:15:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.672+0000 7f2a3e59c700 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9d4690 0x55eecd9d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.673+0000 7f2a25ffb700 10 client.25502 ms_handle_connect on v2:172.21.15.184:6832/3616311663 2023-10-08T09:15:37.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.673+0000 7f2a3e59c700 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9d4690 0x55eecd9d4a70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e3fded1ed7c0e250 server_cookie=35ed8b0c24d21015 in_seq=0 out_seq=0 2023-10-08T09:15:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.673+0000 7f2a25ffb700 1 -- 172.21.15.49:0/2252596577 <== mds.3 v2:172.21.15.184:6832/3616311663 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eecd90fb30 con 0x55eecd9d4690 2023-10-08T09:15:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.673+0000 7f2a25ffb700 10 client.25502 handle_command_reply: tid=0 2023-10-08T09:15:37.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.673+0000 7f2a50e17b80 1 client.25502 shutdown 2023-10-08T09:15:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.674+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c5800 msgr2=0x55eecd9dbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.674+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c5800 0x55eecd9dbbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c210 tx=0x7f2a3800aaf0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 shutdown_connections 2023-10-08T09:15:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:6832/3616311663,v1:172.21.15.184:6833/3616311663] conn(0x55eecd9d4690 0x55eecd9d4a70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55eecd9c5800 0x55eecd9dbbd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55eecd9c4060 0x55eecd9dd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 --2- 172.21.15.49:0/2252596577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55eecd9c49c0 0x55eecd9dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 >> 172.21.15.49:0/2252596577 conn(0x55eecd938260 msgr2=0x55eecd9cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a25ffb700 0 client.25502 ms_handle_reset on v2:172.21.15.184:6832/3616311663 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 shutdown_connections 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 1 -- 172.21.15.49:0/2252596577 wait complete. 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.675+0000 7f2a50e17b80 20 client.25502 trim_cache size 0 max 16384 2023-10-08T09:15:37.676 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2023-10-08T09:15:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.676+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2a280a1500 msgr2=0x7f2a280a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.676+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2a280a1500 0x7f2a280a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2a380088c0 tx=0x7f2a3800c920 comp rx=0 tx=0).stop 2023-10-08T09:15:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.676+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a401088a0 msgr2=0x7f2a401a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.676+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a401088a0 0x7f2a401a16c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b450 tx=0x7f2a3400b810 comp rx=0 tx=0).stop 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 shutdown_connections 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2a280a1500 0x7f2a280a39c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a40109260 0x7f2a401a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2a40107b40 0x7f2a401a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 --2- 172.21.15.49:0/2383688080 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a401088a0 0x7f2a401a16c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 >> 172.21.15.49:0/2383688080 conn(0x7f2a4006e030 msgr2=0x7f2a40197ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 shutdown_connections 2023-10-08T09:15:37.678 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.677+0000 7f2a44c2e700 1 -- 172.21.15.49:0/2383688080 wait complete. 2023-10-08T09:15:37.698 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-08T09:15:37.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:37 smithi132 ceph-mon[103389]: pgmap v350: 129 pgs: 129 active+clean; 1.1 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 99 MiB/s wr, 867 op/s 2023-10-08T09:15:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:37 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1087779368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:37.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.889+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/1346699592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbb410cb30 msgr2=0x7fdbb410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.889+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/1346699592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbb410cb30 0x7fdbb410cfb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009990 tx=0x7fdbb0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.890+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/1346699592 shutdown_connections 2023-10-08T09:15:37.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.890+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/1346699592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbb410cb30 0x7fdbb410cfb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.890+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/1346699592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbb41077f0 0x7fdbb4109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.890+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/1346699592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb410a160 0x7fdbb410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.890+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/1346699592 >> 172.21.15.49:0/1346699592 conn(0x7fdbb406d790 msgr2=0x7fdbb406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.891+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/1346699592 shutdown_connections 2023-10-08T09:15:37.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.897+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/1346699592 wait complete. 2023-10-08T09:15:37.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 Processor -- start 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 -- start start 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb41077f0 0x7fdbb41a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbb410a160 0x7fdbb41a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbb410cb30 0x7fdbb41aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb410ef20 con 0x7fdbb410cb30 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb410f090 con 0x7fdbb410a160 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbbb7c0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb410f250 con 0x7fdbb41077f0 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbb955c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb41077f0 0x7fdbb41a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbb955c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb41077f0 0x7fdbb41a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37468/0 (socket says 172.21.15.49:37468) 2023-10-08T09:15:37.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.898+0000 7fdbb955c700 1 -- 172.21.15.49:0/623222525 learned_addr learned my addr 172.21.15.49:0/623222525 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbb955c700 1 -- 172.21.15.49:0/623222525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbb410a160 msgr2=0x7fdbb41a5a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:15:37.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbb955c700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbb410a160 0x7fdbb41a5a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbb955c700 1 -- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbb410cb30 msgr2=0x7fdbb41aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbb955c700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbb410cb30 0x7fdbb41aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbb955c700 1 -- 172.21.15.49:0/623222525 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb00095f0 con 0x7fdbb41077f0 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbb955c700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb41077f0 0x7fdbb41a54d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00ba50 tx=0x7fdbac00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbaa7fc700 1 -- 172.21.15.49:0/623222525 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00cab0 con 0x7fdbb41077f0 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb410f4d0 con 0x7fdbb41077f0 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbaa7fc700 1 -- 172.21.15.49:0/623222525 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbac00cc10 con 0x7fdbb41077f0 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.899+0000 7fdbaa7fc700 1 -- 172.21.15.49:0/623222525 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0075d0 con 0x7fdbb41077f0 2023-10-08T09:15:37.901 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.900+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb410fa30 con 0x7fdbb41077f0 2023-10-08T09:15:37.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.903+0000 7fdbaa7fc700 1 -- 172.21.15.49:0/623222525 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdbac014440 con 0x7fdbb41077f0 2023-10-08T09:15:37.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.904+0000 7fdbaa7fc700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdba00a1550 0x7fdba00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.904+0000 7fdbaa7fc700 1 -- 172.21.15.49:0/623222525 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdbac006e40 con 0x7fdbb41077f0 2023-10-08T09:15:37.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.906+0000 7fdbb8d5b700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdba00a1550 0x7fdba00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.907+0000 7fdbc79a9b80 1 Processor -- start 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.908+0000 7fdbc79a9b80 1 -- start start 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.908+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a218480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.908+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a219270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.908+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a219c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.908+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a180460 con 0x56226a2189c0 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.908+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a171c20 con 0x56226a218060 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.908+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a16f540 con 0x56226a219800 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a218480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a218480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40054/0 (socket says 172.21.15.49:40054) 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 -- 172.21.15.49:0/2588006644 learned_addr learned my addr 172.21.15.49:0/2588006644 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb8d5b700 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a219270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a219c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 -- 172.21.15.49:0/2588006644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 msgr2=0x56226a219c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a219c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 -- 172.21.15.49:0/2588006644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 msgr2=0x56226a219270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a219270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 -- 172.21.15.49:0/2588006644 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562269f95de0 con 0x56226a218060 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb955c700 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a218480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbac095ca0 tx=0x7fdbac004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d4ce390d0d289b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.909+0000 7fdbb8d5b700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdba00a1550 0x7fdba00a3a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdba400a9b0 tx=0x7fdba4005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.910+0000 7fdb9f7fe700 1 -- 172.21.15.49:0/2588006644 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0997a0 con 0x56226a218060 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.910+0000 7fdb9f7fe700 1 -- 172.21.15.49:0/2588006644 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbac09d8f0 con 0x56226a218060 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.910+0000 7fdb9f7fe700 1 -- 172.21.15.49:0/2588006644 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0b9140 con 0x56226a218060 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/2588006644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 msgr2=0x56226a218480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a218480 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdbac095ca0 tx=0x7fdbac004be0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/2588006644 shutdown_connections 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a219c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a218480 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/2588006644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a219270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/2588006644 >> 172.21.15.49:0/2588006644 conn(0x56226a18c260 msgr2=0x56226a212e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/2588006644 shutdown_connections 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/2588006644 wait complete. 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 -- start start 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.911+0000 7fdbc79a9b80 1 Processor -- start 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a218060 0x56226a221e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a2189c0 0x56226a222380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a219800 0x56226a2293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a16ae30 con 0x56226a219800 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a107710 con 0x56226a2189c0 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a1073b0 con 0x56226a218060 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbc79a9b80 10 client.0 _ll_register_callbacks cb 0x562269f64800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:37.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a2189c0 0x56226a222380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb9d5d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a219800 0x56226a2293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a2189c0 0x56226a222380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40062/0 (socket says 172.21.15.49:40062) 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 -- 172.21.15.49:0/3229884448 learned_addr learned my addr 172.21.15.49:0/3229884448 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb955c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a218060 0x56226a221e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 -- 172.21.15.49:0/3229884448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a218060 msgr2=0x56226a221e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a218060 0x56226a221e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 -- 172.21.15.49:0/3229884448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a219800 msgr2=0x56226a2293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a219800 0x56226a2293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb8d5b700 1 -- 172.21.15.49:0/3229884448 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb410fa30 con 0x56226a2189c0 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.912+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a219800 0x56226a2293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.913+0000 7fdbb8d5b700 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a2189c0 0x56226a222380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdba400a790 tx=0x7fdba40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d5bce22772f285 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.913+0000 7fdbb955c700 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a218060 0x56226a221e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:37.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.913+0000 7fdb9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:37.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.914+0000 7fdb9cff9700 1 -- 172.21.15.49:0/3229884448 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4004220 con 0x56226a2189c0 2023-10-08T09:15:37.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.914+0000 7fdb9cff9700 1 -- 172.21.15.49:0/3229884448 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdba4004380 con 0x56226a2189c0 2023-10-08T09:15:37.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.914+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56226a132eb0 con 0x56226a2189c0 2023-10-08T09:15:37.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.914+0000 7fdb9cff9700 1 -- 172.21.15.49:0/3229884448 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdba400fe90 con 0x56226a2189c0 2023-10-08T09:15:37.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.915+0000 7fdbc79a9b80 10 client.25508 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:37.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.915+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562269f7ec60 con 0x56226a2189c0 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.915+0000 7fdb9cff9700 1 -- 172.21.15.49:0/3229884448 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56226a132eb0 con 0x56226a2189c0 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdb9cff9700 1 -- 172.21.15.49:0/3229884448 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdba4022030 con 0x56226a2189c0 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdbc79a9b80 10 client.25508 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdbc79a9b80 10 client.25508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdbc79a9b80 20 client.25508 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a22dca0 0x56226a2300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdbc79a9b80 4 client.25508 mds_command: new command op to 24608 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 --> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56226a163e10 con 0x56226a22dca0 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.916+0000 7fdbb955c700 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a22dca0 0x56226a2300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.917+0000 7fdbb955c700 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a22dca0 0x56226a2300c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=393a009bb914bf18 server_cookie=7d8f7a7a3727dff3 in_seq=0 out_seq=0 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.917+0000 7fdb9cff9700 10 client.25508 ms_handle_connect on v2:172.21.15.132:6836/2064856316 2023-10-08T09:15:37.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdb9cff9700 1 -- 172.21.15.49:0/3229884448 <== mds.2 v2:172.21.15.132:6836/2064856316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56226a163e10 con 0x56226a22dca0 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdb9cff9700 10 client.25508 handle_command_reply: tid=0 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 client.25508 shutdown 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a2189c0 msgr2=0x56226a222380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a2189c0 0x56226a222380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdba400a790 tx=0x7fdba40047c0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 shutdown_connections 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a218060 0x56226a221e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a22dca0 0x56226a2300c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a219800 0x56226a2293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3229884448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a2189c0 0x56226a222380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdb9cff9700 0 client.25508 ms_handle_reset on v2:172.21.15.132:6836/2064856316 2023-10-08T09:15:37.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 >> 172.21.15.49:0/3229884448 conn(0x56226a18c260 msgr2=0x56226a2124d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 shutdown_connections 2023-10-08T09:15:37.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3229884448 wait complete. 2023-10-08T09:15:37.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.918+0000 7fdbc79a9b80 20 client.25508 trim_cache size 0 max 16384 2023-10-08T09:15:37.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.940+0000 7fdbc79a9b80 1 Processor -- start 2023-10-08T09:15:37.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.940+0000 7fdbc79a9b80 1 -- start start 2023-10-08T09:15:37.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a218060 0x56226a230090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a2189c0 0x56226a221350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a219800 0x56226a229380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562269feb920 con 0x56226a218060 2023-10-08T09:15:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a219100 con 0x56226a219800 2023-10-08T09:15:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a132eb0 con 0x56226a2189c0 2023-10-08T09:15:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a219800 0x56226a229380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb955c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a218060 0x56226a230090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a219800 0x56226a229380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40068/0 (socket says 172.21.15.49:40068) 2023-10-08T09:15:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/3669538408 learned_addr learned my addr 172.21.15.49:0/3669538408 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb955c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a218060 0x56226a230090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50456/0 (socket says 172.21.15.49:50456) 2023-10-08T09:15:37.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/3669538408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a2189c0 msgr2=0x56226a221350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:37.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a2189c0 0x56226a221350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/3669538408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a218060 msgr2=0x56226a230090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a218060 0x56226a230090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/3669538408 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562269fbc250 con 0x56226a219800 2023-10-08T09:15:37.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.941+0000 7fdbb955c700 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a218060 0x56226a230090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.942+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a219800 0x56226a229380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0015000 tx=0x7fdbb0005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6864193fe5dcbfd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.942+0000 7fdb9ffff700 1 -- 172.21.15.49:0/3669538408 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb001e050 con 0x56226a219800 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.942+0000 7fdb9ffff700 1 -- 172.21.15.49:0/3669538408 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb001f040 con 0x56226a219800 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.942+0000 7fdb9ffff700 1 -- 172.21.15.49:0/3669538408 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0004d80 con 0x56226a219800 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.943+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3669538408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a219800 msgr2=0x56226a229380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.943+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a219800 0x56226a229380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0015000 tx=0x7fdbb0005c70 comp rx=0 tx=0).stop 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.944+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3669538408 shutdown_connections 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.944+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a219800 0x56226a229380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.944+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a218060 0x56226a230090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.944+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/3669538408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a2189c0 0x56226a221350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.944+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3669538408 >> 172.21.15.49:0/3669538408 conn(0x56226a18c260 msgr2=0x56226a212dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.944+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3669538408 shutdown_connections 2023-10-08T09:15:37.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.945+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/3669538408 wait complete. 2023-10-08T09:15:37.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.945+0000 7fdbc79a9b80 1 -- start start 2023-10-08T09:15:37.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.945+0000 7fdbc79a9b80 1 Processor -- start 2023-10-08T09:15:37.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a227640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a227b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbc79a9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a228930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a228f00 con 0x56226a2189c0 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a229070 con 0x56226a218060 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbc79a9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56226a229230 con 0x56226a219800 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbc79a9b80 10 client.0 _ll_register_callbacks cb 0x7fdba400cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbb9d5d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a228930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbb9d5d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a228930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37496/0 (socket says 172.21.15.49:37496) 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.946+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/83852186 learned_addr learned my addr 172.21.15.49:0/83852186 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb8d5b700 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a227b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb955c700 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a227640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 msgr2=0x56226a227640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a227640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/83852186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 msgr2=0x56226a227b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a227b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb9d5d700 1 -- 172.21.15.49:0/83852186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562269f95de0 con 0x56226a219800 2023-10-08T09:15:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdbb9d5d700 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a228930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0005270 tx=0x7fdbb00052a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d99958d87302 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.947+0000 7fdb9cff9700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:15:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.948+0000 7fdb9cff9700 1 -- 172.21.15.49:0/83852186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb001e070 con 0x56226a219800 2023-10-08T09:15:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.948+0000 7fdb9cff9700 1 -- 172.21.15.49:0/83852186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdbb0004710 con 0x56226a219800 2023-10-08T09:15:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.948+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbac09d8f0 con 0x56226a219800 2023-10-08T09:15:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.948+0000 7fdb9cff9700 1 -- 172.21.15.49:0/83852186 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbb0005a00 con 0x56226a219800 2023-10-08T09:15:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.948+0000 7fdbc79a9b80 10 client.25531 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:37.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.948+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fdbb410dae0 con 0x56226a219800 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdb9cff9700 1 -- 172.21.15.49:0/83852186 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbac09d8f0 con 0x56226a219800 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdb9cff9700 1 -- 172.21.15.49:0/83852186 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdbb002c420 con 0x56226a219800 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdbc79a9b80 10 client.25531 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdbc79a9b80 10 client.25531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdbc79a9b80 20 client.25531 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a2829d0 0x56226a284df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdbc79a9b80 4 client.25531 mds_command: new command op to 24608 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.949+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 --> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56226a163b30 con 0x56226a2829d0 2023-10-08T09:15:37.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.950+0000 7fdbb955c700 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a2829d0 0x56226a284df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.950+0000 7fdb9cff9700 10 client.25531 ms_handle_connect on v2:172.21.15.132:6836/2064856316 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.950+0000 7fdbb955c700 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a2829d0 0x56226a284df0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80faa84f0c9763ae server_cookie=df59dce88c088bf2 in_seq=0 out_seq=0 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.950+0000 7fdb9cff9700 1 -- 172.21.15.49:0/83852186 <== mds.2 v2:172.21.15.132:6836/2064856316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56226a163b30 con 0x56226a2829d0 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.950+0000 7fdb9cff9700 10 client.25531 handle_command_reply: tid=0 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.950+0000 7fdbc79a9b80 1 client.25531 shutdown 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 msgr2=0x56226a228930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a228930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0005270 tx=0x7fdbb00052a0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 shutdown_connections 2023-10-08T09:15:37.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56226a219800 0x56226a228930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:6836/2064856316,v1:172.21.15.132:6837/2064856316] conn(0x56226a2829d0 0x56226a284df0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56226a218060 0x56226a227640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 --2- 172.21.15.49:0/83852186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56226a2189c0 0x56226a227b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 >> 172.21.15.49:0/83852186 conn(0x56226a18c260 msgr2=0x56226a212d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.951+0000 7fdb9cff9700 0 client.25531 ms_handle_reset on v2:172.21.15.132:6836/2064856316 2023-10-08T09:15:37.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.952+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 shutdown_connections 2023-10-08T09:15:37.954 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2023-10-08T09:15:37.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.952+0000 7fdbc79a9b80 1 -- 172.21.15.49:0/83852186 wait complete. 2023-10-08T09:15:37.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.952+0000 7fdbc79a9b80 20 client.25531 trim_cache size 0 max 16384 2023-10-08T09:15:37.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.954+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdba00a1550 msgr2=0x7fdba00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.954+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdba00a1550 0x7fdba00a3a10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdba400a9b0 tx=0x7fdba4005c90 comp rx=0 tx=0).stop 2023-10-08T09:15:37.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.954+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb41077f0 msgr2=0x7fdbb41a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:37.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.954+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb41077f0 0x7fdbb41a54d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00ba50 tx=0x7fdbac00be10 comp rx=0 tx=0).stop 2023-10-08T09:15:37.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 shutdown_connections 2023-10-08T09:15:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdba00a1550 0x7fdba00a3a10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbb41077f0 0x7fdbb41a54d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbb410cb30 0x7fdbb41aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 --2- 172.21.15.49:0/623222525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbb410a160 0x7fdbb41a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 >> 172.21.15.49:0/623222525 conn(0x7fdbb406d790 msgr2=0x7fdbb4109940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 shutdown_connections 2023-10-08T09:15:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:37.955+0000 7fdbbb7c0700 1 -- 172.21.15.49:0/623222525 wait complete. 2023-10-08T09:15:37.977 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-08T09:15:37.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:37 smithi184 ceph-mon[103675]: pgmap v350: 129 pgs: 129 active+clean; 1.1 GiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 99 MiB/s wr, 867 op/s 2023-10-08T09:15:37.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:37 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1087779368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:15:38.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.200+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3416471887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e410db60 msgr2=0x7fa3e410fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.200+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3416471887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e410db60 0x7fa3e410fff0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000dd00 tx=0x7fa3e00084b0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.200+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3416471887 shutdown_connections 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.200+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3416471887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e4108900 0x7fa3e410acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.200+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3416471887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e410db60 0x7fa3e410fff0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.200+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3416471887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e410b230 0x7fa3e410d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.200+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3416471887 >> 172.21.15.49:0/3416471887 conn(0x7fa3e406d790 msgr2=0x7fa3e406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.201+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3416471887 shutdown_connections 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.201+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3416471887 wait complete. 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 Processor -- start 2023-10-08T09:15:38.205 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 -- start start 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4108900 0x7fa3e4072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e410b230 0x7fa3e4073340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e410db60 0x7fa3e407a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e407a930 con 0x7fa3e4108900 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e407aaa0 con 0x7fa3e410b230 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.202+0000 7fa3ec45d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e407ac60 con 0x7fa3e410db60 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e410db60 0x7fa3e407a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea1f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4108900 0x7fa3e4072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e410db60 0x7fa3e407a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37508/0 (socket says 172.21.15.49:37508) 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/3559821150 learned_addr learned my addr 172.21.15.49:0/3559821150 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3e99f8700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e410b230 0x7fa3e4073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/3559821150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e410b230 msgr2=0x7fa3e4073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e410b230 0x7fa3e4073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4108900 msgr2=0x7fa3e4072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4108900 0x7fa3e4072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/3559821150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e000d930 con 0x7fa3e410db60 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e410db60 0x7fa3e407a360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d900 tx=0x7fa3e0009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3db7fe700 1 -- 172.21.15.49:0/3559821150 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0005450 con 0x7fa3e410db60 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3db7fe700 1 -- 172.21.15.49:0/3559821150 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e00055b0 con 0x7fa3e410db60 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e407ae80 con 0x7fa3e410db60 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.203+0000 7fa3db7fe700 1 -- 172.21.15.49:0/3559821150 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0003aa0 con 0x7fa3e410db60 2023-10-08T09:15:38.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.204+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e407b380 con 0x7fa3e410db60 2023-10-08T09:15:38.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.208+0000 7fa3db7fe700 1 -- 172.21.15.49:0/3559821150 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa3e0003c40 con 0x7fa3e410db60 2023-10-08T09:15:38.209 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.209+0000 7fa3f8646b80 1 Processor -- start 2023-10-08T09:15:38.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.209+0000 7fa3f8646b80 1 -- start start 2023-10-08T09:15:38.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.209+0000 7fa3db7fe700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa3d00a15d0 0x7fa3d00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.210 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3db7fe700 1 -- 172.21.15.49:0/3559821150 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa3e00d6a80 con 0x7fa3e410db60 2023-10-08T09:15:38.211 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bd0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bd1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bd1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042b38460 con 0x557042bd1800 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042b29c20 con 0x557042bd09c0 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042b27540 con 0x557042bd0060 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.210+0000 7fa3e99f8700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa3d00a15d0 0x7fa3d00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bd1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bd1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50478/0 (socket says 172.21.15.49:50478) 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/4139933151 learned_addr learned my addr 172.21.15.49:0/4139933151 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3e99f8700 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bd1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bd0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/4139933151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 msgr2=0x557042bd0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bd0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/4139933151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 msgr2=0x557042bd1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bd1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/4139933151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55704294dde0 con 0x557042bd1800 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3e99f8700 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bd1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:38.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.211+0000 7fa3e99f8700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa3d00a15d0 0x7fa3d00a3a90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4006fd0 tx=0x7fa3d4009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.212+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bd1c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa3e009ffa0 tx=0x7fa3e000d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f0d0e540833ce0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.212+0000 7fa3d8d82700 1 -- 172.21.15.49:0/4139933151 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00a3aa0 con 0x557042bd1800 2023-10-08T09:15:38.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.212+0000 7fa3d8d82700 1 -- 172.21.15.49:0/4139933151 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e00a7bf0 con 0x557042bd1800 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.212+0000 7fa3d8d82700 1 -- 172.21.15.49:0/4139933151 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0000ed0 con 0x557042bd1800 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.213+0000 7fa3f8646b80 1 -- 172.21.15.49:0/4139933151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 msgr2=0x557042bd1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.213+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bd1c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa3e009ffa0 tx=0x7fa3e000d670 comp rx=0 tx=0).stop 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.213+0000 7fa3f8646b80 1 -- 172.21.15.49:0/4139933151 shutdown_connections 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.213+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bd0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.213+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bd1c90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.213+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/4139933151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bd1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.213+0000 7fa3f8646b80 1 -- 172.21.15.49:0/4139933151 >> 172.21.15.49:0/4139933151 conn(0x557042b44260 msgr2=0x557042bcae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.214+0000 7fa3f8646b80 1 -- 172.21.15.49:0/4139933151 shutdown_connections 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.214+0000 7fa3f8646b80 1 -- 172.21.15.49:0/4139933151 wait complete. 2023-10-08T09:15:38.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.214+0000 7fa3f8646b80 1 -- start start 2023-10-08T09:15:38.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.214+0000 7fa3f8646b80 1 Processor -- start 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd0060 0x557042bd9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042bd9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd1800 0x557042be1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042b22e30 con 0x557042bd0060 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042abf710 con 0x557042bd1800 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042abf3b0 con 0x557042bd09c0 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3f8646b80 10 client.0 _ll_register_callbacks cb 0x55704291c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea1f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd0060 0x557042bd9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea9fa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd1800 0x557042be1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea1f9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd0060 0x557042bd9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50484/0 (socket says 172.21.15.49:50484) 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea1f9700 1 -- 172.21.15.49:0/296389024 learned_addr learned my addr 172.21.15.49:0/296389024 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3e99f8700 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042bd9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 msgr2=0x557042bd9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042bd9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/296389024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd0060 msgr2=0x557042bd9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd0060 0x557042bd9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/296389024 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d4015c20 con 0x557042bd1800 2023-10-08T09:15:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3e99f8700 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042bd9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:38.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd1800 0x557042be1020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3e009ffa0 tx=0x7fa3e00a0450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2c2bef9f429d0e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.215+0000 7fa3cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.217+0000 7fa3cdffb700 1 -- 172.21.15.49:0/296389024 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00a3aa0 con 0x557042bd1800 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.217+0000 7fa3cdffb700 1 -- 172.21.15.49:0/296389024 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa3e00a3f10 con 0x557042bd1800 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.217+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557042aeaeb0 con 0x557042bd1800 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3cdffb700 1 -- 172.21.15.49:0/296389024 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3e00bc120 con 0x557042bd1800 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 10 client.25523 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fa3e407b380 con 0x557042bd1800 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3cdffb700 1 -- 172.21.15.49:0/296389024 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557042aeaeb0 con 0x557042bd1800 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3cdffb700 1 -- 172.21.15.49:0/296389024 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa3e00ab9a0 con 0x557042bd1800 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 10 client.25523 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:38.220 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 10 client.25523 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:15:38.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 20 client.25523 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:38.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042be5510 0x557042be7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 4 client.25523 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:38.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.218+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 --> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557042b1be10 con 0x557042be5510 2023-10-08T09:15:38.221 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.219+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042be5510 0x557042be7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.220+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042be5510 0x557042be7930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee8bc6967b0c3f1 server_cookie=63c2846bdffef825 in_seq=0 out_seq=0 2023-10-08T09:15:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.220+0000 7fa3cdffb700 10 client.25523 ms_handle_connect on v2:172.21.15.184:6838/2795080121 2023-10-08T09:15:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.220+0000 7fa3cdffb700 1 -- 172.21.15.49:0/296389024 <== mds.1 v2:172.21.15.184:6838/2795080121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557042b1be10 con 0x557042be5510 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.220+0000 7fa3cdffb700 10 client.25523 handle_command_reply: tid=0 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.220+0000 7fa3f8646b80 1 client.25523 shutdown 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd1800 msgr2=0x557042be1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd1800 0x557042be1020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3e009ffa0 tx=0x7fa3e00a0450 comp rx=0 tx=0).stop 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 shutdown_connections 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042be5510 0x557042be7930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd1800 0x557042be1020 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd0060 0x557042bd9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/296389024 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042bd9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.221+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 >> 172.21.15.49:0/296389024 conn(0x557042b44260 msgr2=0x557042bca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.222+0000 7fa3cdffb700 0 client.25523 ms_handle_reset on v2:172.21.15.184:6838/2795080121 2023-10-08T09:15:38.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.222+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 shutdown_connections 2023-10-08T09:15:38.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.222+0000 7fa3f8646b80 1 -- 172.21.15.49:0/296389024 wait complete. 2023-10-08T09:15:38.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.222+0000 7fa3f8646b80 20 client.25523 trim_cache size 0 max 16384 2023-10-08T09:15:38.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.246+0000 7fa3f8646b80 1 Processor -- start 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3f8646b80 1 -- start start 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd0060 0x557042bd7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042be0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042be7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570429a3920 con 0x557042bd1800 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042bc7ae0 con 0x557042bd0060 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042bd8580 con 0x557042bd09c0 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3ea1f9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd0060 0x557042bd7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3ea1f9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd0060 0x557042bd7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40106/0 (socket says 172.21.15.49:40106) 2023-10-08T09:15:38.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3ea1f9700 1 -- 172.21.15.49:0/3571121206 learned_addr learned my addr 172.21.15.49:0/3571121206 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.247+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042be7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.248+0000 7fa3ea1f9700 1 -- 172.21.15.49:0/3571121206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 msgr2=0x557042be0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.248+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042be0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.248+0000 7fa3ea1f9700 1 -- 172.21.15.49:0/3571121206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 msgr2=0x557042be7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.248+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042be7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.248+0000 7fa3ea1f9700 1 -- 172.21.15.49:0/3571121206 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557042936c60 con 0x557042bd0060 2023-10-08T09:15:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.248+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042be7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:38.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.249+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd0060 0x557042bd7d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc012770 tx=0x7fa3dc0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc052c64bf39f65f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.249+0000 7fa3d9583700 1 -- 172.21.15.49:0/3571121206 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc00e040 con 0x557042bd0060 2023-10-08T09:15:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.249+0000 7fa3d9583700 1 -- 172.21.15.49:0/3571121206 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3dc00f040 con 0x557042bd0060 2023-10-08T09:15:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.249+0000 7fa3d9583700 1 -- 172.21.15.49:0/3571121206 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc01f880 con 0x557042bd0060 2023-10-08T09:15:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.250+0000 7fa3f8646b80 1 -- 172.21.15.49:0/3571121206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd0060 msgr2=0x557042bd7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.250+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd0060 0x557042bd7d60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc012770 tx=0x7fa3dc0099e0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.252+0000 7fa3f8646b80 1 -- 172.21.15.49:0/3571121206 shutdown_connections 2023-10-08T09:15:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.252+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042be7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.252+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd0060 0x557042bd7d60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.252 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.252+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/3571121206 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd09c0 0x557042be0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.253 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.252+0000 7fa3f8646b80 1 -- 172.21.15.49:0/3571121206 >> 172.21.15.49:0/3571121206 conn(0x557042b44260 msgr2=0x557042bc6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.257+0000 7fa3f8646b80 1 -- 172.21.15.49:0/3571121206 shutdown_connections 2023-10-08T09:15:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.258+0000 7fa3f8646b80 1 -- 172.21.15.49:0/3571121206 wait complete. 2023-10-08T09:15:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 -- start start 2023-10-08T09:15:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 Processor -- start 2023-10-08T09:15:38.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bdc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bdcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bdfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042be0130 con 0x557042bd1800 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042be02a0 con 0x557042bd09c0 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042be0460 con 0x557042bd0060 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.259+0000 7fa3f8646b80 10 client.0 _ll_register_callbacks cb 0x7fa3d4009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.260+0000 7fa3ea1f9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bdc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.260+0000 7fa3ea1f9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bdc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37532/0 (socket says 172.21.15.49:37532) 2023-10-08T09:15:38.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.260+0000 7fa3ea1f9700 1 -- 172.21.15.49:0/264281783 learned_addr learned my addr 172.21.15.49:0/264281783 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.260+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bdfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/264281783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 msgr2=0x557042bdc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bdc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/264281783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 msgr2=0x557042bdcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bdcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3ea9fa700 1 -- 172.21.15.49:0/264281783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e407b380 con 0x557042bd1800 2023-10-08T09:15:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3ea9fa700 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bdfbf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c530 tx=0x7fa3e00a3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d009fad13e60b2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3cdffb700 1 -- 172.21.15.49:0/264281783 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00abb10 con 0x557042bd1800 2023-10-08T09:15:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3cdffb700 1 -- 172.21.15.49:0/264281783 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa3e00a3aa0 con 0x557042bd1800 2023-10-08T09:15:38.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.261+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3e00a7bf0 con 0x557042bd1800 2023-10-08T09:15:38.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.265+0000 7fa3cdffb700 1 -- 172.21.15.49:0/264281783 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3e00a2f90 con 0x557042bd1800 2023-10-08T09:15:38.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.266+0000 7fa3f8646b80 10 client.15897 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:38.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.267+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fa3dc012390 con 0x557042bd1800 2023-10-08T09:15:38.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.267+0000 7fa3cdffb700 1 -- 172.21.15.49:0/264281783 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e00a7bf0 con 0x557042bd1800 2023-10-08T09:15:38.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.268+0000 7fa3cdffb700 1 -- 172.21.15.49:0/264281783 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa3e00a2a00 con 0x557042bd1800 2023-10-08T09:15:38.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.268+0000 7fa3f8646b80 10 client.15897 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:38.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.268+0000 7fa3f8646b80 10 client.15897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-08T09:15:38.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.268+0000 7fa3f8646b80 20 client.15897 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:38.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.268+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042c4a8f0 0x557042c4cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.268+0000 7fa3f8646b80 4 client.15897 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:15:38.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.268+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 --> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557042b1bb30 con 0x557042c4a8f0 2023-10-08T09:15:38.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.269+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042c4a8f0 0x557042c4cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.271+0000 7fa3ea1f9700 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042c4a8f0 0x557042c4cd10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e85e0d554c09f2d server_cookie=b6df335b93cf73b8 in_seq=0 out_seq=0 2023-10-08T09:15:38.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.271+0000 7fa3cdffb700 10 client.15897 ms_handle_connect on v2:172.21.15.184:6838/2795080121 2023-10-08T09:15:38.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.271+0000 7fa3cdffb700 1 -- 172.21.15.49:0/264281783 <== mds.1 v2:172.21.15.184:6838/2795080121 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557042b1bb30 con 0x557042c4a8f0 2023-10-08T09:15:38.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.271+0000 7fa3cdffb700 10 client.15897 handle_command_reply: tid=0 2023-10-08T09:15:38.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.271+0000 7fa3f8646b80 1 client.15897 shutdown 2023-10-08T09:15:38.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 msgr2=0x557042bdfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bdfbf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000c530 tx=0x7fa3e00a3920 comp rx=0 tx=0).stop 2023-10-08T09:15:38.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 shutdown_connections 2023-10-08T09:15:38.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.184:6838/2795080121,v1:172.21.15.184:6839/2795080121] conn(0x557042c4a8f0 0x557042c4cd10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557042bd0060 0x557042bdc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557042bd1800 0x557042bdfbf0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 --2- 172.21.15.49:0/264281783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557042bd09c0 0x557042bdcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.276+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 >> 172.21.15.49:0/264281783 conn(0x557042b44260 msgr2=0x557042bd7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.277+0000 7fa3cdffb700 0 client.15897 ms_handle_reset on v2:172.21.15.184:6838/2795080121 2023-10-08T09:15:38.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.277+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 shutdown_connections 2023-10-08T09:15:38.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.277+0000 7fa3f8646b80 1 -- 172.21.15.49:0/264281783 wait complete. 2023-10-08T09:15:38.289 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2023-10-08T09:15:38.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.289+0000 7fa3f8646b80 20 client.15897 trim_cache size 0 max 16384 2023-10-08T09:15:38.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa3d00a15d0 msgr2=0x7fa3d00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa3d00a15d0 0x7fa3d00a3a90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4006fd0 tx=0x7fa3d4009f90 comp rx=0 tx=0).stop 2023-10-08T09:15:38.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e410db60 msgr2=0x7fa3e407a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e410db60 0x7fa3e407a360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d900 tx=0x7fa3e0009ee0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 shutdown_connections 2023-10-08T09:15:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa3d00a15d0 0x7fa3d00a3a90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3e410db60 0x7fa3e407a360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa3e4108900 0x7fa3e4072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 --2- 172.21.15.49:0/3559821150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3e410b230 0x7fa3e4073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.291+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 >> 172.21.15.49:0/3559821150 conn(0x7fa3e406d790 msgr2=0x7fa3e4109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.294+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 shutdown_connections 2023-10-08T09:15:38.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.296+0000 7fa3ec45d700 1 -- 172.21.15.49:0/3559821150 wait complete. 2023-10-08T09:15:38.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 -- 172.21.15.49:0/3938763542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb60100010 msgr2=0x7fdb60100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 --2- 172.21.15.49:0/3938763542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb60100010 0x7fdb60100490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800aad0 tx=0x7fdb4800ade0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 -- 172.21.15.49:0/3938763542 shutdown_connections 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 --2- 172.21.15.49:0/3938763542 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb601009d0 0x7fdb601072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 --2- 172.21.15.49:0/3938763542 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb600fef90 0x7fdb600ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 --2- 172.21.15.49:0/3938763542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb60100010 0x7fdb60100490 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 -- 172.21.15.49:0/3938763542 >> 172.21.15.49:0/3938763542 conn(0x7fdb600f9f50 msgr2=0x7fdb600fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 -- 172.21.15.49:0/3938763542 shutdown_connections 2023-10-08T09:15:38.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.543+0000 7fdb6651f700 1 -- 172.21.15.49:0/3938763542 wait complete. 2023-10-08T09:15:38.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 Processor -- start 2023-10-08T09:15:38.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 -- start start 2023-10-08T09:15:38.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb600fef90 0x7fdb6019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb60100010 0x7fdb6019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb601009d0 0x7fdb601a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601a4930 con 0x7fdb601009d0 2023-10-08T09:15:38.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601a4a70 con 0x7fdb600fef90 2023-10-08T09:15:38.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.544+0000 7fdb6651f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601a4c00 con 0x7fdb60100010 2023-10-08T09:15:38.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb600fef90 0x7fdb6019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb60100010 0x7fdb6019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb600fef90 0x7fdb6019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40124/0 (socket says 172.21.15.49:40124) 2023-10-08T09:15:38.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5ffff700 1 -- 172.21.15.49:0/2010354637 learned_addr learned my addr 172.21.15.49:0/2010354637 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5f7fe700 1 -- 172.21.15.49:0/2010354637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb600fef90 msgr2=0x7fdb6019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5f7fe700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb600fef90 0x7fdb6019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5f7fe700 1 -- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb601009d0 msgr2=0x7fdb601a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5f7fe700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb601009d0 0x7fdb601a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5f7fe700 1 -- 172.21.15.49:0/2010354637 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4800a700 con 0x7fdb60100010 2023-10-08T09:15:38.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.545+0000 7fdb5f7fe700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb60100010 0x7fdb6019d340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c690 tx=0x7fdb4800ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.546+0000 7fdb5d7fa700 1 -- 172.21.15.49:0/2010354637 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48004030 con 0x7fdb60100010 2023-10-08T09:15:38.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.546+0000 7fdb5d7fa700 1 -- 172.21.15.49:0/2010354637 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb48004190 con 0x7fdb60100010 2023-10-08T09:15:38.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.546+0000 7fdb5d7fa700 1 -- 172.21.15.49:0/2010354637 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48016680 con 0x7fdb60100010 2023-10-08T09:15:38.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.547+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb601a4e80 con 0x7fdb60100010 2023-10-08T09:15:38.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.547+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb601a51f0 con 0x7fdb60100010 2023-10-08T09:15:38.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.552+0000 7fdb72708b80 1 Processor -- start 2023-10-08T09:15:38.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.552+0000 7fdb72708b80 1 -- start start 2023-10-08T09:15:38.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.552+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e810070 0x561a1e810490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.552+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e811280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.552+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e811810 0x561a1e811ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb72708b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e767460 con 0x561a1e8109d0 2023-10-08T09:15:38.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb72708b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e758c20 con 0x561a1e810070 2023-10-08T09:15:38.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb72708b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e756540 con 0x561a1e811810 2023-10-08T09:15:38.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb64abc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e810070 0x561a1e810490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb64abc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e810070 0x561a1e810490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40132/0 (socket says 172.21.15.49:40132) 2023-10-08T09:15:38.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb64abc700 1 -- 172.21.15.49:0/1026439859 learned_addr learned my addr 172.21.15.49:0/1026439859 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb5d7fa700 1 -- 172.21.15.49:0/2010354637 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdb4800e450 con 0x7fdb60100010 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb5f7fe700 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e811810 0x561a1e811ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.553+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e811280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb64abc700 1 -- 172.21.15.49:0/1026439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e811810 msgr2=0x561a1e811ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e811810 0x561a1e811ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb64abc700 1 -- 172.21.15.49:0/1026439859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 msgr2=0x561a1e811280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e811280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb64abc700 1 -- 172.21.15.49:0/1026439859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a1e57cde0 con 0x561a1e810070 2023-10-08T09:15:38.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e810070 0x561a1e810490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009c60 tx=0x7fdb58009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3adb6659c8cc63b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb5d7fa700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb4c0a1600 0x7fdb4c0a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb567fc700 1 -- 172.21.15.49:0/1026439859 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58017070 con 0x561a1e810070 2023-10-08T09:15:38.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb567fc700 1 -- 172.21.15.49:0/1026439859 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5800f040 con 0x561a1e810070 2023-10-08T09:15:38.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb567fc700 1 -- 172.21.15.49:0/1026439859 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58005880 con 0x561a1e810070 2023-10-08T09:15:38.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb5d7fa700 1 -- 172.21.15.49:0/2010354637 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdb48012070 con 0x7fdb60100010 2023-10-08T09:15:38.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.554+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb4c0a1600 0x7fdb4c0a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 -- 172.21.15.49:0/1026439859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e810070 msgr2=0x561a1e810490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb4c0a1600 0x7fdb4c0a3ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x561a1e810e10 tx=0x7fdb50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e810070 0x561a1e810490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009c60 tx=0x7fdb58009f70 comp rx=0 tx=0).stop 2023-10-08T09:15:38.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 -- 172.21.15.49:0/1026439859 shutdown_connections 2023-10-08T09:15:38.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e811810 0x561a1e811ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e810070 0x561a1e810490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1026439859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e811280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 -- 172.21.15.49:0/1026439859 >> 172.21.15.49:0/1026439859 conn(0x561a1e773260 msgr2=0x561a1e7f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.555+0000 7fdb72708b80 1 -- 172.21.15.49:0/1026439859 shutdown_connections 2023-10-08T09:15:38.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.556+0000 7fdb72708b80 1 -- 172.21.15.49:0/1026439859 wait complete. 2023-10-08T09:15:38.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.556+0000 7fdb72708b80 1 -- start start 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.556+0000 7fdb72708b80 1 Processor -- start 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e819de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e81a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e811810 0x561a1e821360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb72708b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e751e30 con 0x561a1e8109d0 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb72708b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e6ee710 con 0x561a1e811810 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb72708b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e6ee3b0 con 0x561a1e810070 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb72708b80 10 client.0 _ll_register_callbacks cb 0x561a1e54b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb64abc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e811810 0x561a1e821360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb64abc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e811810 0x561a1e821360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40138/0 (socket says 172.21.15.49:40138) 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb64abc700 1 -- 172.21.15.49:0/1851794118 learned_addr learned my addr 172.21.15.49:0/1851794118 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.557+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e819de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb5f7fe700 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e81a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb64abc700 1 -- 172.21.15.49:0/1851794118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 msgr2=0x561a1e819de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e819de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb64abc700 1 -- 172.21.15.49:0/1851794118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 msgr2=0x561a1e81a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e81a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb64abc700 1 -- 172.21.15.49:0/1851794118 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb580098c0 con 0x561a1e811810 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e811810 0x561a1e821360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb58000f80 tx=0x7fdb58009390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7969a58a286f23c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1851794118 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58017040 con 0x561a1e811810 2023-10-08T09:15:38.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a1e719eb0 con 0x561a1e811810 2023-10-08T09:15:38.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.558+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1851794118 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdb58005a10 con 0x561a1e811810 2023-10-08T09:15:38.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.559+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1851794118 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb58003c00 con 0x561a1e811810 2023-10-08T09:15:38.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.559+0000 7fdb72708b80 10 client.25538 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:38.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.559+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561a1e565c60 con 0x561a1e811810 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1851794118 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a1e719eb0 con 0x561a1e811810 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1851794118 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdb58009c90 con 0x561a1e811810 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb72708b80 10 client.25538 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb72708b80 10 client.25538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb72708b80 20 client.25538 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e836b50 0x561a1e838f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb72708b80 4 client.25538 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.560+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a1e74ae10 con 0x561a1e836b50 2023-10-08T09:15:38.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.561+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e836b50 0x561a1e838f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.561+0000 7fdb3f7fe700 10 client.25538 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.561+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e836b50 0x561a1e838f70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8566588fef13ee0 server_cookie=94696d20d953d251 in_seq=0 out_seq=0 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1851794118 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a1e74ae10 con 0x561a1e836b50 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb3f7fe700 10 client.25538 handle_command_reply: tid=0 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb72708b80 1 client.25538 shutdown 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e811810 msgr2=0x561a1e821360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e811810 0x561a1e821360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb58000f80 tx=0x7fdb58009390 comp rx=0 tx=0).stop 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 shutdown_connections 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e836b50 0x561a1e838f70 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.562+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e819de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.563+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e811810 0x561a1e821360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.563+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1851794118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e8109d0 0x561a1e81a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.563+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 >> 172.21.15.49:0/1851794118 conn(0x561a1e773260 msgr2=0x561a1e7f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.563+0000 7fdb3f7fe700 0 client.25538 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:38.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.563+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 shutdown_connections 2023-10-08T09:15:38.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.563+0000 7fdb72708b80 1 -- 172.21.15.49:0/1851794118 wait complete. 2023-10-08T09:15:38.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.563+0000 7fdb72708b80 20 client.25538 trim_cache size 0 max 16384 2023-10-08T09:15:38.588 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 Processor -- start 2023-10-08T09:15:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 -- start start 2023-10-08T09:15:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e810450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 0x561a1e821320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e817d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e5d2920 con 0x561a1e811810 2023-10-08T09:15:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e834080 con 0x561a1e8109d0 2023-10-08T09:15:38.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.588+0000 7fdb72708b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e7f6ae0 con 0x561a1e810070 2023-10-08T09:15:38.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e810450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e810450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37566/0 (socket says 172.21.15.49:37566) 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 -- 172.21.15.49:0/1940125670 learned_addr learned my addr 172.21.15.49:0/1940125670 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e817d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 -- 172.21.15.49:0/1940125670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 msgr2=0x561a1e821320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 0x561a1e821320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 -- 172.21.15.49:0/1940125670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 msgr2=0x561a1e817d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e817d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.589+0000 7fdb5ffff700 1 -- 172.21.15.49:0/1940125670 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a1e5a3250 con 0x561a1e810070 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.590+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e810450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009fd0 tx=0x7fdb50017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba42d7a60bd2fb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.590+0000 7fdb56ffd700 1 -- 172.21.15.49:0/1940125670 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50027040 con 0x561a1e810070 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.590+0000 7fdb56ffd700 1 -- 172.21.15.49:0/1940125670 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb50015850 con 0x561a1e810070 2023-10-08T09:15:38.591 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.590+0000 7fdb56ffd700 1 -- 172.21.15.49:0/1940125670 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50028bf0 con 0x561a1e810070 2023-10-08T09:15:38.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.591+0000 7fdb72708b80 1 -- 172.21.15.49:0/1940125670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 msgr2=0x561a1e810450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.591+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e810450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009fd0 tx=0x7fdb50017000 comp rx=0 tx=0).stop 2023-10-08T09:15:38.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.592+0000 7fdb72708b80 1 -- 172.21.15.49:0/1940125670 shutdown_connections 2023-10-08T09:15:38.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.592+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e810070 0x561a1e810450 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009fd0 tx=0x7fdb50017000 comp rx=0 tx=0).stop 2023-10-08T09:15:38.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.592+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e817d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.592+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1940125670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 0x561a1e821320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.592+0000 7fdb72708b80 1 -- 172.21.15.49:0/1940125670 >> 172.21.15.49:0/1940125670 conn(0x561a1e773260 msgr2=0x561a1e7f5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.594+0000 7fdb72708b80 1 -- 172.21.15.49:0/1940125670 shutdown_connections 2023-10-08T09:15:38.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 -- 172.21.15.49:0/1940125670 wait complete. 2023-10-08T09:15:38.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 -- start start 2023-10-08T09:15:38.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 Processor -- start 2023-10-08T09:15:38.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 0x561a1e820b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e821090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e81d5f0 0x561a1e81da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e81dfb0 con 0x561a1e811810 2023-10-08T09:15:38.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e81e120 con 0x561a1e8109d0 2023-10-08T09:15:38.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a1e88a970 con 0x561a1e81d5f0 2023-10-08T09:15:38.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.595+0000 7fdb72708b80 10 client.0 _ll_register_callbacks cb 0x7fdb480d1670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:38.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e821090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb5f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e821090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50538/0 (socket says 172.21.15.49:50538) 2023-10-08T09:15:38.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb5f7fe700 1 -- 172.21.15.49:0/1433544487 learned_addr learned my addr 172.21.15.49:0/1433544487 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:38.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 0x561a1e820b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e81d5f0 0x561a1e81da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb64abc700 1 -- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 msgr2=0x561a1e820b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 0x561a1e820b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb64abc700 1 -- 172.21.15.49:0/1433544487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 msgr2=0x561a1e821090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:38.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e821090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.596+0000 7fdb64abc700 1 -- 172.21.15.49:0/1433544487 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb50011930 con 0x561a1e81d5f0 2023-10-08T09:15:38.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.597+0000 7fdb64abc700 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e81d5f0 0x561a1e81da70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009890 tx=0x7fdb58006070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33be74bf6acc763b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.597+0000 7fdb3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.597+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1433544487 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb58017040 con 0x561a1e81d5f0 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.597+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1433544487 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdb58005a10 con 0x561a1e81d5f0 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.597+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb5800f040 con 0x561a1e81d5f0 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1433544487 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb5801cb30 con 0x561a1e81d5f0 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 10 client.25564 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561a1e57b2b0 con 0x561a1e81d5f0 2023-10-08T09:15:38.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1433544487 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5800f040 con 0x561a1e81d5f0 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1433544487 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdb58012070 con 0x561a1e81d5f0 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 10 client.25564 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 20 client.25564 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e88bee0 0x561a1e88e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 4 client.25564 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a1e74ab30 con 0x561a1e88bee0 2023-10-08T09:15:38.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.598+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e88bee0 0x561a1e88e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb3f7fe700 10 client.25564 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb5ffff700 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e88bee0 0x561a1e88e300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3700be33c78c52c0 server_cookie=c7ed92e9dcdd518f in_seq=0 out_seq=0 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb3f7fe700 1 -- 172.21.15.49:0/1433544487 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a1e74ab30 con 0x561a1e88bee0 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb3f7fe700 10 client.25564 handle_command_reply: tid=0 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 client.25564 shutdown 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e81d5f0 msgr2=0x561a1e81da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e81d5f0 0x561a1e81da70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009890 tx=0x7fdb58006070 comp rx=0 tx=0).stop 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 shutdown_connections 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a1e88bee0 0x561a1e88e300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a1e81d5f0 0x561a1e81da70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a1e8109d0 0x561a1e820b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 --2- 172.21.15.49:0/1433544487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a1e811810 0x561a1e821090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb3f7fe700 0 client.25564 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:38.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.599+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 >> 172.21.15.49:0/1433544487 conn(0x561a1e773260 msgr2=0x561a1e814710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.600+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 shutdown_connections 2023-10-08T09:15:38.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.600+0000 7fdb72708b80 1 -- 172.21.15.49:0/1433544487 wait complete. 2023-10-08T09:15:38.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.600+0000 7fdb72708b80 20 client.25564 trim_cache size 0 max 16384 2023-10-08T09:15:38.603 INFO:teuthology.orchestra.run.smithi049.stdout:[] 2023-10-08T09:15:38.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb4c0a1600 msgr2=0x7fdb4c0a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb4c0a1600 0x7fdb4c0a3ac0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x561a1e810e10 tx=0x7fdb50008040 comp rx=0 tx=0).stop 2023-10-08T09:15:38.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb60100010 msgr2=0x7fdb6019d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb60100010 0x7fdb6019d340 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c690 tx=0x7fdb4800ce10 comp rx=0 tx=0).stop 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 shutdown_connections 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdb4c0a1600 0x7fdb4c0a3ac0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb601009d0 0x7fdb601a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb600fef90 0x7fdb6019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 --2- 172.21.15.49:0/2010354637 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb60100010 0x7fdb6019d340 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 >> 172.21.15.49:0/2010354637 conn(0x7fdb600f9f50 msgr2=0x7fdb60101460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 shutdown_connections 2023-10-08T09:15:38.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:38.603+0000 7fdb6651f700 1 -- 172.21.15.49:0/2010354637 wait complete. 2023-10-08T09:15:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 15.0 s, 3144.9 tps, lat 31.500 ms stddev 38.351 2023-10-08T09:15:39.628 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-08T09:15:39.628 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-08T09:15:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:39 smithi132 ceph-mon[103389]: pgmap v351: 129 pgs: 129 active+clean; 1.2 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 96 MiB/s wr, 1.06k op/s 2023-10-08T09:15:39.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:39 smithi049 ceph-mon[99485]: pgmap v351: 129 pgs: 129 active+clean; 1.2 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 96 MiB/s wr, 1.06k op/s 2023-10-08T09:15:39.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.888+0000 7f240b890700 1 -- 172.21.15.49:0/4108350300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24041077f0 msgr2=0x7f2404109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.888+0000 7f240b890700 1 --2- 172.21.15.49:0/4108350300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24041077f0 0x7f2404109c20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009990 tx=0x7f23f4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.890+0000 7f240b890700 1 -- 172.21.15.49:0/4108350300 shutdown_connections 2023-10-08T09:15:39.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.890+0000 7f240b890700 1 --2- 172.21.15.49:0/4108350300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f240410cb30 0x7f240410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.890+0000 7f240b890700 1 --2- 172.21.15.49:0/4108350300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24041077f0 0x7f2404109c20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.890+0000 7f240b890700 1 --2- 172.21.15.49:0/4108350300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410a160 0x7f240410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.890+0000 7f240b890700 1 -- 172.21.15.49:0/4108350300 >> 172.21.15.49:0/4108350300 conn(0x7f240406d790 msgr2=0x7f240406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:39.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.890+0000 7f240b890700 1 -- 172.21.15.49:0/4108350300 shutdown_connections 2023-10-08T09:15:39.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.890+0000 7f240b890700 1 -- 172.21.15.49:0/4108350300 wait complete. 2023-10-08T09:15:39.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.891+0000 7f240b890700 1 Processor -- start 2023-10-08T09:15:39.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.891+0000 7f240b890700 1 -- start start 2023-10-08T09:15:39.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.892+0000 7f240b890700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24041077f0 0x7f24041a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.892+0000 7f240b890700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f240410a160 0x7f24041a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.892+0000 7f240b890700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410cb30 0x7f24041aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.892+0000 7f240b890700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24041ad050 con 0x7f24041077f0 2023-10-08T09:15:39.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.892+0000 7f240b890700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240410ef20 con 0x7f240410cb30 2023-10-08T09:15:39.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.892+0000 7f240b890700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240410f0b0 con 0x7f240410a160 2023-10-08T09:15:39.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.893+0000 7f2409e2d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410cb30 0x7f24041aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.893+0000 7f2409e2d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410cb30 0x7f24041aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40166/0 (socket says 172.21.15.49:40166) 2023-10-08T09:15:39.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.893+0000 7f2409e2d700 1 -- 172.21.15.49:0/3983035540 learned_addr learned my addr 172.21.15.49:0/3983035540 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:39.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.894+0000 7f2409e2d700 1 -- 172.21.15.49:0/3983035540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f240410a160 msgr2=0x7f24041a5ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:15:39.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.894+0000 7f2409e2d700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f240410a160 0x7f24041a5ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.894+0000 7f2409e2d700 1 -- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24041077f0 msgr2=0x7f24041a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.894+0000 7f2409e2d700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24041077f0 0x7f24041a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.894+0000 7f2409e2d700 1 -- 172.21.15.49:0/3983035540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f40095f0 con 0x7f240410cb30 2023-10-08T09:15:39.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.894+0000 7f2409e2d700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410cb30 0x7f24041aca80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f240000e940 tx=0x7f240000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:39.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.896+0000 7f23fa7fc700 1 -- 172.21.15.49:0/3983035540 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24000097b0 con 0x7f240410cb30 2023-10-08T09:15:39.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.896+0000 7f23fa7fc700 1 -- 172.21.15.49:0/3983035540 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2400004cf0 con 0x7f240410cb30 2023-10-08T09:15:39.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.896+0000 7f23fa7fc700 1 -- 172.21.15.49:0/3983035540 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24000055d0 con 0x7f240410cb30 2023-10-08T09:15:39.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.896+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240410f390 con 0x7f240410cb30 2023-10-08T09:15:39.897 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.896+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240410f850 con 0x7f240410cb30 2023-10-08T09:15:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.902+0000 7f2417a79b80 1 Processor -- start 2023-10-08T09:15:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.902+0000 7f23fa7fc700 1 -- 172.21.15.49:0/3983035540 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f240000cc60 con 0x7f240410cb30 2023-10-08T09:15:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.902+0000 7f2417a79b80 1 -- start start 2023-10-08T09:15:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.902+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982a0e0 0x563b7982a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.902+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982aa40 0x563b7982b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b7982bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2417a79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b797924e0 con 0x563b7982a0e0 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2417a79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79783ca0 con 0x563b7982aa40 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2417a79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b797815c0 con 0x563b7982b880 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2408e2b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982a0e0 0x563b7982a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2408e2b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982a0e0 0x563b7982a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50558/0 (socket says 172.21.15.49:50558) 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2408e2b700 1 -- 172.21.15.49:0/3692029699 learned_addr learned my addr 172.21.15.49:0/3692029699 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:39.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f2408e2b700 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b7982bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.903+0000 7f23fa7fc700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f23f00a15d0 0x7f23f00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f23fa7fc700 1 -- 172.21.15.49:0/3983035540 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2400014070 con 0x7f240410cb30 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f2409e2d700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f23f00a15d0 0x7f23f00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f240962c700 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982aa40 0x563b7982b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f240962c700 1 -- 172.21.15.49:0/3692029699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 msgr2=0x563b7982bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f240962c700 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b7982bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f240962c700 1 -- 172.21.15.49:0/3692029699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982a0e0 msgr2=0x563b7982a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f240962c700 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982a0e0 0x563b7982a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f240962c700 1 -- 172.21.15.49:0/3692029699 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b795a7e60 con 0x563b7982aa40 2023-10-08T09:15:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.904+0000 7f2409e2d700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f23f00a15d0 0x7f23f00a3a90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2400096cb0 tx=0x7f2400005020 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.905+0000 7f240962c700 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982aa40 0x563b7982b2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23f40098e0 tx=0x7f23f4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2c748050778149b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.905+0000 7f23ef7fe700 1 -- 172.21.15.49:0/3692029699 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f401d070 con 0x563b7982aa40 2023-10-08T09:15:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.905+0000 7f23ef7fe700 1 -- 172.21.15.49:0/3692029699 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f4004cf0 con 0x563b7982aa40 2023-10-08T09:15:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.905+0000 7f23ef7fe700 1 -- 172.21.15.49:0/3692029699 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4005280 con 0x563b7982aa40 2023-10-08T09:15:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 -- 172.21.15.49:0/3692029699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982aa40 msgr2=0x563b7982b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982aa40 0x563b7982b2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23f40098e0 tx=0x7f23f4009580 comp rx=0 tx=0).stop 2023-10-08T09:15:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 -- 172.21.15.49:0/3692029699 shutdown_connections 2023-10-08T09:15:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b7982bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982a0e0 0x563b7982a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3692029699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982aa40 0x563b7982b2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 -- 172.21.15.49:0/3692029699 >> 172.21.15.49:0/3692029699 conn(0x563b7979e2e0 msgr2=0x563b79824e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 -- 172.21.15.49:0/3692029699 shutdown_connections 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.906+0000 7f2417a79b80 1 -- 172.21.15.49:0/3692029699 wait complete. 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 -- start start 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 Processor -- start 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982a0e0 0x563b79833df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b79834330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982b880 0x563b7983b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7977ceb0 con 0x563b7982aa40 2023-10-08T09:15:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79719790 con 0x563b7982b880 2023-10-08T09:15:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79719430 con 0x563b7982a0e0 2023-10-08T09:15:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.907+0000 7f2417a79b80 10 client.0 _ll_register_callbacks cb 0x563b79576880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982a0e0 0x563b79833df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982a0e0 0x563b79833df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37602/0 (socket says 172.21.15.49:37602) 2023-10-08T09:15:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 -- 172.21.15.49:0/427913354 learned_addr learned my addr 172.21.15.49:0/427913354 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982b880 0x563b7983b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f240962c700 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b79834330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 -- 172.21.15.49:0/427913354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982a0e0 msgr2=0x563b79833df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982a0e0 0x563b79833df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 -- 172.21.15.49:0/427913354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 msgr2=0x563b79834330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b79834330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.908+0000 7f2408e2b700 1 -- 172.21.15.49:0/427913354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240410dae0 con 0x563b7982b880 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.909+0000 7f2408e2b700 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982b880 0x563b7983b370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23fc014f40 tx=0x7f23fc007960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=233bd6aece280161 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.909+0000 7f23ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.909+0000 7f23ecff9700 1 -- 172.21.15.49:0/427913354 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc016620 con 0x563b7982b880 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.909+0000 7f23ecff9700 1 -- 172.21.15.49:0/427913354 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23fc016780 con 0x563b7982b880 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.909+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b79744f30 con 0x563b7982b880 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.910+0000 7f23ecff9700 1 -- 172.21.15.49:0/427913354 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23fc0148d0 con 0x563b7982b880 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.910+0000 7f2417a79b80 10 client.25550 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.910+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563b79590ce0 con 0x563b7982b880 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.910+0000 7f23ecff9700 1 -- 172.21.15.49:0/427913354 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563b79744f30 con 0x563b7982b880 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.910+0000 7f23ecff9700 1 -- 172.21.15.49:0/427913354 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f23fc019810 con 0x563b7982b880 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.911+0000 7f2417a79b80 10 client.25550 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.911+0000 7f2417a79b80 10 client.25550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.911+0000 7f2417a79b80 20 client.25550 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.911+0000 7f2417a79b80 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b7983fc00 0x563b79842020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.911+0000 7f2417a79b80 4 client.25550 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.911+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b79775e90 con 0x563b7983fc00 2023-10-08T09:15:39.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.912+0000 7f240962c700 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b7983fc00 0x563b79842020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.913+0000 7f23ecff9700 10 client.25550 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.913+0000 7f240962c700 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b7983fc00 0x563b79842020 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee24055d4555b70 server_cookie=a6a85bfbd3bf0156 in_seq=0 out_seq=0 2023-10-08T09:15:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.913+0000 7f23ecff9700 1 -- 172.21.15.49:0/427913354 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563b79775e90 con 0x563b7983fc00 2023-10-08T09:15:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.913+0000 7f23ecff9700 10 client.25550 handle_command_reply: tid=0 2023-10-08T09:15:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 client.25550 shutdown 2023-10-08T09:15:39.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982b880 msgr2=0x563b7983b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982b880 0x563b7983b370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23fc014f40 tx=0x7f23fc007960 comp rx=0 tx=0).stop 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 shutdown_connections 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b7983fc00 0x563b79842020 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982a0e0 0x563b79833df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982b880 0x563b7983b370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f23ecff9700 0 client.25550 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 --2- 172.21.15.49:0/427913354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b79834330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 >> 172.21.15.49:0/427913354 conn(0x563b7979e2e0 msgr2=0x563b79824550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 shutdown_connections 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 1 -- 172.21.15.49:0/427913354 wait complete. 2023-10-08T09:15:39.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.914+0000 7f2417a79b80 20 client.25550 trim_cache size 0 max 16384 2023-10-08T09:15:39.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 Processor -- start 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 -- start start 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b79833dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b7983b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b79834310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b795fd9a0 con 0x563b7982aa40 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79833980 con 0x563b7982a0e0 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.948+0000 7f2417a79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7983c4f0 con 0x563b7982b880 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f240962c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b7983b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b79833dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f240962c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b7983b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50576/0 (socket says 172.21.15.49:50576) 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f240962c700 1 -- 172.21.15.49:0/2501118573 learned_addr learned my addr 172.21.15.49:0/2501118573 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b79834310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 -- 172.21.15.49:0/2501118573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 msgr2=0x563b79834310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b79834310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 -- 172.21.15.49:0/2501118573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 msgr2=0x563b7983b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b7983b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 -- 172.21.15.49:0/2501118573 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b795ce2d0 con 0x563b7982a0e0 2023-10-08T09:15:39.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.949+0000 7f2408e2b700 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b79833dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23fc002230 tx=0x7f23fc009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fda6b56ba1150f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.950+0000 7f23effff700 1 -- 172.21.15.49:0/2501118573 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc004a70 con 0x563b7982a0e0 2023-10-08T09:15:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.950+0000 7f23effff700 1 -- 172.21.15.49:0/2501118573 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23fc00d070 con 0x563b7982a0e0 2023-10-08T09:15:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.950+0000 7f23effff700 1 -- 172.21.15.49:0/2501118573 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc005290 con 0x563b7982a0e0 2023-10-08T09:15:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.951+0000 7f2417a79b80 1 -- 172.21.15.49:0/2501118573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 msgr2=0x563b79833dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.951+0000 7f2417a79b80 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b79833dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23fc002230 tx=0x7f23fc009040 comp rx=0 tx=0).stop 2023-10-08T09:15:39.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.952+0000 7f2417a79b80 1 -- 172.21.15.49:0/2501118573 shutdown_connections 2023-10-08T09:15:39.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.952+0000 7f2417a79b80 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982b880 0x563b79834310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.952+0000 7f2417a79b80 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b79833dd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.952+0000 7f2417a79b80 1 --2- 172.21.15.49:0/2501118573 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982aa40 0x563b7983b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.952+0000 7f2417a79b80 1 -- 172.21.15.49:0/2501118573 >> 172.21.15.49:0/2501118573 conn(0x563b7979e2e0 msgr2=0x563b7982e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:39.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.953+0000 7f2417a79b80 1 -- 172.21.15.49:0/2501118573 shutdown_connections 2023-10-08T09:15:39.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.953+0000 7f2417a79b80 1 -- 172.21.15.49:0/2501118573 wait complete. 2023-10-08T09:15:39.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 -- start start 2023-10-08T09:15:39.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 Processor -- start 2023-10-08T09:15:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b7983a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982aa40 0x563b7983ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982b880 0x563b79893b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79894090 con 0x563b7982b880 2023-10-08T09:15:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b798941d0 con 0x563b7982a0e0 2023-10-08T09:15:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79894390 con 0x563b7982aa40 2023-10-08T09:15:39.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.954+0000 7f2417a79b80 10 client.0 _ll_register_callbacks cb 0x7f23fc014e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982aa40 0x563b7983ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982aa40 0x563b7983ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37614/0 (socket says 172.21.15.49:37614) 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 -- 172.21.15.49:0/3481440848 learned_addr learned my addr 172.21.15.49:0/3481440848 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 -- 172.21.15.49:0/3481440848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 msgr2=0x563b7983a5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b7983a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 -- 172.21.15.49:0/3481440848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982b880 msgr2=0x563b79893b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982b880 0x563b79893b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.955+0000 7f240962c700 1 -- 172.21.15.49:0/3481440848 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b79590ce0 con 0x563b7982aa40 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f240962c700 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982aa40 0x563b7983ab10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b680 tx=0x7f23f400b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7159216edfe13399 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f23ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:15:39.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f23ecff9700 1 -- 172.21.15.49:0/3481440848 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f401d040 con 0x563b7982aa40 2023-10-08T09:15:39.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f23ecff9700 1 -- 172.21.15.49:0/3481440848 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f23f400bb00 con 0x563b7982aa40 2023-10-08T09:15:39.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23f4004cf0 con 0x563b7982aa40 2023-10-08T09:15:39.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f23ecff9700 1 -- 172.21.15.49:0/3481440848 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23f400bcc0 con 0x563b7982aa40 2023-10-08T09:15:39.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f2417a79b80 10 client.25576 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:39.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.956+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563b795a6330 con 0x563b7982aa40 2023-10-08T09:15:39.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.957+0000 7f23ecff9700 1 -- 172.21.15.49:0/3481440848 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f4004cf0 con 0x563b7982aa40 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f23ecff9700 1 -- 172.21.15.49:0/3481440848 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f23f4004500 con 0x563b7982aa40 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f2417a79b80 10 client.25576 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f2417a79b80 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f2417a79b80 20 client.25576 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b79895a10 0x563b79897e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f2417a79b80 4 client.25576 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563b79775bb0 con 0x563b79895a10 2023-10-08T09:15:39.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.958+0000 7f2408e2b700 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b79895a10 0x563b79897e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.959+0000 7f2408e2b700 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b79895a10 0x563b79897e30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6b8e95210fd1a6 server_cookie=51e9f970e8484849 in_seq=0 out_seq=0 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.959+0000 7f23ecff9700 10 client.25576 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.959+0000 7f23ecff9700 1 -- 172.21.15.49:0/3481440848 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563b79775bb0 con 0x563b79895a10 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f23ecff9700 10 client.25576 handle_command_reply: tid=0 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 client.25576 shutdown 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982aa40 msgr2=0x563b7983ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982aa40 0x563b7983ab10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b680 tx=0x7f23f400b8e0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 shutdown_connections 2023-10-08T09:15:39.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563b79895a10 0x563b79897e30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563b7982b880 0x563b79893b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563b7982a0e0 0x563b7983a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 --2- 172.21.15.49:0/3481440848 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563b7982aa40 0x563b7983ab10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 >> 172.21.15.49:0/3481440848 conn(0x563b7979e2e0 msgr2=0x563b7982e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:39.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f23ecff9700 0 client.25576 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:39.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 shutdown_connections 2023-10-08T09:15:39.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 1 -- 172.21.15.49:0/3481440848 wait complete. 2023-10-08T09:15:39.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.960+0000 7f2417a79b80 20 client.25576 trim_cache size 0 max 16384 2023-10-08T09:15:39.963 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:15:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "return_code": 0, 2023-10-08T09:15:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:15:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": "asynchronous" 2023-10-08T09:15:39.963 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:15:39.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.961+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f23f00a15d0 msgr2=0x7f23f00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.961+0000 7f240b890700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f23f00a15d0 0x7f23f00a3a90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2400096cb0 tx=0x7f2400005020 comp rx=0 tx=0).stop 2023-10-08T09:15:39.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.961+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410cb30 msgr2=0x7f24041aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:39.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.961+0000 7f240b890700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410cb30 0x7f24041aca80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f240000e940 tx=0x7f240000ed00 comp rx=0 tx=0).stop 2023-10-08T09:15:39.964 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.963+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 shutdown_connections 2023-10-08T09:15:39.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.963+0000 7f240b890700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f23f00a15d0 0x7f23f00a3a90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.963+0000 7f240b890700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f240410cb30 0x7f24041aca80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.963+0000 7f240b890700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24041077f0 0x7f24041a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.963+0000 7f240b890700 1 --2- 172.21.15.49:0/3983035540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f240410a160 0x7f24041a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:39.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.963+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 >> 172.21.15.49:0/3983035540 conn(0x7f240406d790 msgr2=0x7f240410b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:39.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.964+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 shutdown_connections 2023-10-08T09:15:39.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:39.964+0000 7f240b890700 1 -- 172.21.15.49:0/3983035540 wait complete. 2023-10-08T09:15:39.986 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:15:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:39 smithi184 ceph-mon[103675]: pgmap v351: 129 pgs: 129 active+clean; 1.2 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 96 MiB/s wr, 1.06k op/s 2023-10-08T09:15:40.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.212+0000 7f3322d58700 1 -- 172.21.15.49:0/3503796803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f331c077e70 msgr2=0x7f331c07a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.212+0000 7f3322d58700 1 --2- 172.21.15.49:0/3503796803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f331c077e70 0x7f331c07a330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3314009790 tx=0x7f3314009aa0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.212+0000 7f3322d58700 1 -- 172.21.15.49:0/3503796803 shutdown_connections 2023-10-08T09:15:40.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.212+0000 7f3322d58700 1 --2- 172.21.15.49:0/3503796803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f331c072650 0x7f331c072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.212+0000 7f3322d58700 1 --2- 172.21.15.49:0/3503796803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f331c077e70 0x7f331c07a330 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.212+0000 7f3322d58700 1 --2- 172.21.15.49:0/3503796803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 0x7f331c077930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.212+0000 7f3322d58700 1 -- 172.21.15.49:0/3503796803 >> 172.21.15.49:0/3503796803 conn(0x7f331c06d790 msgr2=0x7f331c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:40.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.214+0000 7f3322d58700 1 -- 172.21.15.49:0/3503796803 shutdown_connections 2023-10-08T09:15:40.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.214+0000 7f3322d58700 1 -- 172.21.15.49:0/3503796803 wait complete. 2023-10-08T09:15:40.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 Processor -- start 2023-10-08T09:15:40.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 -- start start 2023-10-08T09:15:40.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f331c072650 0x7f331c081c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 0x7f331c082190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f331c077e70 0x7f331c07e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c07ef80 con 0x7f331c072650 2023-10-08T09:15:40.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c07f0f0 con 0x7f331c0754a0 2023-10-08T09:15:40.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.215+0000 7f3322d58700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c07f2b0 con 0x7f331c077e70 2023-10-08T09:15:40.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 0x7f331c082190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 0x7f331c082190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40204/0 (socket says 172.21.15.49:40204) 2023-10-08T09:15:40.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 -- 172.21.15.49:0/1986602377 learned_addr learned my addr 172.21.15.49:0/1986602377 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:40.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f33212f5700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f331c077e70 0x7f331c07e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 -- 172.21.15.49:0/1986602377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f331c077e70 msgr2=0x7f331c07e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f331c077e70 0x7f331c07e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 -- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f331c072650 msgr2=0x7f331c081c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f331c072650 0x7f331c081c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.216+0000 7f331bfff700 1 -- 172.21.15.49:0/1986602377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040095f0 con 0x7f331c0754a0 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.217+0000 7f331bfff700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 0x7f331c082190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f330400ec10 tx=0x7f330400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.217+0000 7f3319ffb700 1 -- 172.21.15.49:0/1986602377 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330400cde0 con 0x7f331c0754a0 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.217+0000 7f3319ffb700 1 -- 172.21.15.49:0/1986602377 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33040044e0 con 0x7f331c0754a0 2023-10-08T09:15:40.218 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.217+0000 7f3319ffb700 1 -- 172.21.15.49:0/1986602377 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304010490 con 0x7f331c0754a0 2023-10-08T09:15:40.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.217+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33140093f0 con 0x7f331c0754a0 2023-10-08T09:15:40.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.218+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c1b2110 con 0x7f331c0754a0 2023-10-08T09:15:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.222+0000 7f3319ffb700 1 -- 172.21.15.49:0/1986602377 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3304003780 con 0x7f331c0754a0 2023-10-08T09:15:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.222+0000 7f332ef41b80 1 Processor -- start 2023-10-08T09:15:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.222+0000 7f332ef41b80 1 -- start start 2023-10-08T09:15:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd2f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd30270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd30c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f332ef41b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acc97460 con 0x5613acd2f9c0 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f332ef41b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acc88c20 con 0x5613acd2f060 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f332ef41b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acc86540 con 0x5613acd30800 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd30c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f33212f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd2f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd30c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37628/0 (socket says 172.21.15.49:37628) 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 -- 172.21.15.49:0/3507554538 learned_addr learned my addr 172.21.15.49:0/3507554538 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 -- 172.21.15.49:0/3507554538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 msgr2=0x5613acd2f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f3319ffb700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f33080a1500 0x7f33080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 --2- 172.21.15.49:0/3507554538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd2f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 -- 172.21.15.49:0/3507554538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 msgr2=0x5613acd30270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 --2- 172.21.15.49:0/3507554538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd30270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.223+0000 7f331bfff700 1 -- 172.21.15.49:0/3507554538 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613acaacde0 con 0x5613acd30800 2023-10-08T09:15:40.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.224+0000 7f33212f5700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f33080a1500 0x7f33080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.224+0000 7f331bfff700 1 --2- 172.21.15.49:0/3507554538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd30c90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33040aca40 tx=0x7f3304096ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6624247e8520f915 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:40.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.224+0000 7f3319ffb700 1 -- 172.21.15.49:0/1986602377 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3304014070 con 0x7f331c0754a0 2023-10-08T09:15:40.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.224+0000 7f3312ffd700 1 -- 172.21.15.49:0/3507554538 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330409a6b0 con 0x5613acd30800 2023-10-08T09:15:40.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.224+0000 7f3312ffd700 1 -- 172.21.15.49:0/3507554538 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33040d2070 con 0x5613acd30800 2023-10-08T09:15:40.226 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.224+0000 7f3312ffd700 1 -- 172.21.15.49:0/3507554538 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330409a240 con 0x5613acd30800 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f332ef41b80 1 -- 172.21.15.49:0/3507554538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 msgr2=0x5613acd30c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f332ef41b80 1 --2- 172.21.15.49:0/3507554538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd30c90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33040aca40 tx=0x7f3304096ca0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f33212f5700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f33080a1500 0x7f33080a39c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3314005e90 tx=0x7f3314009f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f332ef41b80 1 -- 172.21.15.49:0/3507554538 shutdown_connections 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f332ef41b80 1 --2- 172.21.15.49:0/3507554538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd30c90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f332ef41b80 1 --2- 172.21.15.49:0/3507554538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f332ef41b80 1 --2- 172.21.15.49:0/3507554538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd30270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.225+0000 7f332ef41b80 1 -- 172.21.15.49:0/3507554538 >> 172.21.15.49:0/3507554538 conn(0x5613acca3260 msgr2=0x5613acd29e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.227+0000 7f332ef41b80 1 -- 172.21.15.49:0/3507554538 shutdown_connections 2023-10-08T09:15:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.227+0000 7f332ef41b80 1 -- 172.21.15.49:0/3507554538 wait complete. 2023-10-08T09:15:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.227+0000 7f332ef41b80 1 -- start start 2023-10-08T09:15:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 1 Processor -- start 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd38dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd39310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acc81e30 con 0x5613acd2f9c0 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acc1e710 con 0x5613acd2f060 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acc1e3b0 con 0x5613acd30800 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.228+0000 7f332ef41b80 10 client.0 _ll_register_callbacks cb 0x5613aca7b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd38dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd38dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40216/0 (socket says 172.21.15.49:40216) 2023-10-08T09:15:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 -- 172.21.15.49:0/1529987108 learned_addr learned my addr 172.21.15.49:0/1529987108 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f331bfff700 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd39310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f33212f5700 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 -- 172.21.15.49:0/1529987108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 msgr2=0x5613acd40350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd40350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 -- 172.21.15.49:0/1529987108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 msgr2=0x5613acd39310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd39310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 -- 172.21.15.49:0/1529987108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c1b2110 con 0x5613acd2f060 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f3320af4700 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd38dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f330c009fd0 tx=0x7f330c007b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4884215c937d744 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:40.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.229+0000 7f32fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.230+0000 7f32fbfff700 1 -- 172.21.15.49:0/1529987108 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c004db0 con 0x5613acd2f060 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.230+0000 7f32fbfff700 1 -- 172.21.15.49:0/1529987108 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f330c0125a0 con 0x5613acd2f060 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.230+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613acc49eb0 con 0x5613acd2f060 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.230+0000 7f32fbfff700 1 -- 172.21.15.49:0/1529987108 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f330c012760 con 0x5613acd2f060 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.230+0000 7f332ef41b80 10 client.25565 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.230+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5613aca95c60 con 0x5613acd2f060 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f32fbfff700 1 -- 172.21.15.49:0/1529987108 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613acc49eb0 con 0x5613acd2f060 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f32fbfff700 1 -- 172.21.15.49:0/1529987108 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f330c015a40 con 0x5613acd2f060 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f332ef41b80 10 client.25565 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:40.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f332ef41b80 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f332ef41b80 20 client.25565 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f332ef41b80 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd44c40 0x5613acd47060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f332ef41b80 4 client.25565 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613acc7ae10 con 0x5613acd44c40 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.231+0000 7f331bfff700 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd44c40 0x5613acd47060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.232+0000 7f331bfff700 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd44c40 0x5613acd47060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8ea0251330fd74 server_cookie=9207bcad2bcb7257 in_seq=0 out_seq=0 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.232+0000 7f32fbfff700 10 client.25565 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.232+0000 7f32fbfff700 1 -- 172.21.15.49:0/1529987108 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5613acc7ae10 con 0x5613acd44c40 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.232+0000 7f32fbfff700 10 client.25565 handle_command_reply: tid=0 2023-10-08T09:15:40.234 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 client.25565 shutdown 2023-10-08T09:15:40.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 msgr2=0x5613acd38dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd38dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f330c009fd0 tx=0x7f330c007b80 comp rx=0 tx=0).stop 2023-10-08T09:15:40.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 shutdown_connections 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd44c40 0x5613acd47060 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd40350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd38dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 --2- 172.21.15.49:0/1529987108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd39310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.233+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 >> 172.21.15.49:0/1529987108 conn(0x5613acca3260 msgr2=0x5613acd294d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.234+0000 7f32fbfff700 0 client.25565 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.234+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 shutdown_connections 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.234+0000 7f332ef41b80 1 -- 172.21.15.49:0/1529987108 wait complete. 2023-10-08T09:15:40.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.234+0000 7f332ef41b80 20 client.25565 trim_cache size 0 max 16384 2023-10-08T09:15:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 Processor -- start 2023-10-08T09:15:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 -- start start 2023-10-08T09:15:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd33720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd40310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd33c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acb02920 con 0x5613acd2f9c0 2023-10-08T09:15:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acd39fc0 con 0x5613acd2f060 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.266+0000 7f332ef41b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acd30040 con 0x5613acd30800 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f331bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd40310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33212f5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd33c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f3320af4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd33720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f331bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd40310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50610/0 (socket says 172.21.15.49:50610) 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f331bfff700 1 -- 172.21.15.49:0/297499237 learned_addr learned my addr 172.21.15.49:0/297499237 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33212f5700 1 -- 172.21.15.49:0/297499237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 msgr2=0x5613acd33720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33212f5700 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd33720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33212f5700 1 -- 172.21.15.49:0/297499237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 msgr2=0x5613acd40310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33212f5700 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd40310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33212f5700 1 -- 172.21.15.49:0/297499237 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613acad3250 con 0x5613acd30800 2023-10-08T09:15:40.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f331bfff700 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd40310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33212f5700 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd33c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33140096e0 tx=0x7f3314022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c277080b1f894c24 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.267+0000 7f33137fe700 1 -- 172.21.15.49:0/297499237 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400ce30 con 0x5613acd30800 2023-10-08T09:15:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.268+0000 7f33137fe700 1 -- 172.21.15.49:0/297499237 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33140168e0 con 0x5613acd30800 2023-10-08T09:15:40.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.268+0000 7f33137fe700 1 -- 172.21.15.49:0/297499237 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314030770 con 0x5613acd30800 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 -- 172.21.15.49:0/297499237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 msgr2=0x5613acd33c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd33c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33140096e0 tx=0x7f3314022000 comp rx=0 tx=0).stop 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 -- 172.21.15.49:0/297499237 shutdown_connections 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd30800 0x5613acd33c60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd33720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 --2- 172.21.15.49:0/297499237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd2f9c0 0x5613acd40310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 -- 172.21.15.49:0/297499237 >> 172.21.15.49:0/297499237 conn(0x5613acca3260 msgr2=0x5613acd36d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 -- 172.21.15.49:0/297499237 shutdown_connections 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 -- 172.21.15.49:0/297499237 wait complete. 2023-10-08T09:15:40.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 -- start start 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.269+0000 7f332ef41b80 1 Processor -- start 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd3fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd2f9c0 0x5613acd3ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f332ef41b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd30800 0x5613acd3cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f332ef41b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acd3d680 con 0x5613acd30800 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f332ef41b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acd40500 con 0x5613acd2f060 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f332ef41b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613acd40690 con 0x5613acd2f9c0 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f332ef41b80 10 client.0 _ll_register_callbacks cb 0x7f330c009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd3fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd3fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40228/0 (socket says 172.21.15.49:40228) 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 -- 172.21.15.49:0/206053131 learned_addr learned my addr 172.21.15.49:0/206053131 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f331bfff700 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd2f9c0 0x5613acd3ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f33212f5700 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd30800 0x5613acd3cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 -- 172.21.15.49:0/206053131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd2f9c0 msgr2=0x5613acd3ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd2f9c0 0x5613acd3ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 -- 172.21.15.49:0/206053131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd30800 msgr2=0x5613acd3cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd30800 0x5613acd3cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 -- 172.21.15.49:0/206053131 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613acaacde0 con 0x5613acd2f060 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f3320af4700 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd3fe80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f330c009690 tx=0x7f330c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40aafd89312bad88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.270+0000 7f32fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:15:40.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.274+0000 7f32fbfff700 1 -- 172.21.15.49:0/206053131 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c01e7d0 con 0x5613acd2f060 2023-10-08T09:15:40.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.274+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33040d2070 con 0x5613acd2f060 2023-10-08T09:15:40.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.274+0000 7f32fbfff700 1 -- 172.21.15.49:0/206053131 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f330c01e930 con 0x5613acd2f060 2023-10-08T09:15:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.282+0000 7f32fbfff700 1 -- 172.21.15.49:0/206053131 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f330c01eaf0 con 0x5613acd2f060 2023-10-08T09:15:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.282+0000 7f332ef41b80 10 client.25571 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:40.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.282+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f331401a040 con 0x5613acd2f060 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.283+0000 7f32fbfff700 1 -- 172.21.15.49:0/206053131 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33040d2070 con 0x5613acd2f060 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.283+0000 7f32fbfff700 1 -- 172.21.15.49:0/206053131 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f330c0125a0 con 0x5613acd2f060 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.284+0000 7f332ef41b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.284+0000 7f332ef41b80 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.284+0000 7f332ef41b80 20 client.25571 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.284+0000 7f332ef41b80 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd9a350 0x5613acd9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.284+0000 7f332ef41b80 4 client.25571 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:15:40.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.284+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5613acc7ab30 con 0x5613acd9a350 2023-10-08T09:15:40.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.288+0000 7f331bfff700 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd9a350 0x5613acd9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:40.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.289+0000 7f331bfff700 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd9a350 0x5613acd9c770 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4583df1e94904b server_cookie=7bd8616006620d07 in_seq=0 out_seq=0 2023-10-08T09:15:40.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.289+0000 7f32fbfff700 10 client.25571 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:40.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f32fbfff700 1 -- 172.21.15.49:0/206053131 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5613acc7ab30 con 0x5613acd9a350 2023-10-08T09:15:40.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f32fbfff700 10 client.25571 handle_command_reply: tid=0 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 client.25571 shutdown 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 msgr2=0x5613acd3fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd3fe80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f330c009690 tx=0x7f330c009c60 comp rx=0 tx=0).stop 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 shutdown_connections 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5613acd9a350 0x5613acd9c770 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5613acd30800 0x5613acd3cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5613acd2f060 0x5613acd3fe80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f32fbfff700 0 client.25571 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:40.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 --2- 172.21.15.49:0/206053131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5613acd2f9c0 0x5613acd3ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.290+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 >> 172.21.15.49:0/206053131 conn(0x5613acca3260 msgr2=0x5613acd33720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:40.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.291+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 shutdown_connections 2023-10-08T09:15:40.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.292+0000 7f332ef41b80 1 -- 172.21.15.49:0/206053131 wait complete. 2023-10-08T09:15:40.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.295+0000 7f332ef41b80 20 client.25571 trim_cache size 0 max 16384 2023-10-08T09:15:40.297 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:15:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:15:40.299 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f33080a1500 msgr2=0x7f33080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f33080a1500 0x7f33080a39c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3314005e90 tx=0x7f3314009f40 comp rx=0 tx=0).stop 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 msgr2=0x7f331c082190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 0x7f331c082190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f330400ec10 tx=0x7f330400c2d0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 shutdown_connections 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f33080a1500 0x7f33080a39c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f331c077e70 0x7f331c07e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f331c072650 0x7f331c081c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 --2- 172.21.15.49:0/1986602377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c0754a0 0x7f331c082190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:40.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 >> 172.21.15.49:0/1986602377 conn(0x7f331c06d790 msgr2=0x7f331c076410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:40.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 shutdown_connections 2023-10-08T09:15:40.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:15:40.301+0000 7f3322d58700 1 -- 172.21.15.49:0/1986602377 wait complete. 2023-10-08T09:15:40.317 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:15:40.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub queued for path: / 2023-10-08T09:15:40.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:40.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:40 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub queued for path: / 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:40.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:40.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:40.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:40 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub queued for path: / 2023-10-08T09:15:40.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:40.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:40.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:40.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:40.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:40.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:40.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:40.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:40 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:41 smithi132 ceph-mon[103389]: pgmap v352: 129 pgs: 129 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 81 MiB/s wr, 1.08k op/s 2023-10-08T09:15:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:41 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:41 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:41 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:41 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:15:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:41 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:41 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:41 smithi049 ceph-mon[99485]: pgmap v352: 129 pgs: 129 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 81 MiB/s wr, 1.08k op/s 2023-10-08T09:15:41.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:41 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:41.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:41 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:41 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:41.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:41 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:15:41.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:41 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:41 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:41 smithi184 ceph-mon[103675]: pgmap v352: 129 pgs: 129 active+clean; 1.2 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 81 MiB/s wr, 1.08k op/s 2023-10-08T09:15:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:41 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:41 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:41 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:41 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:15:41.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:41 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:41.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:41 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:15:43.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:43 smithi132 ceph-mon[103389]: pgmap v353: 129 pgs: 129 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 73 MiB/s wr, 1.15k op/s 2023-10-08T09:15:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:43 smithi049 ceph-mon[99485]: pgmap v353: 129 pgs: 129 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 73 MiB/s wr, 1.15k op/s 2023-10-08T09:15:43.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:43 smithi184 ceph-mon[103675]: pgmap v353: 129 pgs: 129 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 73 MiB/s wr, 1.15k op/s 2023-10-08T09:15:44.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 20.0 s, 3224.2 tps, lat 31.232 ms stddev 36.921 2023-10-08T09:15:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.844+0000 7f8b86abc700 1 -- 172.21.15.49:0/1591963345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801081b0 msgr2=0x7f8b8010fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.844+0000 7f8b86abc700 1 --2- 172.21.15.49:0/1591963345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801081b0 0x7f8b8010fae0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800b0a0 tx=0x7f8b7800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.845+0000 7f8b86abc700 1 -- 172.21.15.49:0/1591963345 shutdown_connections 2023-10-08T09:15:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.845+0000 7f8b86abc700 1 --2- 172.21.15.49:0/1591963345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b80108b00 0x7f8b80108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.845+0000 7f8b86abc700 1 --2- 172.21.15.49:0/1591963345 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801081b0 0x7f8b8010fae0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.845+0000 7f8b86abc700 1 --2- 172.21.15.49:0/1591963345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b801077f0 0x7f8b80107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.845+0000 7f8b86abc700 1 -- 172.21.15.49:0/1591963345 >> 172.21.15.49:0/1591963345 conn(0x7f8b8006d770 msgr2=0x7f8b8006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.845+0000 7f8b86abc700 1 -- 172.21.15.49:0/1591963345 shutdown_connections 2023-10-08T09:15:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.845+0000 7f8b86abc700 1 -- 172.21.15.49:0/1591963345 wait complete. 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 Processor -- start 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 -- start start 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801077f0 0x7f8b80116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b801081b0 0x7f8b80117370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b80108b00 0x7f8b80118350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80118950 con 0x7f8b801077f0 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80118ac0 con 0x7f8b80108b00 2023-10-08T09:15:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.846+0000 7f8b86abc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80118c80 con 0x7f8b801081b0 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b801081b0 0x7f8b80117370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b801081b0 0x7f8b80117370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37666/0 (socket says 172.21.15.49:37666) 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 -- 172.21.15.49:0/3576321813 learned_addr learned my addr 172.21.15.49:0/3576321813 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b84858700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801077f0 0x7f8b80116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b85059700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b80108b00 0x7f8b80118350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 -- 172.21.15.49:0/3576321813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b80108b00 msgr2=0x7f8b80118350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b80108b00 0x7f8b80118350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 -- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801077f0 msgr2=0x7f8b80116e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801077f0 0x7f8b80116e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b7ffff700 1 -- 172.21.15.49:0/3576321813 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74009520 con 0x7f8b801081b0 2023-10-08T09:15:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.847+0000 7f8b85059700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b80108b00 0x7f8b80118350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:15:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.848+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b801081b0 0x7f8b80117370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400eb30 tx=0x7f8b7400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.848+0000 7f8b7dffb700 1 -- 172.21.15.49:0/3576321813 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74009880 con 0x7f8b801081b0 2023-10-08T09:15:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.848+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b78009d00 con 0x7f8b801081b0 2023-10-08T09:15:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.848+0000 7f8b7dffb700 1 -- 172.21.15.49:0/3576321813 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b740044e0 con 0x7f8b801081b0 2023-10-08T09:15:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.848+0000 7f8b7dffb700 1 -- 172.21.15.49:0/3576321813 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74003e90 con 0x7f8b801081b0 2023-10-08T09:15:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.848+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b80078a50 con 0x7f8b801081b0 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.851+0000 7f8b7dffb700 1 -- 172.21.15.49:0/3576321813 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8b74018470 con 0x7f8b801081b0 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.852+0000 7f8b7dffb700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8b680a1530 0x7f8b680a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.853+0000 7f8b7dffb700 1 -- 172.21.15.49:0/3576321813 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8b74014070 con 0x7f8b801081b0 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.853+0000 7f8b84858700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8b680a1530 0x7f8b680a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.853+0000 7f8b84858700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8b680a1530 0x7f8b680a39f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b80117ff0 tx=0x7f8b70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.853+0000 7f8b92ca5b80 1 Processor -- start 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.853+0000 7f8b92ca5b80 1 -- start start 2023-10-08T09:15:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a292f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a292fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a29302b0 0x5556a2930730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2897400 con 0x5556a29302b0 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2888bc0 con 0x5556a292ee40 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a28864e0 con 0x5556a292f7a0 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b84858700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a292f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b85059700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a29302b0 0x5556a2930730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a292fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a292fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37672/0 (socket says 172.21.15.49:37672) 2023-10-08T09:15:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b85059700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a29302b0 0x5556a2930730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50640/0 (socket says 172.21.15.49:50640) 2023-10-08T09:15:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 -- 172.21.15.49:0/327161755 learned_addr learned my addr 172.21.15.49:0/327161755 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 -- 172.21.15.49:0/327161755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 msgr2=0x5556a292f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/327161755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a292f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 -- 172.21.15.49:0/327161755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a29302b0 msgr2=0x5556a2930730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/327161755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a29302b0 0x5556a2930730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.854+0000 7f8b7ffff700 1 -- 172.21.15.49:0/327161755 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a26acd80 con 0x5556a292f7a0 2023-10-08T09:15:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.855+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/327161755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a292fcd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b74097880 tx=0x7f8b7409ba80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4caa991175a7d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.855+0000 7f8b66ffd700 1 -- 172.21.15.49:0/327161755 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740cd070 con 0x5556a292f7a0 2023-10-08T09:15:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.855+0000 7f8b66ffd700 1 -- 172.21.15.49:0/327161755 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7409f4d0 con 0x5556a292f7a0 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.855+0000 7f8b66ffd700 1 -- 172.21.15.49:0/327161755 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7409f290 con 0x5556a292f7a0 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.856+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/327161755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 msgr2=0x5556a292fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.856+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/327161755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a292fcd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b74097880 tx=0x7f8b7409ba80 comp rx=0 tx=0).stop 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.856+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/327161755 shutdown_connections 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.856+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/327161755 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a29302b0 0x5556a2930730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.856+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/327161755 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a292f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.856+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/327161755 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a292fcd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.856+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/327161755 >> 172.21.15.49:0/327161755 conn(0x5556a28a3200 msgr2=0x5556a29298a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.857+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/327161755 shutdown_connections 2023-10-08T09:15:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.857+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/327161755 wait complete. 2023-10-08T09:15:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 -- start start 2023-10-08T09:15:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 Processor -- start 2023-10-08T09:15:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a2938930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a2938e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a2881dd0 con 0x5556a292f7a0 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a281e6b0 con 0x5556a292ee40 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a281e350 con 0x5556a29302b0 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.858+0000 7f8b92ca5b80 10 client.0 _ll_register_callbacks cb 0x5556a267b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a2938e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37680/0 (socket says 172.21.15.49:37680) 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 -- 172.21.15.49:0/3737940763 learned_addr learned my addr 172.21.15.49:0/3737940763 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b84858700 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a2938930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 -- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 msgr2=0x5556a2938930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a2938930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 -- 172.21.15.49:0/3737940763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 msgr2=0x5556a2938e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a2938e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 -- 172.21.15.49:0/3737940763 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b80078a50 con 0x5556a29302b0 2023-10-08T09:15:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b85059700 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293fdc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009cd0 tx=0x7f8b78007850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dd694d54e319ca8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.859+0000 7f8b5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:15:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.861+0000 7f8b5ffff700 1 -- 172.21.15.49:0/3737940763 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800e040 con 0x5556a29302b0 2023-10-08T09:15:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.861+0000 7f8b5ffff700 1 -- 172.21.15.49:0/3737940763 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8b78004a50 con 0x5556a29302b0 2023-10-08T09:15:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.861+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556a2849e50 con 0x5556a29302b0 2023-10-08T09:15:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.861+0000 7f8b5ffff700 1 -- 172.21.15.49:0/3737940763 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b78004c10 con 0x5556a29302b0 2023-10-08T09:15:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.861+0000 7f8b92ca5b80 10 client.25606 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.861+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5556a2695c00 con 0x5556a29302b0 2023-10-08T09:15:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.862+0000 7f8b5ffff700 1 -- 172.21.15.49:0/3737940763 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556a2849e50 con 0x5556a29302b0 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.863+0000 7f8b5ffff700 1 -- 172.21.15.49:0/3737940763 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8b78019070 con 0x5556a29302b0 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.863+0000 7f8b92ca5b80 10 client.25606 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.863+0000 7f8b92ca5b80 10 client.25606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.863+0000 7f8b92ca5b80 20 client.25606 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.863+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2944780 0x5556a2946ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.863+0000 7f8b92ca5b80 4 client.25606 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.863+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556a287adb0 con 0x5556a2944780 2023-10-08T09:15:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.864+0000 7f8b84858700 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2944780 0x5556a2946ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.864+0000 7f8b5ffff700 10 client.25606 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.864+0000 7f8b84858700 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2944780 0x5556a2946ba0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df47ddca5169464 server_cookie=a1546b9f557c2ba2 in_seq=0 out_seq=0 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b5ffff700 1 -- 172.21.15.49:0/3737940763 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5556a287adb0 con 0x5556a2944780 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b5ffff700 10 client.25606 handle_command_reply: tid=0 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 client.25606 shutdown 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 msgr2=0x5556a293fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293fdc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009cd0 tx=0x7f8b78007850 comp rx=0 tx=0).stop 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 shutdown_connections 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2944780 0x5556a2946ba0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293fdc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a2938930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/3737940763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a2938e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b5ffff700 0 client.25606 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.865+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 >> 172.21.15.49:0/3737940763 conn(0x5556a28a3200 msgr2=0x5556a2929010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.866+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 shutdown_connections 2023-10-08T09:15:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.866+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/3737940763 wait complete. 2023-10-08T09:15:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.866+0000 7f8b92ca5b80 20 client.25606 trim_cache size 0 max 16384 2023-10-08T09:15:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:45 smithi049 ceph-mon[99485]: pgmap v354: 129 pgs: 129 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 71 MiB/s wr, 1.17k op/s 2023-10-08T09:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.893+0000 7f8b92ca5b80 1 Processor -- start 2023-10-08T09:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.893+0000 7f8b92ca5b80 1 -- start start 2023-10-08T09:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292ee40 0x5556a2930690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a293fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a29302b0 0x5556a29368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a27028c0 con 0x5556a292ee40 2023-10-08T09:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a287adb0 con 0x5556a29302b0 2023-10-08T09:15:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a29242b0 con 0x5556a292f7a0 2023-10-08T09:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292ee40 0x5556a2930690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292ee40 0x5556a2930690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50650/0 (socket says 172.21.15.49:50650) 2023-10-08T09:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 -- 172.21.15.49:0/836595566 learned_addr learned my addr 172.21.15.49:0/836595566 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 -- 172.21.15.49:0/836595566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 msgr2=0x5556a293fce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 --2- 172.21.15.49:0/836595566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a293fce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 -- 172.21.15.49:0/836595566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a29302b0 msgr2=0x5556a29368e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 --2- 172.21.15.49:0/836595566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a29302b0 0x5556a29368e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.894+0000 7f8b84858700 1 -- 172.21.15.49:0/836595566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556a26d31f0 con 0x5556a292ee40 2023-10-08T09:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.895+0000 7f8b84858700 1 --2- 172.21.15.49:0/836595566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292ee40 0x5556a2930690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5556a2947470 tx=0x7f8b70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac00b4c214de44e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.895+0000 7f8b677fe700 1 -- 172.21.15.49:0/836595566 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70022030 con 0x5556a292ee40 2023-10-08T09:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.895+0000 7f8b677fe700 1 -- 172.21.15.49:0/836595566 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b70023040 con 0x5556a292ee40 2023-10-08T09:15:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.895+0000 7f8b677fe700 1 -- 172.21.15.49:0/836595566 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70003b10 con 0x5556a292ee40 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/836595566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292ee40 msgr2=0x5556a2930690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/836595566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292ee40 0x5556a2930690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5556a2947470 tx=0x7f8b70014000 comp rx=0 tx=0).stop 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/836595566 shutdown_connections 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/836595566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a29302b0 0x5556a29368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/836595566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292ee40 0x5556a2930690 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/836595566 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a292f7a0 0x5556a293fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/836595566 >> 172.21.15.49:0/836595566 conn(0x5556a28a3200 msgr2=0x5556a2929870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/836595566 shutdown_connections 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.896+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/836595566 wait complete. 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 -- start start 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 Processor -- start 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a293dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a293e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a293f710 con 0x5556a292f7a0 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a293f880 con 0x5556a292ee40 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556a293fa40 con 0x5556a29302b0 2023-10-08T09:15:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.897+0000 7f8b92ca5b80 10 client.0 _ll_register_callbacks cb 0x7f8b78009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a293e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b84858700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a293dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b85059700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a293e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50656/0 (socket says 172.21.15.49:50656) 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 -- 172.21.15.49:0/1435915966 learned_addr learned my addr 172.21.15.49:0/1435915966 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 -- 172.21.15.49:0/1435915966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 msgr2=0x5556a293f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 -- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 msgr2=0x5556a293dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:45.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a293dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 -- 172.21.15.49:0/1435915966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b70017070 con 0x5556a292f7a0 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b84858700 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a293dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b7ffff700 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a293e1c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8b7409b540 tx=0x7f8b7409b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ada7fd492951b4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.898+0000 7f8b5ffff700 1 -- 172.21.15.49:0/1435915966 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740cd040 con 0x5556a292f7a0 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.899+0000 7f8b5ffff700 1 -- 172.21.15.49:0/1435915966 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8b7409f170 con 0x5556a292f7a0 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.899+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b7409f4d0 con 0x5556a292f7a0 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.899+0000 7f8b5ffff700 1 -- 172.21.15.49:0/1435915966 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b7400cc90 con 0x5556a292f7a0 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.899+0000 7f8b92ca5b80 10 client.15951 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.899+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5556a26ab250 con 0x5556a292f7a0 2023-10-08T09:15:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b5ffff700 1 -- 172.21.15.49:0/1435915966 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7409f4d0 con 0x5556a292f7a0 2023-10-08T09:15:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b5ffff700 1 -- 172.21.15.49:0/1435915966 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8b7409a870 con 0x5556a292f7a0 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b92ca5b80 10 client.15951 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b92ca5b80 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b92ca5b80 20 client.15951 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2999350 0x5556a299b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b92ca5b80 4 client.15951 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5556a287aad0 con 0x5556a2999350 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.900+0000 7f8b84858700 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2999350 0x5556a299b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.901+0000 7f8b5ffff700 10 client.15951 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.901+0000 7f8b84858700 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2999350 0x5556a299b770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ee1dd38e1834dc server_cookie=44c9272ed1332a8 in_seq=0 out_seq=0 2023-10-08T09:15:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b5ffff700 1 -- 172.21.15.49:0/1435915966 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5556a287aad0 con 0x5556a2999350 2023-10-08T09:15:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b5ffff700 10 client.15951 handle_command_reply: tid=0 2023-10-08T09:15:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b92ca5b80 1 client.15951 shutdown 2023-10-08T09:15:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 msgr2=0x5556a293e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a293e1c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8b7409b540 tx=0x7f8b7409b150 comp rx=0 tx=0).stop 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 shutdown_connections 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5556a2999350 0x5556a299b770 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5556a29302b0 0x5556a293f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b5ffff700 0 client.15951 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.920+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5556a292ee40 0x5556a293dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b92ca5b80 1 --2- 172.21.15.49:0/1435915966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5556a292f7a0 0x5556a293e1c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 >> 172.21.15.49:0/1435915966 conn(0x5556a28a3200 msgr2=0x5556a29386e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 shutdown_connections 2023-10-08T09:15:45.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b92ca5b80 1 -- 172.21.15.49:0/1435915966 wait complete. 2023-10-08T09:15:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b92ca5b80 20 client.15951 trim_cache size 0 max 16384 2023-10-08T09:15:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:15:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:15:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:15:45.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:15:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8b680a1530 msgr2=0x7f8b680a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b86abc700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8b680a1530 0x7f8b680a39f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8b80117ff0 tx=0x7f8b70009260 comp rx=0 tx=0).stop 2023-10-08T09:15:45.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b801081b0 msgr2=0x7f8b80117370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.921+0000 7f8b86abc700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b801081b0 0x7f8b80117370 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400eb30 tx=0x7f8b7400ee40 comp rx=0 tx=0).stop 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 shutdown_connections 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8b680a1530 0x7f8b680a39f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b80108b00 0x7f8b80118350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b801077f0 0x7f8b80116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 --2- 172.21.15.49:0/3576321813 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b801081b0 0x7f8b80117370 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 >> 172.21.15.49:0/3576321813 conn(0x7f8b8006d770 msgr2=0x7f8b8010bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:45.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 shutdown_connections 2023-10-08T09:15:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:45.922+0000 7f8b86abc700 1 -- 172.21.15.49:0/3576321813 wait complete. 2023-10-08T09:15:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:45 smithi184 ceph-mon[103675]: pgmap v354: 129 pgs: 129 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 71 MiB/s wr, 1.17k op/s 2023-10-08T09:15:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:45 smithi132 ceph-mon[103389]: pgmap v354: 129 pgs: 129 active+clean; 1.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 71 MiB/s wr, 1.17k op/s 2023-10-08T09:15:47.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:47 smithi049 ceph-mon[99485]: pgmap v355: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 52 MiB/s wr, 1.01k op/s 2023-10-08T09:15:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:47 smithi184 ceph-mon[103675]: pgmap v355: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 52 MiB/s wr, 1.01k op/s 2023-10-08T09:15:48.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:47 smithi132 ceph-mon[103389]: pgmap v355: 129 pgs: 129 active+clean; 1.2 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 52 MiB/s wr, 1.01k op/s 2023-10-08T09:15:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 25.0 s, 2904.2 tps, lat 34.006 ms stddev 42.430 2023-10-08T09:15:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:49 smithi049 ceph-mon[99485]: pgmap v356: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 39 MiB/s wr, 950 op/s 2023-10-08T09:15:49.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:49 smithi184 ceph-mon[103675]: pgmap v356: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 39 MiB/s wr, 950 op/s 2023-10-08T09:15:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:49 smithi132 ceph-mon[103389]: pgmap v356: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 39 MiB/s wr, 950 op/s 2023-10-08T09:15:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:51 smithi049 ceph-mon[99485]: pgmap v357: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 27 MiB/s wr, 822 op/s 2023-10-08T09:15:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:51 smithi184 ceph-mon[103675]: pgmap v357: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 27 MiB/s wr, 822 op/s 2023-10-08T09:15:52.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:51 smithi132 ceph-mon[103389]: pgmap v357: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 27 MiB/s wr, 822 op/s 2023-10-08T09:15:53.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:53 smithi049 ceph-mon[99485]: pgmap v358: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 28 MiB/s wr, 925 op/s 2023-10-08T09:15:53.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:53 smithi184 ceph-mon[103675]: pgmap v358: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 28 MiB/s wr, 925 op/s 2023-10-08T09:15:54.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:53 smithi132 ceph-mon[103389]: pgmap v358: 129 pgs: 129 active+clean; 1.3 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 28 MiB/s wr, 925 op/s 2023-10-08T09:15:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 30.0 s, 2017.4 tps, lat 50.083 ms stddev 59.381 2023-10-08T09:15:55.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:55 smithi049 ceph-mon[99485]: pgmap v359: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 1.05k op/s 2023-10-08T09:15:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:55 smithi184 ceph-mon[103675]: pgmap v359: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 1.05k op/s 2023-10-08T09:15:56.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:55 smithi132 ceph-mon[103389]: pgmap v359: 129 pgs: 129 active+clean; 1.4 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 1.05k op/s 2023-10-08T09:15:57.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:57 smithi184 ceph-mon[103675]: pgmap v360: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 37 MiB/s wr, 1.10k op/s 2023-10-08T09:15:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:57 smithi132 ceph-mon[103389]: pgmap v360: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 37 MiB/s wr, 1.10k op/s 2023-10-08T09:15:58.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:57 smithi049 ceph-mon[99485]: pgmap v360: 129 pgs: 129 active+clean; 1.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 37 MiB/s wr, 1.10k op/s 2023-10-08T09:15:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 -- 172.21.15.49:0/2642721402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe4010b150 msgr2=0x7fbe4010b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 --2- 172.21.15.49:0/2642721402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe4010b150 0x7fbe4010b570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009fc0 tx=0x7fbe3800b330 comp rx=0 tx=0).stop 2023-10-08T09:15:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 -- 172.21.15.49:0/2642721402 shutdown_connections 2023-10-08T09:15:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 --2- 172.21.15.49:0/2642721402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe4010ccd0 0x7fbe40113580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 --2- 172.21.15.49:0/2642721402 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe4010b150 0x7fbe4010b570 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 --2- 172.21.15.49:0/2642721402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010c330 0x7fbe4010c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 -- 172.21.15.49:0/2642721402 >> 172.21.15.49:0/2642721402 conn(0x7fbe4006b340 msgr2=0x7fbe4006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 -- 172.21.15.49:0/2642721402 shutdown_connections 2023-10-08T09:15:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.086+0000 7fbe460a2700 1 -- 172.21.15.49:0/2642721402 wait complete. 2023-10-08T09:15:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.087+0000 7fbe460a2700 1 Processor -- start 2023-10-08T09:15:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.087+0000 7fbe460a2700 1 -- start start 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe460a2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe4010b150 0x7fbe401a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe460a2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe4010c330 0x7fbe401a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe460a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010ccd0 0x7fbe401acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe460a2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401ad1d0 con 0x7fbe4010c330 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe460a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401ad310 con 0x7fbe4010ccd0 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe460a2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401ad4a0 con 0x7fbe4010b150 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010ccd0 0x7fbe401acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010ccd0 0x7fbe401acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40284/0 (socket says 172.21.15.49:40284) 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 -- 172.21.15.49:0/1924763821 learned_addr learned my addr 172.21.15.49:0/1924763821 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe4010b150 0x7fbe401a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 -- 172.21.15.49:0/1924763821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe4010b150 msgr2=0x7fbe401a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe4010b150 0x7fbe401a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 -- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe4010c330 msgr2=0x7fbe401a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe4010c330 0x7fbe401a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.088+0000 7fbe3ffff700 1 -- 172.21.15.49:0/1924763821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe38009c40 con 0x7fbe4010ccd0 2023-10-08T09:15:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.089+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010ccd0 0x7fbe401acc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400e970 tx=0x7fbe3400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.090+0000 7fbe3cff9700 1 -- 172.21.15.49:0/1924763821 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3400cbd0 con 0x7fbe4010ccd0 2023-10-08T09:15:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.090+0000 7fbe3cff9700 1 -- 172.21.15.49:0/1924763821 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3400cd30 con 0x7fbe4010ccd0 2023-10-08T09:15:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.090+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe401ad780 con 0x7fbe4010ccd0 2023-10-08T09:15:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.090+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe401adc90 con 0x7fbe4010ccd0 2023-10-08T09:15:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.090+0000 7fbe3cff9700 1 -- 172.21.15.49:0/1924763821 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34005600 con 0x7fbe4010ccd0 2023-10-08T09:15:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.093+0000 7fbe3cff9700 1 -- 172.21.15.49:0/1924763821 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbe34010430 con 0x7fbe4010ccd0 2023-10-08T09:15:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.094+0000 7fbe3cff9700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe280a1580 0x7fbe280a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.094+0000 7fbe3cff9700 1 -- 172.21.15.49:0/1924763821 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe34014070 con 0x7fbe4010ccd0 2023-10-08T09:15:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.095+0000 7fbe5228bb80 1 Processor -- start 2023-10-08T09:15:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.098+0000 7fbe5228bb80 1 -- start start 2023-10-08T09:15:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.098+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d0433e40 0x5576d0434260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.098+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0434cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.098+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0435730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.098+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d039c400 con 0x5576d04347a0 2023-10-08T09:15:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.098+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d038dbc0 con 0x5576d0433e40 2023-10-08T09:15:59.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.098+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d038b4e0 con 0x5576d04352b0 2023-10-08T09:15:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.102+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0435730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.102+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0435730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37708/0 (socket says 172.21.15.49:37708) 2023-10-08T09:15:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.102+0000 7fbe3ffff700 1 -- 172.21.15.49:0/3555857862 learned_addr learned my addr 172.21.15.49:0/3555857862 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.102+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe280a1580 0x7fbe280a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.103+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d0433e40 0x5576d0434260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.103+0000 7fbe3effd700 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0434cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.104+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/3555857862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 msgr2=0x5576d0435730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.104+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0435730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.104+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/3555857862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 msgr2=0x5576d0434cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.104+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0434cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.104+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/3555857862 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576d01b1d80 con 0x5576d0433e40 2023-10-08T09:15:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.104+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d0433e40 0x5576d0434260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007a30 tx=0x7fbe38008b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c0980f4d7ea994 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.105+0000 7fbe25ffb700 1 -- 172.21.15.49:0/3555857862 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800e070 con 0x5576d0433e40 2023-10-08T09:15:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.105+0000 7fbe25ffb700 1 -- 172.21.15.49:0/3555857862 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe38004d30 con 0x5576d0433e40 2023-10-08T09:15:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.105+0000 7fbe25ffb700 1 -- 172.21.15.49:0/3555857862 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38024070 con 0x5576d0433e40 2023-10-08T09:15:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.106+0000 7fbe5228bb80 1 -- 172.21.15.49:0/3555857862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d0433e40 msgr2=0x5576d0434260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.106+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d0433e40 0x5576d0434260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007a30 tx=0x7fbe38008b40 comp rx=0 tx=0).stop 2023-10-08T09:15:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.107+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe280a1580 0x7fbe280a3a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007900 tx=0x7fbe38021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.110+0000 7fbe5228bb80 1 -- 172.21.15.49:0/3555857862 shutdown_connections 2023-10-08T09:15:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.110+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0435730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.110+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d0433e40 0x5576d0434260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.110+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/3555857862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0434cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.110+0000 7fbe5228bb80 1 -- 172.21.15.49:0/3555857862 >> 172.21.15.49:0/3555857862 conn(0x5576d03a8200 msgr2=0x5576d042e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.112+0000 7fbe5228bb80 1 -- 172.21.15.49:0/3555857862 shutdown_connections 2023-10-08T09:15:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.115+0000 7fbe5228bb80 1 -- 172.21.15.49:0/3555857862 wait complete. 2023-10-08T09:15:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.115+0000 7fbe5228bb80 1 -- start start 2023-10-08T09:15:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.116+0000 7fbe5228bb80 1 Processor -- start 2023-10-08T09:15:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.117+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d0433e40 0x5576d044e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.117+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04347a0 0x5576d044ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.117+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0455e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.117+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d0386dd0 con 0x5576d0433e40 2023-10-08T09:15:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.117+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d03236b0 con 0x5576d04347a0 2023-10-08T09:15:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.117+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d0323350 con 0x5576d04352b0 2023-10-08T09:15:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.117+0000 7fbe5228bb80 10 client.0 _ll_register_callbacks cb 0x5576d01807a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d0433e40 0x5576d044e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d0433e40 0x5576d044e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50684/0 (socket says 172.21.15.49:50684) 2023-10-08T09:15:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/4096631823 learned_addr learned my addr 172.21.15.49:0/4096631823 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0455e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/4096631823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 msgr2=0x5576d0455e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0455e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/4096631823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04347a0 msgr2=0x5576d044ee30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:15:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04347a0 0x5576d044ee30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.118+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/4096631823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe40113ed0 con 0x5576d0433e40 2023-10-08T09:15:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.119+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d0433e40 0x5576d044e8f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbe38008300 tx=0x7fbe38009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d95e7b1e2e5c183 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.120+0000 7fbe1effd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.120+0000 7fbe1effd700 1 -- 172.21.15.49:0/4096631823 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800e040 con 0x5576d0433e40 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.120+0000 7fbe1effd700 1 -- 172.21.15.49:0/4096631823 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe3800bca0 con 0x5576d0433e40 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.120+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576d034ee50 con 0x5576d0433e40 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.120+0000 7fbe1effd700 1 -- 172.21.15.49:0/4096631823 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe38024040 con 0x5576d0433e40 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.120+0000 7fbe5228bb80 10 client.15960 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.120+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5576d019ac00 con 0x5576d0433e40 2023-10-08T09:15:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.121+0000 7fbe1effd700 1 -- 172.21.15.49:0/4096631823 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576d034ee50 con 0x5576d0433e40 2023-10-08T09:15:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.122+0000 7fbe1effd700 1 -- 172.21.15.49:0/4096631823 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbe3802f070 con 0x5576d0433e40 2023-10-08T09:15:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.122+0000 7fbe5228bb80 10 client.15960 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.122+0000 7fbe5228bb80 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.122+0000 7fbe5228bb80 20 client.15960 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.122+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d045a520 0x5576d045c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.122+0000 7fbe5228bb80 4 client.15960 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:15:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.122+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576d037fdb0 con 0x5576d045a520 2023-10-08T09:15:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.124+0000 7fbe3effd700 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d045a520 0x5576d045c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.127+0000 7fbe3effd700 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d045a520 0x5576d045c940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a0061077600268 server_cookie=2ac20097a2626173 in_seq=0 out_seq=0 2023-10-08T09:15:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.127+0000 7fbe1effd700 10 client.15960 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.128+0000 7fbe1effd700 1 -- 172.21.15.49:0/4096631823 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5576d037fdb0 con 0x5576d045a520 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.128+0000 7fbe1effd700 10 client.15960 handle_command_reply: tid=0 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.128+0000 7fbe5228bb80 1 client.15960 shutdown 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.128+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d0433e40 msgr2=0x5576d044e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.128+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d0433e40 0x5576d044e8f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbe38008300 tx=0x7fbe38009720 comp rx=0 tx=0).stop 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.129+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 shutdown_connections 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.129+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d045a520 0x5576d045c940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.129+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04352b0 0x5576d0455e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.129+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d0433e40 0x5576d044e8f0 secure :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbe38008300 tx=0x7fbe38009720 comp rx=0 tx=0).stop 2023-10-08T09:15:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.129+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/4096631823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04347a0 0x5576d044ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.129+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 >> 172.21.15.49:0/4096631823 conn(0x5576d03a8200 msgr2=0x5576d042e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.131+0000 7fbe1effd700 0 client.15960 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.131+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 shutdown_connections 2023-10-08T09:15:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.131+0000 7fbe5228bb80 1 -- 172.21.15.49:0/4096631823 wait complete. 2023-10-08T09:15:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.132+0000 7fbe5228bb80 20 client.15960 trim_cache size 0 max 16384 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.161+0000 7fbe5228bb80 1 Processor -- start 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.162+0000 7fbe5228bb80 1 -- start start 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.162+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0435690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.162+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d0455db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.162+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d042f940 0x5576d044de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.162+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d02078c0 con 0x5576d04347a0 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.162+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d0435130 con 0x5576d04352b0 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.162+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d044d360 con 0x5576d042f940 2023-10-08T09:15:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0435690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0435690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50688/0 (socket says 172.21.15.49:50688) 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3f7fe700 1 -- 172.21.15.49:0/2669545676 learned_addr learned my addr 172.21.15.49:0/2669545676 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d042f940 0x5576d044de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3effd700 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d0455db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3ffff700 1 -- 172.21.15.49:0/2669545676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 msgr2=0x5576d0455db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d0455db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3ffff700 1 -- 172.21.15.49:0/2669545676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 msgr2=0x5576d0435690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0435690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.163+0000 7fbe3ffff700 1 -- 172.21.15.49:0/2669545676 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576d01d81f0 con 0x5576d042f940 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.164+0000 7fbe3ffff700 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d042f940 0x5576d044de30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400ef90 tx=0x7fbe34097670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1522c574e745adf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.165+0000 7fbe267fc700 1 -- 172.21.15.49:0/2669545676 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3409b330 con 0x5576d042f940 2023-10-08T09:15:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.165+0000 7fbe267fc700 1 -- 172.21.15.49:0/2669545676 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3409f480 con 0x5576d042f940 2023-10-08T09:15:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.165+0000 7fbe267fc700 1 -- 172.21.15.49:0/2669545676 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3401f560 con 0x5576d042f940 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2669545676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d042f940 msgr2=0x5576d044de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d042f940 0x5576d044de30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400ef90 tx=0x7fbe34097670 comp rx=0 tx=0).stop 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2669545676 shutdown_connections 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d042f940 0x5576d044de30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d04347a0 0x5576d0435690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2669545676 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d0455db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2669545676 >> 172.21.15.49:0/2669545676 conn(0x5576d03a8200 msgr2=0x5576d04612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2669545676 shutdown_connections 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.166+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2669545676 wait complete. 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.167+0000 7fbe5228bb80 1 -- start start 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.167+0000 7fbe5228bb80 1 Processor -- start 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d042f940 0x5576d0455b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04347a0 0x5576d04541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe5228bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d04546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d0454cb0 con 0x5576d042f940 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d0454e20 con 0x5576d04352b0 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe5228bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d0454fe0 con 0x5576d04347a0 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe5228bb80 10 client.0 _ll_register_callbacks cb 0x7fbe34005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:15:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04347a0 0x5576d04541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d042f940 0x5576d0455b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d04546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04347a0 0x5576d04541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37728/0 (socket says 172.21.15.49:37728) 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 -- 172.21.15.49:0/2399012266 learned_addr learned my addr 172.21.15.49:0/2399012266 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 -- 172.21.15.49:0/2399012266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 msgr2=0x5576d04546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d04546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 -- 172.21.15.49:0/2399012266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d042f940 msgr2=0x5576d0455b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d042f940 0x5576d0455b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 -- 172.21.15.49:0/2399012266 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe401adc90 con 0x5576d04347a0 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d042f940 0x5576d0455b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe3effd700 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04347a0 0x5576d04541a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5576d045d250 tx=0x7fbe3001a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=257ba6191bbd77f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.168+0000 7fbe1effd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:15:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe1effd700 1 -- 172.21.15.49:0/2399012266 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3001a880 con 0x5576d04347a0 2023-10-08T09:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe1effd700 1 -- 172.21.15.49:0/2399012266 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbe30017040 con 0x5576d04347a0 2023-10-08T09:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe38004d30 con 0x5576d04347a0 2023-10-08T09:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe1effd700 1 -- 172.21.15.49:0/2399012266 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe3001a9e0 con 0x5576d04347a0 2023-10-08T09:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe5228bb80 10 client.25624 fetch_fsmap learned FSMap version 33 2023-10-08T09:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5576d01b0250 con 0x5576d04347a0 2023-10-08T09:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe1effd700 1 -- 172.21.15.49:0/2399012266 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe38004d30 con 0x5576d04347a0 2023-10-08T09:15:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.169+0000 7fbe1effd700 1 -- 172.21.15.49:0/2399012266 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbe30012450 con 0x5576d04347a0 2023-10-08T09:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.170+0000 7fbe5228bb80 10 client.25624 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.170+0000 7fbe5228bb80 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.170+0000 7fbe5228bb80 20 client.25624 populate_metadata read hostname 'smithi049' 2023-10-08T09:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.171+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d04431f0 0x5576d0445610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.171+0000 7fbe5228bb80 4 client.25624 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.171+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576d037fad0 con 0x5576d04431f0 2023-10-08T09:15:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.171+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d04431f0 0x5576d0445610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:15:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.171+0000 7fbe3f7fe700 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d04431f0 0x5576d0445610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a5d3ef296337f3 server_cookie=142b52d07fe6f81c in_seq=0 out_seq=0 2023-10-08T09:15:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.171+0000 7fbe1effd700 10 client.25624 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.199+0000 7fbe1effd700 1 -- 172.21.15.49:0/2399012266 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576d037fad0 con 0x5576d04431f0 2023-10-08T09:15:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.199+0000 7fbe1effd700 10 client.25624 handle_command_reply: tid=0 2023-10-08T09:15:59.201 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.199+0000 7fbe5228bb80 1 client.25624 shutdown 2023-10-08T09:15:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04347a0 msgr2=0x5576d04541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04347a0 0x5576d04541a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5576d045d250 tx=0x7fbe3001a6c0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 shutdown_connections 2023-10-08T09:15:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576d04431f0 0x5576d0445610 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576d04352b0 0x5576d04546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576d042f940 0x5576d0455b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe1effd700 0 client.25624 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:15:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 --2- 172.21.15.49:0/2399012266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576d04347a0 0x5576d04541a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.201+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 >> 172.21.15.49:0/2399012266 conn(0x5576d03a8200 msgr2=0x5576d044de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.204+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 shutdown_connections 2023-10-08T09:15:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.206+0000 7fbe5228bb80 1 -- 172.21.15.49:0/2399012266 wait complete. 2023-10-08T09:15:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.206+0000 7fbe5228bb80 20 client.25624 trim_cache size 0 max 16384 2023-10-08T09:15:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:15:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:15:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:15:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:15:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe280a1580 msgr2=0x7fbe280a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe280a1580 0x7fbe280a3a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007900 tx=0x7fbe38021000 comp rx=0 tx=0).stop 2023-10-08T09:15:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010ccd0 msgr2=0x7fbe401acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:15:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010ccd0 0x7fbe401acc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400e970 tx=0x7fbe3400ed30 comp rx=0 tx=0).stop 2023-10-08T09:15:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 shutdown_connections 2023-10-08T09:15:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbe280a1580 0x7fbe280a3a40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbe4010b150 0x7fbe401a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe4010ccd0 0x7fbe401acc00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 --2- 172.21.15.49:0/1924763821 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe4010c330 0x7fbe401a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:15:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 >> 172.21.15.49:0/1924763821 conn(0x7fbe4006b340 msgr2=0x7fbe4006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:15:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.207+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 shutdown_connections 2023-10-08T09:15:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:15:59.208+0000 7fbe460a2700 1 -- 172.21.15.49:0/1924763821 wait complete. 2023-10-08T09:15:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 35.0 s, 1862.2 tps, lat 53.056 ms stddev 64.511 2023-10-08T09:15:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:15:59 smithi184 ceph-mon[103675]: pgmap v361: 129 pgs: 129 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2023-10-08T09:16:00.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:15:59 smithi132 ceph-mon[103389]: pgmap v361: 129 pgs: 129 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2023-10-08T09:16:00.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:15:59 smithi049 ceph-mon[99485]: pgmap v361: 129 pgs: 129 active+clean; 1.4 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 MiB/s wr, 1.16k op/s 2023-10-08T09:16:01.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:01 smithi184 ceph-mon[103675]: pgmap v362: 129 pgs: 129 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.12k op/s 2023-10-08T09:16:02.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:01 smithi132 ceph-mon[103389]: pgmap v362: 129 pgs: 129 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.12k op/s 2023-10-08T09:16:02.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:01 smithi049 ceph-mon[99485]: pgmap v362: 129 pgs: 129 active+clean; 1.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 MiB/s wr, 1.12k op/s 2023-10-08T09:16:03.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:03 smithi184 ceph-mon[103675]: pgmap v363: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.29k op/s 2023-10-08T09:16:04.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:03 smithi132 ceph-mon[103389]: pgmap v363: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.29k op/s 2023-10-08T09:16:04.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:03 smithi049 ceph-mon[99485]: pgmap v363: 129 pgs: 129 active+clean; 1.5 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.29k op/s 2023-10-08T09:16:04.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 40.0 s, 1810.2 tps, lat 56.039 ms stddev 124.310 2023-10-08T09:16:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:05 smithi184 ceph-mon[103675]: pgmap v364: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 56 MiB/s wr, 1.79k op/s 2023-10-08T09:16:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:05 smithi132 ceph-mon[103389]: pgmap v364: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 56 MiB/s wr, 1.79k op/s 2023-10-08T09:16:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:05 smithi049 ceph-mon[99485]: pgmap v364: 129 pgs: 129 active+clean; 1.5 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 56 MiB/s wr, 1.79k op/s 2023-10-08T09:16:07.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:07 smithi184 ceph-mon[103675]: pgmap v365: 129 pgs: 129 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 56 MiB/s wr, 2.01k op/s 2023-10-08T09:16:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:07 smithi132 ceph-mon[103389]: pgmap v365: 129 pgs: 129 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 56 MiB/s wr, 2.01k op/s 2023-10-08T09:16:08.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:07 smithi049 ceph-mon[99485]: pgmap v365: 129 pgs: 129 active+clean; 1.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 56 MiB/s wr, 2.01k op/s 2023-10-08T09:16:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 45.0 s, 2820.2 tps, lat 35.324 ms stddev 40.714 2023-10-08T09:16:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:09 smithi184 ceph-mon[103675]: pgmap v366: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 MiB/s wr, 2.09k op/s 2023-10-08T09:16:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:09 smithi132 ceph-mon[103389]: pgmap v366: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 MiB/s wr, 2.09k op/s 2023-10-08T09:16:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:09 smithi049 ceph-mon[99485]: pgmap v366: 129 pgs: 129 active+clean; 1.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 MiB/s wr, 2.09k op/s 2023-10-08T09:16:10.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:16:10.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 -- 172.21.15.49:0/1924362516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc12010a5e0 msgr2=0x7fc120113480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.542 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 --2- 172.21.15.49:0/1924362516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc12010a5e0 0x7fc120113480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc1140098f0 tx=0x7fc114009c00 comp rx=0 tx=0).stop 2023-10-08T09:16:10.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 -- 172.21.15.49:0/1924362516 shutdown_connections 2023-10-08T09:16:10.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 --2- 172.21.15.49:0/1924362516 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1201077f0 0x7fc120109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 --2- 172.21.15.49:0/1924362516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc12010a5e0 0x7fc120113480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 --2- 172.21.15.49:0/1924362516 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a160 0x7fc120110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 -- 172.21.15.49:0/1924362516 >> 172.21.15.49:0/1924362516 conn(0x7fc12006d790 msgr2=0x7fc12006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:10.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 -- 172.21.15.49:0/1924362516 shutdown_connections 2023-10-08T09:16:10.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.540+0000 7fc125f6c700 1 -- 172.21.15.49:0/1924362516 wait complete. 2023-10-08T09:16:10.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.542+0000 7fc125f6c700 1 Processor -- start 2023-10-08T09:16:10.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.542+0000 7fc125f6c700 1 -- start start 2023-10-08T09:16:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.543+0000 7fc125f6c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1201077f0 0x7fc1201a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.543+0000 7fc125f6c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc12010a160 0x7fc1201a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.546 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.543+0000 7fc125f6c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a5e0 0x7fc1201acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.543+0000 7fc125f6c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1201ad070 con 0x7fc12010a5e0 2023-10-08T09:16:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.543+0000 7fc125f6c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1201ad1b0 con 0x7fc12010a160 2023-10-08T09:16:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.543+0000 7fc125f6c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1201ad340 con 0x7fc1201077f0 2023-10-08T09:16:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a5e0 0x7fc1201acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a5e0 0x7fc1201acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50706/0 (socket says 172.21.15.49:50706) 2023-10-08T09:16:10.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 -- 172.21.15.49:0/738198104 learned_addr learned my addr 172.21.15.49:0/738198104 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1201077f0 0x7fc1201a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 -- 172.21.15.49:0/738198104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1201077f0 msgr2=0x7fc1201a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1201077f0 0x7fc1201a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 -- 172.21.15.49:0/738198104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc12010a160 msgr2=0x7fc1201a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc12010a160 0x7fc1201a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 -- 172.21.15.49:0/738198104 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114009520 con 0x7fc12010a5e0 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1201077f0 0x7fc1201a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:16:10.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.544+0000 7fc11ffff700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a5e0 0x7fc1201acaa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc114009230 tx=0x7fc11400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:10.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.546+0000 7fc11cff9700 1 -- 172.21.15.49:0/738198104 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11401c070 con 0x7fc12010a5e0 2023-10-08T09:16:10.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.546+0000 7fc11cff9700 1 -- 172.21.15.49:0/738198104 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc11400f7d0 con 0x7fc12010a5e0 2023-10-08T09:16:10.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.546+0000 7fc11cff9700 1 -- 172.21.15.49:0/738198104 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114004b70 con 0x7fc12010a5e0 2023-10-08T09:16:10.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.546+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1201ad5c0 con 0x7fc12010a5e0 2023-10-08T09:16:10.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.546+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1201ad990 con 0x7fc12010a5e0 2023-10-08T09:16:10.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.550+0000 7fc11cff9700 1 -- 172.21.15.49:0/738198104 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc114020070 con 0x7fc12010a5e0 2023-10-08T09:16:10.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.551+0000 7fc11cff9700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1580 0x7fc1080a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.551+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1580 0x7fc1080a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.551+0000 7fc11cff9700 1 -- 172.21.15.49:0/738198104 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc1140cdd90 con 0x7fc12010a5e0 2023-10-08T09:16:10.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.552+0000 7fc132155b80 1 Processor -- start 2023-10-08T09:16:10.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc132155b80 1 -- start start 2023-10-08T09:16:10.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1580 0x7fc1080a3a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc118005fd0 tx=0x7fc118009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:10.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc132155b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc132155b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc132155b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e81c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc132155b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507de8460 con 0x560507e80060 2023-10-08T09:16:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc132155b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507dd9c20 con 0x560507e81800 2023-10-08T09:16:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc132155b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507dd7540 con 0x560507e809c0 2023-10-08T09:16:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc11ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e81c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc11ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e81c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40330/0 (socket says 172.21.15.49:40330) 2023-10-08T09:16:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc11ffff700 1 -- 172.21.15.49:0/558769919 learned_addr learned my addr 172.21.15.49:0/558769919 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.553+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc11ffff700 1 -- 172.21.15.49:0/558769919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 msgr2=0x560507e81270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:16:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc11ffff700 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e81270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc11ffff700 1 -- 172.21.15.49:0/558769919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 msgr2=0x560507e80480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc11ffff700 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e80480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc11ffff700 1 -- 172.21.15.49:0/558769919 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560507bfdde0 con 0x560507e81800 2023-10-08T09:16:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc11ffff700 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e81c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1140a2b80 tx=0x7fc114009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d81c73330a6b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc105ffb700 1 -- 172.21.15.49:0/558769919 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11409b4b0 con 0x560507e81800 2023-10-08T09:16:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc105ffb700 1 -- 172.21.15.49:0/558769919 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc11409e070 con 0x560507e81800 2023-10-08T09:16:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.554+0000 7fc105ffb700 1 -- 172.21.15.49:0/558769919 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140974c0 con 0x560507e81800 2023-10-08T09:16:10.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.555+0000 7fc132155b80 1 -- 172.21.15.49:0/558769919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 msgr2=0x560507e81c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.555+0000 7fc132155b80 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e81c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1140a2b80 tx=0x7fc114009460 comp rx=0 tx=0).stop 2023-10-08T09:16:10.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 -- 172.21.15.49:0/558769919 shutdown_connections 2023-10-08T09:16:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e81c90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 --2- 172.21.15.49:0/558769919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e81270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 -- 172.21.15.49:0/558769919 >> 172.21.15.49:0/558769919 conn(0x560507df4260 msgr2=0x560507e7ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 -- 172.21.15.49:0/558769919 shutdown_connections 2023-10-08T09:16:10.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 -- 172.21.15.49:0/558769919 wait complete. 2023-10-08T09:16:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.557+0000 7fc132155b80 1 -- start start 2023-10-08T09:16:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 1 Processor -- start 2023-10-08T09:16:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e89b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e809c0 0x560507e8a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e81800 0x560507e910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.561 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507dd2e30 con 0x560507e80060 2023-10-08T09:16:10.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507d6f710 con 0x560507e809c0 2023-10-08T09:16:10.562 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507d6f3b0 con 0x560507e81800 2023-10-08T09:16:10.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.558+0000 7fc132155b80 10 client.0 _ll_register_callbacks cb 0x560507bcc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:10.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.559+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e89b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.564 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.559+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e89b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50718/0 (socket says 172.21.15.49:50718) 2023-10-08T09:16:10.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.559+0000 7fc11f7fe700 1 -- 172.21.15.49:0/1647075285 learned_addr learned my addr 172.21.15.49:0/1647075285 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:10.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11ffff700 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e81800 0x560507e910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11effd700 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e809c0 0x560507e8a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11ffff700 1 -- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e809c0 msgr2=0x560507e8a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:10.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11ffff700 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e809c0 0x560507e8a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11ffff700 1 -- 172.21.15.49:0/1647075285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 msgr2=0x560507e89b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:10.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11ffff700 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e89b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11ffff700 1 -- 172.21.15.49:0/1647075285 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1201ad990 con 0x560507e81800 2023-10-08T09:16:10.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc11ffff700 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e81800 0x560507e910c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1140098c0 tx=0x7fc11409a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b81a1cbc951dc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:10.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc0feffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:16:10.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc0feffd700 1 -- 172.21.15.49:0/1647075285 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11409adb0 con 0x560507e81800 2023-10-08T09:16:10.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc0feffd700 1 -- 172.21.15.49:0/1647075285 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc11409b220 con 0x560507e81800 2023-10-08T09:16:10.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.560+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560507d9aeb0 con 0x560507e81800 2023-10-08T09:16:10.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc0feffd700 1 -- 172.21.15.49:0/1647075285 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc11409a2a0 con 0x560507e81800 2023-10-08T09:16:10.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 10 client.25633 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:10.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560507be6c60 con 0x560507e81800 2023-10-08T09:16:10.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc0feffd700 1 -- 172.21.15.49:0/1647075285 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560507d9aeb0 con 0x560507e81800 2023-10-08T09:16:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc0feffd700 1 -- 172.21.15.49:0/1647075285 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc114096ba0 con 0x560507e81800 2023-10-08T09:16:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 10 client.25633 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 20 client.25633 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507e95530 0x560507e97920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 4 client.25633 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:16:10.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560507dcbe10 con 0x560507e95530 2023-10-08T09:16:10.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.561+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507e95530 0x560507e97920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.562+0000 7fc0feffd700 10 client.25633 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:10.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.562+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507e95530 0x560507e97920 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a8994b07306adf server_cookie=97020e6db3fcdc3d in_seq=0 out_seq=0 2023-10-08T09:16:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.563+0000 7fc0feffd700 1 -- 172.21.15.49:0/1647075285 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560507dcbe10 con 0x560507e95530 2023-10-08T09:16:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.563+0000 7fc0feffd700 10 client.25633 handle_command_reply: tid=0 2023-10-08T09:16:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.563+0000 7fc132155b80 1 client.25633 shutdown 2023-10-08T09:16:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e81800 msgr2=0x560507e910c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e81800 0x560507e910c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1140098c0 tx=0x7fc11409a7d0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.571 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 shutdown_connections 2023-10-08T09:16:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507e95530 0x560507e97920 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e81800 0x560507e910c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e80060 0x560507e89b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 --2- 172.21.15.49:0/1647075285 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e809c0 0x560507e8a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 >> 172.21.15.49:0/1647075285 conn(0x560507df4260 msgr2=0x560507e7a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc0feffd700 0 client.25633 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:10.572 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 shutdown_connections 2023-10-08T09:16:10.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.564+0000 7fc132155b80 1 -- 172.21.15.49:0/1647075285 wait complete. 2023-10-08T09:16:10.573 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.565+0000 7fc132155b80 20 client.25633 trim_cache size 0 max 16384 2023-10-08T09:16:10.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 Processor -- start 2023-10-08T09:16:10.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 -- start start 2023-10-08T09:16:10.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e80060 0x560507e87de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e809c0 0x560507e91080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e84800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507c53920 con 0x560507e809c0 2023-10-08T09:16:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507dd9c20 con 0x560507e81800 2023-10-08T09:16:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.595+0000 7fc132155b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507d9aeb0 con 0x560507e80060 2023-10-08T09:16:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e80060 0x560507e87de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11effd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e809c0 0x560507e91080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e80060 0x560507e87de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37760/0 (socket says 172.21.15.49:37760) 2023-10-08T09:16:10.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11f7fe700 1 -- 172.21.15.49:0/145645236 learned_addr learned my addr 172.21.15.49:0/145645236 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11ffff700 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e84800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11effd700 1 -- 172.21.15.49:0/145645236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e80060 msgr2=0x560507e87de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11effd700 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e80060 0x560507e87de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11effd700 1 -- 172.21.15.49:0/145645236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 msgr2=0x560507e84800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11effd700 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e84800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.596+0000 7fc11effd700 1 -- 172.21.15.49:0/145645236 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560507c24250 con 0x560507e809c0 2023-10-08T09:16:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.597+0000 7fc11effd700 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e809c0 0x560507e91080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc110009d80 tx=0x7fc110009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166be363308bfdea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.597+0000 7fc1067fc700 1 -- 172.21.15.49:0/145645236 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110010040 con 0x560507e809c0 2023-10-08T09:16:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.597+0000 7fc1067fc700 1 -- 172.21.15.49:0/145645236 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1100148e0 con 0x560507e809c0 2023-10-08T09:16:10.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.597+0000 7fc1067fc700 1 -- 172.21.15.49:0/145645236 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110013770 con 0x560507e809c0 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.598+0000 7fc132155b80 1 -- 172.21.15.49:0/145645236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e809c0 msgr2=0x560507e91080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.598+0000 7fc132155b80 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e809c0 0x560507e91080 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc110009d80 tx=0x7fc110009550 comp rx=0 tx=0).stop 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.598+0000 7fc132155b80 1 -- 172.21.15.49:0/145645236 shutdown_connections 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.598+0000 7fc132155b80 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e80060 0x560507e87de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.598+0000 7fc132155b80 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e81800 0x560507e84800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.598+0000 7fc132155b80 1 --2- 172.21.15.49:0/145645236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e809c0 0x560507e91080 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.598+0000 7fc132155b80 1 -- 172.21.15.49:0/145645236 >> 172.21.15.49:0/145645236 conn(0x560507df4260 msgr2=0x560507e88800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:10.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.599+0000 7fc132155b80 1 -- 172.21.15.49:0/145645236 shutdown_connections 2023-10-08T09:16:10.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.599+0000 7fc132155b80 1 -- 172.21.15.49:0/145645236 wait complete. 2023-10-08T09:16:10.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 -- start start 2023-10-08T09:16:10.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 Processor -- start 2023-10-08T09:16:10.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e80060 0x560507e909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e90f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e81800 0x560507e8efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507e8f5b0 con 0x560507e81800 2023-10-08T09:16:10.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507e8f720 con 0x560507e80060 2023-10-08T09:16:10.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560507e8f8e0 con 0x560507e809c0 2023-10-08T09:16:10.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.600+0000 7fc132155b80 10 client.0 _ll_register_callbacks cb 0x7fc118004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:10.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e80060 0x560507e909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e80060 0x560507e909d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40344/0 (socket says 172.21.15.49:40344) 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 -- 172.21.15.49:0/2019321230 learned_addr learned my addr 172.21.15.49:0/2019321230 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11effd700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e90f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11ffff700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e81800 0x560507e8efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 -- 172.21.15.49:0/2019321230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 msgr2=0x560507e90f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e90f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 -- 172.21.15.49:0/2019321230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e81800 msgr2=0x560507e8efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e81800 0x560507e8efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 -- 172.21.15.49:0/2019321230 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc110009a60 con 0x560507e80060 2023-10-08T09:16:10.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11effd700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e90f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:16:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc11f7fe700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e80060 0x560507e909d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc118009f50 tx=0x7fc118004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad45603696711df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc0feffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:16:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.601+0000 7fc0feffd700 1 -- 172.21.15.49:0/2019321230 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118024040 con 0x560507e80060 2023-10-08T09:16:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc11409e070 con 0x560507e80060 2023-10-08T09:16:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc0feffd700 1 -- 172.21.15.49:0/2019321230 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc118003880 con 0x560507e80060 2023-10-08T09:16:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc0feffd700 1 -- 172.21.15.49:0/2019321230 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc118003a40 con 0x560507e80060 2023-10-08T09:16:10.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 10 client.25613 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:10.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x560507bfc2b0 con 0x560507e80060 2023-10-08T09:16:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc0feffd700 1 -- 172.21.15.49:0/2019321230 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc11409e070 con 0x560507e80060 2023-10-08T09:16:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc0feffd700 1 -- 172.21.15.49:0/2019321230 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc118032be0 con 0x560507e80060 2023-10-08T09:16:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 10 client.25613 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 10 client.25613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 20 client.25613 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:10.608 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507efb820 0x560507efdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 4 client.25613 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.602+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560507dcbb30 con 0x560507efb820 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.603+0000 7fc11effd700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507efb820 0x560507efdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.603+0000 7fc0feffd700 10 client.25613 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.603+0000 7fc11effd700 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507efb820 0x560507efdc40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2d9b39e5a04b98 server_cookie=b7af3be1444e7d4a in_seq=0 out_seq=0 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.603+0000 7fc0feffd700 1 -- 172.21.15.49:0/2019321230 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560507dcbb30 con 0x560507efb820 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.603+0000 7fc0feffd700 10 client.25613 handle_command_reply: tid=0 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.603+0000 7fc132155b80 1 client.25613 shutdown 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e80060 msgr2=0x560507e909d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.609 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e80060 0x560507e909d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc118009f50 tx=0x7fc118004750 comp rx=0 tx=0).stop 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 shutdown_connections 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560507efb820 0x560507efdc40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x560507e81800 0x560507e8efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x560507e80060 0x560507e909d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc0feffd700 0 client.25613 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 --2- 172.21.15.49:0/2019321230 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560507e809c0 0x560507e90f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 >> 172.21.15.49:0/2019321230 conn(0x560507df4260 msgr2=0x560507e87de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:10.610 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 shutdown_connections 2023-10-08T09:16:10.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 1 -- 172.21.15.49:0/2019321230 wait complete. 2023-10-08T09:16:10.611 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.604+0000 7fc132155b80 20 client.25613 trim_cache size 0 max 16384 2023-10-08T09:16:10.611 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:16:10.611 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-08T09:16:10.611 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:16:10.611 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:16:10.612 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:16:10.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:16:10.612 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:16:10.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:16:10.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:16:10.612 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:16:10.613 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1580 msgr2=0x7fc1080a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1580 0x7fc1080a3a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc118005fd0 tx=0x7fc118009310 comp rx=0 tx=0).stop 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a5e0 msgr2=0x7fc1201acaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a5e0 0x7fc1201acaa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc114009230 tx=0x7fc11400b840 comp rx=0 tx=0).stop 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 shutdown_connections 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1080a1580 0x7fc1080a3a40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1201077f0 0x7fc1201a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc12010a5e0 0x7fc1201acaa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 --2- 172.21.15.49:0/738198104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc12010a160 0x7fc1201a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:10.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.606+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 >> 172.21.15.49:0/738198104 conn(0x7fc12006d790 msgr2=0x7fc12006f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:10.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.607+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 shutdown_connections 2023-10-08T09:16:10.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:10.608+0000 7fc125f6c700 1 -- 172.21.15.49:0/738198104 wait complete. 2023-10-08T09:16:10.640 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:16:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:11 smithi184 ceph-mon[103675]: pgmap v367: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 MiB/s wr, 1.98k op/s 2023-10-08T09:16:12.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:11 smithi132 ceph-mon[103389]: pgmap v367: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 MiB/s wr, 1.98k op/s 2023-10-08T09:16:12.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:11 smithi049 ceph-mon[99485]: pgmap v367: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 MiB/s wr, 1.98k op/s 2023-10-08T09:16:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 -- 172.21.15.49:0/4249050011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07f8102580 msgr2=0x7f07f81029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 --2- 172.21.15.49:0/4249050011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07f8102580 0x7f07f81029e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009990 tx=0x7f07e8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 -- 172.21.15.49:0/4249050011 shutdown_connections 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 --2- 172.21.15.49:0/4249050011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 0x7f07f810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 --2- 172.21.15.49:0/4249050011 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07f8104050 0x7f07f8104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 --2- 172.21.15.49:0/4249050011 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07f8102580 0x7f07f81029e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 -- 172.21.15.49:0/4249050011 >> 172.21.15.49:0/4249050011 conn(0x7f07f80fdf00 msgr2=0x7f07f8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.382+0000 7f07fedc6700 1 -- 172.21.15.49:0/4249050011 shutdown_connections 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 -- 172.21.15.49:0/4249050011 wait complete. 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 Processor -- start 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 -- start start 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07f8102580 0x7f07f819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 0x7f07f819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07f8104050 0x7f07f81a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81a4960 con 0x7f07f8102f20 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81a4aa0 con 0x7f07f8102580 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07fedc6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81a4c30 con 0x7f07f8104050 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.383+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 0x7f07f819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 0x7f07f819d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50746/0 (socket says 172.21.15.49:50746) 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 -- 172.21.15.49:0/1555807538 learned_addr learned my addr 172.21.15.49:0/1555807538 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07fcb62700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07f8102580 0x7f07f819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 -- 172.21.15.49:0/1555807538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07f8104050 msgr2=0x7f07f81a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07f8104050 0x7f07f81a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 -- 172.21.15.49:0/1555807538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07f8102580 msgr2=0x7f07f819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07f8102580 0x7f07f819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 -- 172.21.15.49:0/1555807538 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80095f0 con 0x7f07f8102f20 2023-10-08T09:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07fcb62700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07f8102580 0x7f07f819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.384+0000 7f07f7fff700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 0x7f07f819d370 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b680 tx=0x7f07e8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.386+0000 7f07f5ffb700 1 -- 172.21.15.49:0/1555807538 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e801d070 con 0x7f07f8102f20 2023-10-08T09:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.386+0000 7f07f5ffb700 1 -- 172.21.15.49:0/1555807538 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07e8022470 con 0x7f07f8102f20 2023-10-08T09:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.386+0000 7f07f5ffb700 1 -- 172.21.15.49:0/1555807538 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8017650 con 0x7f07f8102f20 2023-10-08T09:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.386+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f81a4eb0 con 0x7f07f8102f20 2023-10-08T09:16:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.386+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f81a5220 con 0x7f07f8102f20 2023-10-08T09:16:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.389+0000 7f07f5ffb700 1 -- 172.21.15.49:0/1555807538 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f07e8022870 con 0x7f07f8102f20 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 Processor -- start 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 -- start start 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f65cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f66730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3ecd400 con 0x55a4c3f64e40 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3ebebc0 con 0x55a4c3f662b0 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f080afafb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3ebc4e0 con 0x55a4c3f657a0 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f07fd363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f66730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f07fd363700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f65260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50750/0 (socket says 172.21.15.49:50750) 2023-10-08T09:16:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f07fd363700 1 -- 172.21.15.49:0/2491283117 learned_addr learned my addr 172.21.15.49:0/2491283117 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f07fcb62700 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f65cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.391+0000 7f07f5ffb700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07e00a15d0 0x7f07e00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07f7fff700 1 -- 172.21.15.49:0/2491283117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 msgr2=0x55a4c3f65cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07f7fff700 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f65cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07f7fff700 1 -- 172.21.15.49:0/2491283117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 msgr2=0x55a4c3f65260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07f7fff700 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f65260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07f7fff700 1 -- 172.21.15.49:0/2491283117 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4c3ce2d80 con 0x55a4c3f662b0 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07f7fff700 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f66730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07e80cbb50 tx=0x7f07e809af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1222523acf97c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07f5ffb700 1 -- 172.21.15.49:0/1555807538 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07e80ce330 con 0x7f07f8102f20 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.392+0000 7f07fd363700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07e00a15d0 0x7f07e00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.393+0000 7f07deffd700 1 -- 172.21.15.49:0/2491283117 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e809bab0 con 0x55a4c3f662b0 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.393+0000 7f07deffd700 1 -- 172.21.15.49:0/2491283117 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07e809b3b0 con 0x55a4c3f662b0 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.393+0000 7f07deffd700 1 -- 172.21.15.49:0/2491283117 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80a36b0 con 0x55a4c3f662b0 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.393+0000 7f07fcb62700 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f65cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 -- 172.21.15.49:0/2491283117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 msgr2=0x55a4c3f66730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f66730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07e80cbb50 tx=0x7f07e809af70 comp rx=0 tx=0).stop 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f07fd363700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07e00a15d0 0x7f07e00a3a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009900 tx=0x7f07ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 -- 172.21.15.49:0/2491283117 shutdown_connections 2023-10-08T09:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f66730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f65260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 --2- 172.21.15.49:0/2491283117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f65cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 -- 172.21.15.49:0/2491283117 >> 172.21.15.49:0/2491283117 conn(0x55a4c3ed9200 msgr2=0x55a4c3f5f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 -- 172.21.15.49:0/2491283117 shutdown_connections 2023-10-08T09:16:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.394+0000 7f080afafb80 1 -- 172.21.15.49:0/2491283117 wait complete. 2023-10-08T09:16:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 -- start start 2023-10-08T09:16:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 Processor -- start 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3eb7dd0 con 0x55a4c3f662b0 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3e546b0 con 0x55a4c3f657a0 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3e54350 con 0x55a4c3f64e40 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f080afafb80 10 client.0 _ll_register_callbacks cb 0x55a4c3cb17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.395+0000 7f07fcb62700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40374/0 (socket says 172.21.15.49:40374) 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07f7fff700 1 -- 172.21.15.49:0/1598880619 learned_addr learned my addr 172.21.15.49:0/1598880619 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07fd363700 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07fcb62700 1 -- 172.21.15.49:0/1598880619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f657a0 msgr2=0x55a4c3f6ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07fcb62700 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07fcb62700 1 -- 172.21.15.49:0/1598880619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f662b0 msgr2=0x55a4c3f75a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07fcb62700 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07fcb62700 1 -- 172.21.15.49:0/1598880619 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f81a5220 con 0x55a4c3f64e40 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07fcb62700 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6e4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009c00 tx=0x7f07f00094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484ca2cc1d76c3ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.396+0000 7f07d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.397+0000 7f07d7fff700 1 -- 172.21.15.49:0/1598880619 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00048b0 con 0x55a4c3f64e40 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.397+0000 7f07d7fff700 1 -- 172.21.15.49:0/1598880619 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07f0004bb0 con 0x55a4c3f64e40 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.397+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4c3e7fe50 con 0x55a4c3f64e40 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.397+0000 7f07d7fff700 1 -- 172.21.15.49:0/1598880619 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f0004d70 con 0x55a4c3f64e40 2023-10-08T09:16:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.397+0000 7f080afafb80 10 client.25648 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.397+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a4c3ccbc00 con 0x55a4c3f64e40 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.397+0000 7f07d7fff700 1 -- 172.21.15.49:0/1598880619 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4c3e7fe50 con 0x55a4c3f64e40 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f07d7fff700 1 -- 172.21.15.49:0/1598880619 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f07f000e720 con 0x55a4c3f64e40 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f080afafb80 10 client.25648 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f080afafb80 10 client.25648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f080afafb80 20 client.25648 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f080afafb80 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3f7a400 0x55a4c3f7c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f080afafb80 4 client.25648 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4c3eb0db0 con 0x55a4c3f7a400 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.398+0000 7f07f7fff700 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3f7a400 0x55a4c3f7c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.399+0000 7f07f7fff700 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3f7a400 0x55a4c3f7c820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad37a5eb8c1c2b2 server_cookie=9ba179041d9cb3fc in_seq=0 out_seq=0 2023-10-08T09:16:12.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.399+0000 7f07d7fff700 10 client.25648 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f07d7fff700 1 -- 172.21.15.49:0/1598880619 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a4c3eb0db0 con 0x55a4c3f7a400 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f07d7fff700 10 client.25648 handle_command_reply: tid=0 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f080afafb80 1 client.25648 shutdown 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 msgr2=0x55a4c3f6e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f080afafb80 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6e4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009c00 tx=0x7f07f00094a0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 shutdown_connections 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f080afafb80 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3f7a400 0x55a4c3f7c820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.400+0000 7f080afafb80 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6e4e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.401+0000 7f080afafb80 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.401+0000 7f080afafb80 1 --2- 172.21.15.49:0/1598880619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.401+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 >> 172.21.15.49:0/1598880619 conn(0x55a4c3ed9200 msgr2=0x55a4c3f5f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.401+0000 7f07d7fff700 0 client.25648 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.401+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 shutdown_connections 2023-10-08T09:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.401+0000 7f080afafb80 1 -- 172.21.15.49:0/1598880619 wait complete. 2023-10-08T09:16:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.401+0000 7f080afafb80 20 client.25648 trim_cache size 0 max 16384 2023-10-08T09:16:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.426+0000 7f080afafb80 1 Processor -- start 2023-10-08T09:16:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f080afafb80 1 -- start start 2023-10-08T09:16:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f080afafb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f080afafb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f080afafb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f080afafb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3d388c0 con 0x55a4c3f64e40 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f080afafb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3f5a2b0 con 0x55a4c3f662b0 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f080afafb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3f6d4d0 con 0x55a4c3f657a0 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f07fd363700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f07f7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37800/0 (socket says 172.21.15.49:37800) 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f07fcb62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.428+0000 7f07f7fff700 1 -- 172.21.15.49:0/3778880851 learned_addr learned my addr 172.21.15.49:0/3778880851 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.429+0000 7f07fcb62700 1 -- 172.21.15.49:0/3778880851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 msgr2=0x55a4c3f6dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.429+0000 7f07fcb62700 1 --2- 172.21.15.49:0/3778880851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.429+0000 7f07fcb62700 1 -- 172.21.15.49:0/3778880851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 msgr2=0x55a4c3f75980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.429+0000 7f07fcb62700 1 --2- 172.21.15.49:0/3778880851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.429+0000 7f07fcb62700 1 -- 172.21.15.49:0/3778880851 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4c3d091f0 con 0x55a4c3f64e40 2023-10-08T09:16:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.429+0000 7f07fcb62700 1 --2- 172.21.15.49:0/3778880851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6d6c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f07f000eab0 tx=0x7f07f000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272b3fb6f41bbb60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.429+0000 7f07df7fe700 1 -- 172.21.15.49:0/3778880851 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000e4f0 con 0x55a4c3f64e40 2023-10-08T09:16:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.430+0000 7f07df7fe700 1 -- 172.21.15.49:0/3778880851 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07f000f040 con 0x55a4c3f64e40 2023-10-08T09:16:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.430+0000 7f07df7fe700 1 -- 172.21.15.49:0/3778880851 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0007760 con 0x55a4c3f64e40 2023-10-08T09:16:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 -- 172.21.15.49:0/3778880851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 msgr2=0x55a4c3f6d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 --2- 172.21.15.49:0/3778880851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6d6c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f07f000eab0 tx=0x7f07f000ee90 comp rx=0 tx=0).stop 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 -- 172.21.15.49:0/3778880851 shutdown_connections 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 --2- 172.21.15.49:0/3778880851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f75980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 --2- 172.21.15.49:0/3778880851 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f64e40 0x55a4c3f6d6c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 --2- 172.21.15.49:0/3778880851 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f657a0 0x55a4c3f6dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 -- 172.21.15.49:0/3778880851 >> 172.21.15.49:0/3778880851 conn(0x55a4c3ed9200 msgr2=0x55a4c3f5bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 -- 172.21.15.49:0/3778880851 shutdown_connections 2023-10-08T09:16:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.431+0000 7f080afafb80 1 -- 172.21.15.49:0/3778880851 wait complete. 2023-10-08T09:16:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 -- start start 2023-10-08T09:16:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 Processor -- start 2023-10-08T09:16:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f70d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f657a0 0x55a4c3f712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f74270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3f747b0 con 0x55a4c3f657a0 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3f74920 con 0x55a4c3f662b0 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4c3f74ae0 con 0x55a4c3f64e40 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.432+0000 7f080afafb80 10 client.0 _ll_register_callbacks cb 0x7f07f00093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.433+0000 7f07fd363700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f74270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.433+0000 7f07fd363700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f74270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40388/0 (socket says 172.21.15.49:40388) 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.433+0000 7f07fd363700 1 -- 172.21.15.49:0/2005910925 learned_addr learned my addr 172.21.15.49:0/2005910925 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.433+0000 7f07f7fff700 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f657a0 0x55a4c3f712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.433+0000 7f07fcb62700 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f70d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.435+0000 7f07fd363700 1 -- 172.21.15.49:0/2005910925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 msgr2=0x55a4c3f70d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.435+0000 7f07fd363700 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f70d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.435+0000 7f07fd363700 1 -- 172.21.15.49:0/2005910925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f657a0 msgr2=0x55a4c3f712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.435+0000 7f07fd363700 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f657a0 0x55a4c3f712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.435+0000 7f07fd363700 1 -- 172.21.15.49:0/2005910925 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4c3ccbc00 con 0x55a4c3f662b0 2023-10-08T09:16:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.439+0000 7f07fd363700 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f74270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07ec021b30 tx=0x7f07ec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d119f5cebbc1bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.439+0000 7f07d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:16:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.441+0000 7f07d7fff700 1 -- 172.21.15.49:0/2005910925 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec026070 con 0x55a4c3f662b0 2023-10-08T09:16:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.442+0000 7f07d7fff700 1 -- 172.21.15.49:0/2005910925 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07ec004030 con 0x55a4c3f662b0 2023-10-08T09:16:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.442+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07e809b3b0 con 0x55a4c3f662b0 2023-10-08T09:16:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.445+0000 7f07d7fff700 1 -- 172.21.15.49:0/2005910925 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07ec012270 con 0x55a4c3f662b0 2023-10-08T09:16:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.445+0000 7f080afafb80 10 client.25631 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.445+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55a4c3ce1250 con 0x55a4c3f662b0 2023-10-08T09:16:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f07d7fff700 1 -- 172.21.15.49:0/2005910925 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07e809b3b0 con 0x55a4c3f662b0 2023-10-08T09:16:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f07d7fff700 1 -- 172.21.15.49:0/2005910925 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f07ec010420 con 0x55a4c3f662b0 2023-10-08T09:16:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f080afafb80 10 client.25631 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f080afafb80 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f080afafb80 20 client.25631 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f080afafb80 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3fce9b0 0x55a4c3fd0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f080afafb80 4 client.25631 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:16:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.446+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a4c3eb0ad0 con 0x55a4c3fce9b0 2023-10-08T09:16:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.453+0000 7f07fcb62700 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3fce9b0 0x55a4c3fd0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.462+0000 7f07fcb62700 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3fce9b0 0x55a4c3fd0dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c55d5c9fd3885e2 server_cookie=6381658f7a61784b in_seq=0 out_seq=0 2023-10-08T09:16:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.462+0000 7f07d7fff700 10 client.25631 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f07d7fff700 1 -- 172.21.15.49:0/2005910925 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a4c3eb0ad0 con 0x55a4c3fce9b0 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f07d7fff700 10 client.25631 handle_command_reply: tid=0 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 client.25631 shutdown 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 msgr2=0x55a4c3f74270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f74270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f07ec021b30 tx=0x7f07ec004750 comp rx=0 tx=0).stop 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 shutdown_connections 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a4c3fce9b0 0x55a4c3fd0dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a4c3f64e40 0x55a4c3f70d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a4c3f662b0 0x55a4c3f74270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 --2- 172.21.15.49:0/2005910925 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a4c3f657a0 0x55a4c3f712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 >> 172.21.15.49:0/2005910925 conn(0x55a4c3ed9200 msgr2=0x55a4c3f6d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.511+0000 7f07d7fff700 0 client.25631 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.512+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 shutdown_connections 2023-10-08T09:16:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.512+0000 7f080afafb80 1 -- 172.21.15.49:0/2005910925 wait complete. 2023-10-08T09:16:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.512+0000 7f080afafb80 20 client.25631 trim_cache size 0 max 16384 2023-10-08T09:16:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:16:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:16:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:16:12.514 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07e00a15d0 msgr2=0x7f07e00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07e00a15d0 0x7f07e00a3a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009900 tx=0x7f07ec009260 comp rx=0 tx=0).stop 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 msgr2=0x7f07f819d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 0x7f07f819d370 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b680 tx=0x7f07e8009580 comp rx=0 tx=0).stop 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 shutdown_connections 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07e00a15d0 0x7f07e00a3a90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07f8104050 0x7f07f81a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.515 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07f8102580 0x7f07f819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:12.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 --2- 172.21.15.49:0/1555807538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8102f20 0x7f07f819d370 secure :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b680 tx=0x7f07e8009580 comp rx=0 tx=0).stop 2023-10-08T09:16:12.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 >> 172.21.15.49:0/1555807538 conn(0x7f07f80fdf00 msgr2=0x7f07f80780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:12.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.513+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 shutdown_connections 2023-10-08T09:16:12.516 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:12.514+0000 7f07fedc6700 1 -- 172.21.15.49:0/1555807538 wait complete. 2023-10-08T09:16:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:13 smithi184 ceph-mon[103675]: pgmap v368: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 46 MiB/s wr, 1.98k op/s 2023-10-08T09:16:14.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:13 smithi132 ceph-mon[103389]: pgmap v368: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 46 MiB/s wr, 1.98k op/s 2023-10-08T09:16:14.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:13 smithi049 ceph-mon[99485]: pgmap v368: 129 pgs: 129 active+clean; 1.5 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 46 MiB/s wr, 1.98k op/s 2023-10-08T09:16:14.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 50.0 s, 2895.6 tps, lat 34.357 ms stddev 39.365 2023-10-08T09:16:15.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:15 smithi184 ceph-mon[103675]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2023-10-08T09:16:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:15 smithi132 ceph-mon[103389]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2023-10-08T09:16:16.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:15 smithi049 ceph-mon[99485]: pgmap v369: 129 pgs: 129 active+clean; 1.5 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2023-10-08T09:16:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:17 smithi184 ceph-mon[103675]: pgmap v370: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 1.24k op/s 2023-10-08T09:16:18.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:17 smithi132 ceph-mon[103389]: pgmap v370: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 1.24k op/s 2023-10-08T09:16:18.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:17 smithi049 ceph-mon[99485]: pgmap v370: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 26 MiB/s wr, 1.24k op/s 2023-10-08T09:16:19.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 55.0 s, 2760.0 tps, lat 36.418 ms stddev 44.087 2023-10-08T09:16:19.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:19 smithi184 ceph-mon[103675]: pgmap v371: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 20 MiB/s wr, 900 op/s 2023-10-08T09:16:19.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:19 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:16:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:19 smithi132 ceph-mon[103389]: pgmap v371: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 20 MiB/s wr, 900 op/s 2023-10-08T09:16:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:19 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:16:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:19 smithi049 ceph-mon[99485]: pgmap v371: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 20 MiB/s wr, 900 op/s 2023-10-08T09:16:20.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:19 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:16:21.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:16:21.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:16:21.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:20 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:16:21.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:16:21.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:16:21.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:20 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:16:21.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:16:21.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:16:21.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:20 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:16:22.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:21 smithi132 ceph-mon[103389]: pgmap v372: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 16 MiB/s wr, 718 op/s 2023-10-08T09:16:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:21 smithi049 ceph-mon[99485]: pgmap v372: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 16 MiB/s wr, 718 op/s 2023-10-08T09:16:22.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:21 smithi184 ceph-mon[103675]: pgmap v372: 129 pgs: 129 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 16 MiB/s wr, 718 op/s 2023-10-08T09:16:23.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:22 smithi049 ceph-mon[99485]: pgmap v373: 129 pgs: 129 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 751 op/s 2023-10-08T09:16:23.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:22 smithi184 ceph-mon[103675]: pgmap v373: 129 pgs: 129 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 751 op/s 2023-10-08T09:16:23.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:22 smithi132 ceph-mon[103389]: pgmap v373: 129 pgs: 129 active+clean; 1.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 751 op/s 2023-10-08T09:16:24.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 60.0 s, 1899.3 tps, lat 52.630 ms stddev 66.090 2023-10-08T09:16:25.296 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:25 smithi132 ceph-mon[103389]: pgmap v374: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 15 MiB/s wr, 753 op/s 2023-10-08T09:16:25.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:24 smithi049 ceph-mon[99485]: pgmap v374: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 15 MiB/s wr, 753 op/s 2023-10-08T09:16:25.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:24 smithi184 ceph-mon[103675]: pgmap v374: 129 pgs: 129 active+clean; 1.5 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 15 MiB/s wr, 753 op/s 2023-10-08T09:16:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.696+0000 7f5ba6548700 1 -- 172.21.15.49:0/1799256840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100c00 msgr2=0x7f5ba0101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.696+0000 7f5ba6548700 1 --2- 172.21.15.49:0/1799256840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100c00 0x7f5ba0101020 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b900098f0 tx=0x7f5b90009c00 comp rx=0 tx=0).stop 2023-10-08T09:16:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.697+0000 7f5ba6548700 1 -- 172.21.15.49:0/1799256840 shutdown_connections 2023-10-08T09:16:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.697+0000 7f5ba6548700 1 --2- 172.21.15.49:0/1799256840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100c00 0x7f5ba0101020 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.697+0000 7f5ba6548700 1 --2- 172.21.15.49:0/1799256840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba0100210 0x7f5ba0107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.697+0000 7f5ba6548700 1 --2- 172.21.15.49:0/1799256840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 0x7f5ba00ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.697+0000 7f5ba6548700 1 -- 172.21.15.49:0/1799256840 >> 172.21.15.49:0/1799256840 conn(0x7f5ba00fa340 msgr2=0x7f5ba00fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.698+0000 7f5ba6548700 1 -- 172.21.15.49:0/1799256840 shutdown_connections 2023-10-08T09:16:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.698+0000 7f5ba6548700 1 -- 172.21.15.49:0/1799256840 wait complete. 2023-10-08T09:16:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 Processor -- start 2023-10-08T09:16:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 -- start start 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 0x7f5ba006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100210 0x7f5ba006f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba0100c00 0x7f5ba01a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00726d0 con 0x7f5ba00ff850 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba0072840 con 0x7f5ba0100c00 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba6548700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba01a4af0 con 0x7f5ba0100210 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 0x7f5ba006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 0x7f5ba006f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50786/0 (socket says 172.21.15.49:50786) 2023-10-08T09:16:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5b9ffff700 1 -- 172.21.15.49:0/3792854304 learned_addr learned my addr 172.21.15.49:0/3792854304 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba0100c00 0x7f5ba01a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.699+0000 7f5b9f7fe700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100210 0x7f5ba006f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9ffff700 1 -- 172.21.15.49:0/3792854304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100210 msgr2=0x7f5ba006f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100210 0x7f5ba006f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9ffff700 1 -- 172.21.15.49:0/3792854304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba0100c00 msgr2=0x7f5ba01a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba0100c00 0x7f5ba01a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9ffff700 1 -- 172.21.15.49:0/3792854304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b90009520 con 0x7f5ba00ff850 2023-10-08T09:16:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 0x7f5ba006f1a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5b90005e90 tx=0x7f5b9000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9d7fa700 1 -- 172.21.15.49:0/3792854304 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90005450 con 0x7f5ba00ff850 2023-10-08T09:16:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9d7fa700 1 -- 172.21.15.49:0/3792854304 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9000be80 con 0x7f5ba00ff850 2023-10-08T09:16:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.700+0000 7f5b9d7fa700 1 -- 172.21.15.49:0/3792854304 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90003c40 con 0x7f5ba00ff850 2023-10-08T09:16:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.701+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba01a4d20 con 0x7f5ba00ff850 2023-10-08T09:16:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.702+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba01a50f0 con 0x7f5ba00ff850 2023-10-08T09:16:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.705+0000 7f5b9d7fa700 1 -- 172.21.15.49:0/3792854304 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5b9001f070 con 0x7f5ba00ff850 2023-10-08T09:16:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.707+0000 7f5b9d7fa700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b880a15d0 0x7f5b880a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.707+0000 7f5b9d7fa700 1 -- 172.21.15.49:0/3792854304 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5b900d2340 con 0x7f5ba00ff850 2023-10-08T09:16:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.707+0000 7f5b9f7fe700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b880a15d0 0x7f5b880a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.707+0000 7f5bb2731b80 1 Processor -- start 2023-10-08T09:16:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5bb2731b80 1 -- start start 2023-10-08T09:16:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5b9f7fe700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b880a15d0 0x7f5b880a3a90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005fd0 tx=0x7f5b94009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c205260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2057a0 0x55bd9c205cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2062b0 0x55bd9c206730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c16d400 con 0x55bd9c2057a0 2023-10-08T09:16:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c15ebc0 con 0x55bd9c204e40 2023-10-08T09:16:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c15c4e0 con 0x55bd9c2062b0 2023-10-08T09:16:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c205260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.708+0000 7f5b9f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2057a0 0x55bd9c205cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c205260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40410/0 (socket says 172.21.15.49:40410) 2023-10-08T09:16:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 -- 172.21.15.49:0/4272209993 learned_addr learned my addr 172.21.15.49:0/4272209993 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 -- 172.21.15.49:0/4272209993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2062b0 msgr2=0x55bd9c206730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:16:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/4272209993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2062b0 0x55bd9c206730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 -- 172.21.15.49:0/4272209993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2057a0 msgr2=0x55bd9c205cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/4272209993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2057a0 0x55bd9c205cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 -- 172.21.15.49:0/4272209993 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9bf82d80 con 0x55bd9c204e40 2023-10-08T09:16:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.709+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/4272209993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c205260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5b9009b8c0 tx=0x7f5b90003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cdaddeb56d50e6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.710+0000 7f5b867fc700 1 -- 172.21.15.49:0/4272209993 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900223b0 con 0x55bd9c204e40 2023-10-08T09:16:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.710+0000 7f5b867fc700 1 -- 172.21.15.49:0/4272209993 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b900055b0 con 0x55bd9c204e40 2023-10-08T09:16:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.710+0000 7f5b867fc700 1 -- 172.21.15.49:0/4272209993 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900a76c0 con 0x55bd9c204e40 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4272209993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 msgr2=0x55bd9c205260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4272209993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c205260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5b9009b8c0 tx=0x7f5b90003680 comp rx=0 tx=0).stop 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4272209993 shutdown_connections 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4272209993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2062b0 0x55bd9c206730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4272209993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c205260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4272209993 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2057a0 0x55bd9c205cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4272209993 >> 172.21.15.49:0/4272209993 conn(0x55bd9c179200 msgr2=0x55bd9c1ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4272209993 shutdown_connections 2023-10-08T09:16:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.711+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4272209993 wait complete. 2023-10-08T09:16:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 -- start start 2023-10-08T09:16:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 Processor -- start 2023-10-08T09:16:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c204e40 0x55bd9c20e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2057a0 0x55bd9c20ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c215ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c157dd0 con 0x55bd9c2062b0 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c0f46b0 con 0x55bd9c2057a0 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.712+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c0f4350 con 0x55bd9c204e40 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5bb2731b80 10 client.0 _ll_register_callbacks cb 0x55bd9bf517a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c204e40 0x55bd9c20e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5ba4ae5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c215ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c204e40 0x55bd9c20e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37834/0 (socket says 172.21.15.49:37834) 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5b9ffff700 1 -- 172.21.15.49:0/4049944022 learned_addr learned my addr 172.21.15.49:0/4049944022 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5b9f7fe700 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2057a0 0x55bd9c20ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5ba4ae5700 1 -- 172.21.15.49:0/4049944022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c204e40 msgr2=0x55bd9c20e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c204e40 0x55bd9c20e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5ba4ae5700 1 -- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2057a0 msgr2=0x55bd9c20ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2057a0 0x55bd9c20ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5ba4ae5700 1 -- 172.21.15.49:0/4049944022 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba01a50f0 con 0x55bd9c2062b0 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c215ac0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5b98007af0 tx=0x7f5b98007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6deaa726d993914 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.713+0000 7f5b7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/4049944022 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98004db0 con 0x55bd9c2062b0 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/4049944022 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5b98003a00 con 0x55bd9c2062b0 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd9c11fe50 con 0x55bd9c2062b0 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/4049944022 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b98003bc0 con 0x55bd9c2062b0 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 10 client.16011 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bd9bf6bc00 con 0x55bd9c2062b0 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/4049944022 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd9c11fe50 con 0x55bd9c2062b0 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/4049944022 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5b980041f0 con 0x55bd9c2062b0 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 20 client.16011 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c21a480 0x55bd9c21c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 4 client.16011 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.714+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd9c150db0 con 0x55bd9c21a480 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.715+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c21a480 0x55bd9c21c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.716+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c21a480 0x55bd9c21c8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=addf7a0b1e0fa7b8 server_cookie=6f7b064aadc9aba1 in_seq=0 out_seq=0 2023-10-08T09:16:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.716+0000 7f5b7f7fe700 10 client.16011 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.716+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/4049944022 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bd9c150db0 con 0x55bd9c21a480 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.716+0000 7f5b7f7fe700 10 client.16011 handle_command_reply: tid=0 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 client.16011 shutdown 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 msgr2=0x55bd9c215ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c215ac0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5b98007af0 tx=0x7f5b98007e00 comp rx=0 tx=0).stop 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 shutdown_connections 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c21a480 0x55bd9c21c8a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c204e40 0x55bd9c20e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c215ac0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/4049944022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2057a0 0x55bd9c20ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 >> 172.21.15.49:0/4049944022 conn(0x55bd9c179200 msgr2=0x55bd9c1ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5b7f7fe700 0 client.16011 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 shutdown_connections 2023-10-08T09:16:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 1 -- 172.21.15.49:0/4049944022 wait complete. 2023-10-08T09:16:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.717+0000 7f5bb2731b80 20 client.16011 trim_cache size 0 max 16384 2023-10-08T09:16:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 Processor -- start 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 -- start start 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c216170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c215a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c206690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9bfd88c0 con 0x55bd9c2062b0 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c11fe50 con 0x55bd9c204e40 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c1fa2b0 con 0x55bd9c2057a0 2023-10-08T09:16:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5ba4ae5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c206690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5b9f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c215a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5b9f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c215a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37844/0 (socket says 172.21.15.49:37844) 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5ba4ae5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c206690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50812/0 (socket says 172.21.15.49:50812) 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5b9f7fe700 1 -- 172.21.15.49:0/1176595281 learned_addr learned my addr 172.21.15.49:0/1176595281 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.740+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c216170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5ba4ae5700 1 -- 172.21.15.49:0/1176595281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 msgr2=0x55bd9c215a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c215a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5ba4ae5700 1 -- 172.21.15.49:0/1176595281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 msgr2=0x55bd9c216170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c216170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5ba4ae5700 1 -- 172.21.15.49:0/1176595281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9bfa91f0 con 0x55bd9c2062b0 2023-10-08T09:16:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c216170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:16:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c206690 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000f80 tx=0x7f5b98009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c692fa609c9f1d92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5b86ffd700 1 -- 172.21.15.49:0/1176595281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b980119b0 con 0x55bd9c2062b0 2023-10-08T09:16:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5b86ffd700 1 -- 172.21.15.49:0/1176595281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b98016070 con 0x55bd9c2062b0 2023-10-08T09:16:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.741+0000 7f5b86ffd700 1 -- 172.21.15.49:0/1176595281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98012760 con 0x55bd9c2062b0 2023-10-08T09:16:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 -- 172.21.15.49:0/1176595281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 msgr2=0x55bd9c206690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c206690 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000f80 tx=0x7f5b98009ce0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 -- 172.21.15.49:0/1176595281 shutdown_connections 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c2062b0 0x55bd9c206690 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c204e40 0x55bd9c216170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/1176595281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c215a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 -- 172.21.15.49:0/1176595281 >> 172.21.15.49:0/1176595281 conn(0x55bd9c179200 msgr2=0x55bd9c2211e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 -- 172.21.15.49:0/1176595281 shutdown_connections 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.742+0000 7f5bb2731b80 1 -- 172.21.15.49:0/1176595281 wait complete. 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 -- start start 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 Processor -- start 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c204e40 0x55bd9c2130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c213610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2062b0 0x55bd9c214620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c214b60 con 0x55bd9c204e40 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c214cd0 con 0x55bd9c2062b0 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9c214e90 con 0x55bd9c2057a0 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.743+0000 7f5bb2731b80 10 client.0 _ll_register_callbacks cb 0x7f5b98009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c204e40 0x55bd9c2130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c213610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c204e40 0x55bd9c2130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50814/0 (socket says 172.21.15.49:50814) 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c213610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37850/0 (socket says 172.21.15.49:37850) 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 -- 172.21.15.49:0/2370657454 learned_addr learned my addr 172.21.15.49:0/2370657454 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5ba4ae5700 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2062b0 0x55bd9c214620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 -- 172.21.15.49:0/2370657454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 msgr2=0x55bd9c213610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c213610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 -- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2062b0 msgr2=0x55bd9c214620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2062b0 0x55bd9c214620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 -- 172.21.15.49:0/2370657454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd9bf6bc00 con 0x55bd9c204e40 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9f7fe700 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c213610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b9ffff700 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c204e40 0x55bd9c2130d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5b9009f0c0 tx=0x7f5b9009b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4541531a759f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b7f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/2370657454 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900223b0 con 0x55bd9c204e40 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.744+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/2370657454 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5b9009fac0 con 0x55bd9c204e40 2023-10-08T09:16:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b900055b0 con 0x55bd9c204e40 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/2370657454 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b900a76c0 con 0x55bd9c204e40 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 10 client.16017 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55bd9bf81250 con 0x55bd9c204e40 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/2370657454 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b900055b0 con 0x55bd9c204e40 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/2370657454 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5b900a3170 con 0x55bd9c204e40 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 20 client.16017 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c26e7d0 0x55bd9c270bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 4 client.16017 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bd9c150ad0 con 0x55bd9c26e7d0 2023-10-08T09:16:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.745+0000 7f5b9f7fe700 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c26e7d0 0x55bd9c270bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.746+0000 7f5b9f7fe700 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c26e7d0 0x55bd9c270bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98615903dcb32f01 server_cookie=f21be65742f32646 in_seq=0 out_seq=0 2023-10-08T09:16:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.747+0000 7f5b7f7fe700 10 client.16017 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:16:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:16:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:16:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:16:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.833+0000 7f5b7f7fe700 1 -- 172.21.15.49:0/2370657454 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bd9c150ad0 con 0x55bd9c26e7d0 2023-10-08T09:16:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.833+0000 7f5b7f7fe700 10 client.16017 handle_command_reply: tid=0 2023-10-08T09:16:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 client.16017 shutdown 2023-10-08T09:16:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c204e40 msgr2=0x55bd9c2130d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c204e40 0x55bd9c2130d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5b9009f0c0 tx=0x7f5b9009b810 comp rx=0 tx=0).stop 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 shutdown_connections 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bd9c26e7d0 0x55bd9c270bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bd9c2062b0 0x55bd9c214620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bd9c204e40 0x55bd9c2130d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5b7f7fe700 0 client.16017 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 --2- 172.21.15.49:0/2370657454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bd9c2057a0 0x55bd9c213610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 >> 172.21.15.49:0/2370657454 conn(0x55bd9c179200 msgr2=0x55bd9c20c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 shutdown_connections 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 1 -- 172.21.15.49:0/2370657454 wait complete. 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.834+0000 7f5bb2731b80 20 client.16017 trim_cache size 0 max 16384 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b880a15d0 msgr2=0x7f5b880a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b880a15d0 0x7f5b880a3a90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5b94005fd0 tx=0x7f5b94009310 comp rx=0 tx=0).stop 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 msgr2=0x7f5ba006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 0x7f5ba006f1a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5b90005e90 tx=0x7f5b9000bbe0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 shutdown_connections 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5b880a15d0 0x7f5b880a3a90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ba0100c00 0x7f5ba01a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ba00ff850 0x7f5ba006f1a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 --2- 172.21.15.49:0/3792854304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ba0100210 0x7f5ba006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 >> 172.21.15.49:0/3792854304 conn(0x7f5ba00fa340 msgr2=0x7f5ba0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 shutdown_connections 2023-10-08T09:16:25.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:25.835+0000 7f5ba6548700 1 -- 172.21.15.49:0/3792854304 wait complete. 2023-10-08T09:16:27.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:27 smithi184 ceph-mon[103675]: pgmap v375: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 17 MiB/s wr, 861 op/s 2023-10-08T09:16:27.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:27 smithi132 ceph-mon[103389]: pgmap v375: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 17 MiB/s wr, 861 op/s 2023-10-08T09:16:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:27 smithi049 ceph-mon[99485]: pgmap v375: 129 pgs: 129 active+clean; 1.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 17 MiB/s wr, 861 op/s 2023-10-08T09:16:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 65.0 s, 1854.6 tps, lat 53.844 ms stddev 60.042 2023-10-08T09:16:29.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:29 smithi184 ceph-mon[103675]: pgmap v376: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 18 MiB/s wr, 936 op/s 2023-10-08T09:16:29.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:29 smithi132 ceph-mon[103389]: pgmap v376: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 18 MiB/s wr, 936 op/s 2023-10-08T09:16:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:29 smithi049 ceph-mon[99485]: pgmap v376: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 18 MiB/s wr, 936 op/s 2023-10-08T09:16:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:31 smithi184 ceph-mon[103675]: pgmap v377: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 17 MiB/s wr, 899 op/s 2023-10-08T09:16:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:31 smithi132 ceph-mon[103389]: pgmap v377: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 17 MiB/s wr, 899 op/s 2023-10-08T09:16:31.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:31 smithi049 ceph-mon[99485]: pgmap v377: 129 pgs: 129 active+clean; 1.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 17 MiB/s wr, 899 op/s 2023-10-08T09:16:33.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:33 smithi132 ceph-mon[103389]: pgmap v378: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 1.02k op/s 2023-10-08T09:16:33.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:33 smithi049 ceph-mon[99485]: pgmap v378: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 1.02k op/s 2023-10-08T09:16:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:33 smithi184 ceph-mon[103675]: pgmap v378: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 20 MiB/s wr, 1.02k op/s 2023-10-08T09:16:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 70.0 s, 1954.0 tps, lat 50.586 ms stddev 56.450 2023-10-08T09:16:35.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:35 smithi132 ceph-mon[103389]: pgmap v379: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 990 op/s 2023-10-08T09:16:35.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:35 smithi049 ceph-mon[99485]: pgmap v379: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 990 op/s 2023-10-08T09:16:35.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:35 smithi184 ceph-mon[103675]: pgmap v379: 129 pgs: 129 active+clean; 1.5 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 20 MiB/s wr, 990 op/s 2023-10-08T09:16:37.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:37 smithi132 ceph-mon[103389]: pgmap v380: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-10-08T09:16:37.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:37 smithi049 ceph-mon[99485]: pgmap v380: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-10-08T09:16:37.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:37 smithi184 ceph-mon[103675]: pgmap v380: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.03k op/s 2023-10-08T09:16:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.040+0000 7fb653c2f700 1 -- 172.21.15.49:0/3535681905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 msgr2=0x7fb64c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.040+0000 7fb653c2f700 1 --2- 172.21.15.49:0/3535681905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 0x7fb64c1083e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb64801a9d0 tx=0x7fb64801ace0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.048+0000 7fb653c2f700 1 -- 172.21.15.49:0/3535681905 shutdown_connections 2023-10-08T09:16:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.048+0000 7fb653c2f700 1 --2- 172.21.15.49:0/3535681905 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c106d90 0x7fb64c1071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.048+0000 7fb653c2f700 1 --2- 172.21.15.49:0/3535681905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb64c108920 0x7fb64c10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.048+0000 7fb653c2f700 1 --2- 172.21.15.49:0/3535681905 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 0x7fb64c1083e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.048+0000 7fb653c2f700 1 -- 172.21.15.49:0/3535681905 >> 172.21.15.49:0/3535681905 conn(0x7fb64c075d20 msgr2=0x7fb64c078160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.058+0000 7fb653c2f700 1 -- 172.21.15.49:0/3535681905 shutdown_connections 2023-10-08T09:16:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.062+0000 7fb653c2f700 1 -- 172.21.15.49:0/3535681905 wait complete. 2023-10-08T09:16:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.067+0000 7fb653c2f700 1 Processor -- start 2023-10-08T09:16:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.077+0000 7fb653c2f700 1 -- start start 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb653c2f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb64c106d90 0x7fb64c196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb653c2f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 0x7fb64c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb653c2f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c108920 0x7fb64c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb653c2f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c19c000 con 0x7fb64c107f80 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb653c2f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c079070 con 0x7fb64c106d90 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb653c2f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c079230 con 0x7fb64c108920 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb6521cc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c108920 0x7fb64c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb6521cc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c108920 0x7fb64c19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37862/0 (socket says 172.21.15.49:37862) 2023-10-08T09:16:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.078+0000 7fb6521cc700 1 -- 172.21.15.49:0/4087814969 learned_addr learned my addr 172.21.15.49:0/4087814969 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb6511ca700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 0x7fb64c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb6521cc700 1 -- 172.21.15.49:0/4087814969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb64c106d90 msgr2=0x7fb64c196c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:16:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb6521cc700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb64c106d90 0x7fb64c196c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb6521cc700 1 -- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 msgr2=0x7fb64c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb6521cc700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 0x7fb64c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb6521cc700 1 -- 172.21.15.49:0/4087814969 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64801a600 con 0x7fb64c108920 2023-10-08T09:16:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb6521cc700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c108920 0x7fb64c19b7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb644007ee0 tx=0x7fb64400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb642ffd700 1 -- 172.21.15.49:0/4087814969 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644017070 con 0x7fb64c108920 2023-10-08T09:16:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb642ffd700 1 -- 172.21.15.49:0/4087814969 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64400f040 con 0x7fb64c108920 2023-10-08T09:16:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb642ffd700 1 -- 172.21.15.49:0/4087814969 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644005740 con 0x7fb64c108920 2023-10-08T09:16:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.079+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c0794b0 con 0x7fb64c108920 2023-10-08T09:16:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.080+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c079940 con 0x7fb64c108920 2023-10-08T09:16:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.087+0000 7fb65fe18b80 1 Processor -- start 2023-10-08T09:16:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.088+0000 7fb642ffd700 1 -- 172.21.15.49:0/4087814969 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb644004fd0 con 0x7fb64c108920 2023-10-08T09:16:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.089+0000 7fb642ffd700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6380a1580 0x7fb6380a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.089+0000 7fb642ffd700 1 -- 172.21.15.49:0/4087814969 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb6440cc810 con 0x7fb64c108920 2023-10-08T09:16:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.094+0000 7fb65fe18b80 1 -- start start 2023-10-08T09:16:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.094+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e5ce40 0x55b9d8e5d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.094+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e5dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.094+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8e5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.094+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8dc5400 con 0x55b9d8e5e2b0 2023-10-08T09:16:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.094+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8db6bc0 con 0x55b9d8e5ce40 2023-10-08T09:16:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.094+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8db44e0 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6511ca700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e5dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6511ca700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e5dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37870/0 (socket says 172.21.15.49:37870) 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6511ca700 1 -- 172.21.15.49:0/45682201 learned_addr learned my addr 172.21.15.49:0/45682201 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6519cb700 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e5ce40 0x55b9d8e5d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6519cb700 1 -- 172.21.15.49:0/45682201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 msgr2=0x55b9d8e5dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6519cb700 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e5dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6519cb700 1 -- 172.21.15.49:0/45682201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 msgr2=0x55b9d8e5e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6519cb700 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8e5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6519cb700 1 -- 172.21.15.49:0/45682201 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9d8bdad80 con 0x55b9d8e5ce40 2023-10-08T09:16:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.095+0000 7fb6519cb700 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e5ce40 0x55b9d8e5d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb64c107de0 tx=0x7fb63c009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad7837eca98cc8e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.096+0000 7fb637fff700 1 -- 172.21.15.49:0/45682201 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c004a30 con 0x55b9d8e5ce40 2023-10-08T09:16:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.096+0000 7fb637fff700 1 -- 172.21.15.49:0/45682201 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63c00ed00 con 0x55b9d8e5ce40 2023-10-08T09:16:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.096+0000 7fb637fff700 1 -- 172.21.15.49:0/45682201 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c005610 con 0x55b9d8e5ce40 2023-10-08T09:16:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.097+0000 7fb6511ca700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6380a1580 0x7fb6380a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.097+0000 7fb65fe18b80 1 -- 172.21.15.49:0/45682201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e5ce40 msgr2=0x55b9d8e5d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.097+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e5ce40 0x55b9d8e5d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb64c107de0 tx=0x7fb63c009e30 comp rx=0 tx=0).stop 2023-10-08T09:16:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.099+0000 7fb65fe18b80 1 -- 172.21.15.49:0/45682201 shutdown_connections 2023-10-08T09:16:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.099+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8e5e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.099+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e5ce40 0x55b9d8e5d260 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb64c107de0 tx=0x7fb63c009e30 comp rx=0 tx=0).stop 2023-10-08T09:16:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.099+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/45682201 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e5dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.099+0000 7fb65fe18b80 1 -- 172.21.15.49:0/45682201 >> 172.21.15.49:0/45682201 conn(0x55b9d8dd1200 msgr2=0x55b9d8e578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.100+0000 7fb6511ca700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6380a1580 0x7fb6380a3a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb648006010 tx=0x7fb64802a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:39.110 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.109+0000 7fb65fe18b80 1 -- 172.21.15.49:0/45682201 shutdown_connections 2023-10-08T09:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.111+0000 7fb65fe18b80 1 -- 172.21.15.49:0/45682201 wait complete. 2023-10-08T09:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 -- start start 2023-10-08T09:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 Processor -- start 2023-10-08T09:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e66c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8e67150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e6e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8dafdd0 con 0x55b9d8e5e2b0 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8d4c6b0 con 0x55b9d8e6c090 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8d4c350 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.112+0000 7fb65fe18b80 10 client.0 _ll_register_callbacks cb 0x55b9d8ba97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e6e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e6e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40456/0 (socket says 172.21.15.49:40456) 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 -- 172.21.15.49:0/2924531815 learned_addr learned my addr 172.21.15.49:0/2924531815 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 -- 172.21.15.49:0/2924531815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 msgr2=0x55b9d8e66c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e66c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 -- 172.21.15.49:0/2924531815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 msgr2=0x55b9d8e67150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:16:39.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8e67150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 -- 172.21.15.49:0/2924531815 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c009830 con 0x55b9d8e6c090 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.113+0000 7fb6521cc700 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e6e480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb644095c50 tx=0x7fb644099fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1f4e7f12380a28 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.114+0000 7fb6357fa700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.114+0000 7fb6357fa700 1 -- 172.21.15.49:0/2924531815 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440a1b30 con 0x55b9d8e6c090 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.114+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9d8d77e50 con 0x55b9d8e6c090 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.114+0000 7fb6357fa700 1 -- 172.21.15.49:0/2924531815 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb644004ad0 con 0x55b9d8e6c090 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.119+0000 7fb6357fa700 1 -- 172.21.15.49:0/2924531815 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6440ca090 con 0x55b9d8e6c090 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.120+0000 7fb65fe18b80 10 client.25658 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:39.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.120+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b9d8bc3c00 con 0x55b9d8e6c090 2023-10-08T09:16:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.122+0000 7fb6357fa700 1 -- 172.21.15.49:0/2924531815 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9d8d77e50 con 0x55b9d8e6c090 2023-10-08T09:16:39.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.122+0000 7fb6357fa700 1 -- 172.21.15.49:0/2924531815 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb6440b4080 con 0x55b9d8e6c090 2023-10-08T09:16:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.127+0000 7fb65fe18b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.127+0000 7fb65fe18b80 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.127+0000 7fb65fe18b80 20 client.25658 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.127+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8e94e50 0x55b9d8e97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.127+0000 7fb65fe18b80 4 client.25658 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:16:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.127+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9d8da8db0 con 0x55b9d8e94e50 2023-10-08T09:16:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.138+0000 7fb6519cb700 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8e94e50 0x55b9d8e97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.159+0000 7fb6519cb700 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8e94e50 0x55b9d8e97270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcdd3d138e3856d3 server_cookie=3bd8186de0f81a7b in_seq=0 out_seq=0 2023-10-08T09:16:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.166+0000 7fb6357fa700 10 client.25658 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.166+0000 7fb6357fa700 1 -- 172.21.15.49:0/2924531815 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b9d8da8db0 con 0x55b9d8e94e50 2023-10-08T09:16:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.166+0000 7fb6357fa700 10 client.25658 handle_command_reply: tid=0 2023-10-08T09:16:39.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.171+0000 7fb65fe18b80 1 client.25658 shutdown 2023-10-08T09:16:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.187+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 msgr2=0x55b9d8e6e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.187+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e6e480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb644095c50 tx=0x7fb644099fd0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.188+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 shutdown_connections 2023-10-08T09:16:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.188+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8e94e50 0x55b9d8e97270 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.188+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e66c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.188+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e6e480 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.188+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2924531815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8e67150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.188+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 >> 172.21.15.49:0/2924531815 conn(0x55b9d8dd1200 msgr2=0x55b9d8e57710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.188+0000 7fb6357fa700 0 client.25658 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.189+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 shutdown_connections 2023-10-08T09:16:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.189+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2924531815 wait complete. 2023-10-08T09:16:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.195+0000 7fb65fe18b80 20 client.25658 trim_cache size 0 max 16384 2023-10-08T09:16:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.225+0000 7fb65fe18b80 1 Processor -- start 2023-10-08T09:16:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.225+0000 7fb65fe18b80 1 -- start start 2023-10-08T09:16:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8ea4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8c308c0 con 0x55b9d8e5e2b0 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8da8de0 con 0x55b9d8e6c090 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8e58b00 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb6519cb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb6519cb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e6eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37880/0 (socket says 172.21.15.49:37880) 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb6519cb700 1 -- 172.21.15.49:0/2302481486 learned_addr learned my addr 172.21.15.49:0/2302481486 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb6521cc700 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.226+0000 7fb6511ca700 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8ea4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb6521cc700 1 -- 172.21.15.49:0/2302481486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 msgr2=0x55b9d8e6eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb6521cc700 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e6eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb6521cc700 1 -- 172.21.15.49:0/2302481486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 msgr2=0x55b9d8ea4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb6521cc700 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8ea4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb6521cc700 1 -- 172.21.15.49:0/2302481486 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9d8c011f0 con 0x55b9d8e6c090 2023-10-08T09:16:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb6521cc700 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e655f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb644097af0 tx=0x7fb6440b2150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=600325e2dd4d85b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb640d82700 1 -- 172.21.15.49:0/2302481486 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440a1b30 con 0x55b9d8e6c090 2023-10-08T09:16:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb640d82700 1 -- 172.21.15.49:0/2302481486 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb644097b40 con 0x55b9d8e6c090 2023-10-08T09:16:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.227+0000 7fb640d82700 1 -- 172.21.15.49:0/2302481486 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644099cb0 con 0x55b9d8e6c090 2023-10-08T09:16:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.228+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2302481486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 msgr2=0x55b9d8e655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.228+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e655f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb644097af0 tx=0x7fb6440b2150 comp rx=0 tx=0).stop 2023-10-08T09:16:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.235+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2302481486 shutdown_connections 2023-10-08T09:16:39.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.235+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8e6eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.235+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8e655f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.235+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/2302481486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8ea4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.235+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2302481486 >> 172.21.15.49:0/2302481486 conn(0x55b9d8dd1200 msgr2=0x55b9d8e53c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.235+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2302481486 shutdown_connections 2023-10-08T09:16:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.237+0000 7fb65fe18b80 1 -- 172.21.15.49:0/2302481486 wait complete. 2023-10-08T09:16:39.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.237+0000 7fb65fe18b80 1 -- start start 2023-10-08T09:16:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.237+0000 7fb65fe18b80 1 Processor -- start 2023-10-08T09:16:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.238+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8ee3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.238+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8ee58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.238+0000 7fb65fe18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8ee7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.238+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8ea5000 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.238+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8ee8360 con 0x55b9d8e6c090 2023-10-08T09:16:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.238+0000 7fb65fe18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9d8ee8520 con 0x55b9d8e5e2b0 2023-10-08T09:16:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.238+0000 7fb65fe18b80 10 client.0 _ll_register_callbacks cb 0x7fb644099390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.242+0000 7fb6519cb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8ee3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.242+0000 7fb6519cb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8ee3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50852/0 (socket says 172.21.15.49:50852) 2023-10-08T09:16:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.242+0000 7fb6519cb700 1 -- 172.21.15.49:0/3887617406 learned_addr learned my addr 172.21.15.49:0/3887617406 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.243+0000 7fb6521cc700 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8ee7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.243+0000 7fb6519cb700 1 -- 172.21.15.49:0/3887617406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5e2b0 msgr2=0x55b9d8ee58d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:16:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.243+0000 7fb6519cb700 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8ee58d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.243+0000 7fb6519cb700 1 -- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 msgr2=0x55b9d8ee7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.243+0000 7fb6519cb700 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8ee7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.243+0000 7fb6519cb700 1 -- 172.21.15.49:0/3887617406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9d8bc3c00 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.243+0000 7fb6519cb700 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8ee3380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55b9d8e597c0 tx=0x7fb63c00efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd13ec2119bc717 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.245+0000 7fb6357fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:16:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.245+0000 7fb6357fa700 1 -- 172.21.15.49:0/3887617406 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c026aa0 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.245+0000 7fb6357fa700 1 -- 172.21.15.49:0/3887617406 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb63c014070 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.245+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb63c00ed00 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.255+0000 7fb6357fa700 1 -- 172.21.15.49:0/3887617406 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb63c026c00 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.255+0000 7fb65fe18b80 10 client.16029 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.255+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55b9d8bd9250 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.255+0000 7fb6357fa700 1 -- 172.21.15.49:0/3887617406 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb63c00ed00 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb6357fa700 1 -- 172.21.15.49:0/3887617406 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb63c005280 con 0x55b9d8e5d7a0 2023-10-08T09:16:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb65fe18b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb65fe18b80 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb65fe18b80 20 client.16029 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8ee9f60 0x55b9d8eec380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb65fe18b80 4 client.16029 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:16:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9d8da8ad0 con 0x55b9d8ee9f60 2023-10-08T09:16:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.256+0000 7fb6511ca700 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8ee9f60 0x55b9d8eec380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.257+0000 7fb6511ca700 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8ee9f60 0x55b9d8eec380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce4dce997340958 server_cookie=ef283c26386c9c6f in_seq=0 out_seq=0 2023-10-08T09:16:39.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.257+0000 7fb6357fa700 10 client.16029 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 75.0 s, 1318.3 tps, lat 65.893 ms stddev 145.664 2023-10-08T09:16:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.519+0000 7fb6357fa700 1 -- 172.21.15.49:0/3887617406 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9d8da8ad0 con 0x55b9d8ee9f60 2023-10-08T09:16:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.519+0000 7fb6357fa700 10 client.16029 handle_command_reply: tid=0 2023-10-08T09:16:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.519+0000 7fb65fe18b80 1 client.16029 shutdown 2023-10-08T09:16:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:16:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:16:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:16:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:16:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.522+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5d7a0 msgr2=0x55b9d8ee3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.522+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8ee3380 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55b9d8e597c0 tx=0x7fb63c00efa0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 shutdown_connections 2023-10-08T09:16:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b9d8ee9f60 0x55b9d8eec380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b9d8e6c090 0x55b9d8ee7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b9d8e5d7a0 0x55b9d8ee3380 secure :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55b9d8e597c0 tx=0x7fb63c00efa0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 --2- 172.21.15.49:0/3887617406 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b9d8e5e2b0 0x55b9d8ee58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 >> 172.21.15.49:0/3887617406 conn(0x55b9d8dd1200 msgr2=0x55b9d8e655f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb6357fa700 0 client.16029 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 shutdown_connections 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 1 -- 172.21.15.49:0/3887617406 wait complete. 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.523+0000 7fb65fe18b80 20 client.16029 trim_cache size 0 max 16384 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.524+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6380a1580 msgr2=0x7fb6380a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.524+0000 7fb653c2f700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6380a1580 0x7fb6380a3a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb648006010 tx=0x7fb64802a040 comp rx=0 tx=0).stop 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.524+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c108920 msgr2=0x7fb64c19b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.524+0000 7fb653c2f700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c108920 0x7fb64c19b7e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb644007ee0 tx=0x7fb64400d3b0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.525+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 shutdown_connections 2023-10-08T09:16:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.525+0000 7fb653c2f700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6380a1580 0x7fb6380a3a40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.525+0000 7fb653c2f700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb64c108920 0x7fb64c19b7e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.525+0000 7fb653c2f700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb64c106d90 0x7fb64c196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.525+0000 7fb653c2f700 1 --2- 172.21.15.49:0/4087814969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb64c107f80 0x7fb64c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.525+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 >> 172.21.15.49:0/4087814969 conn(0x7fb64c075d20 msgr2=0x7fb64c06f730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.530+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 shutdown_connections 2023-10-08T09:16:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:39.532+0000 7fb653c2f700 1 -- 172.21.15.49:0/4087814969 wait complete. 2023-10-08T09:16:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:39 smithi049 ceph-mon[99485]: pgmap v381: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-10-08T09:16:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:39 smithi184 ceph-mon[103675]: pgmap v381: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-10-08T09:16:40.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:39 smithi132 ceph-mon[103389]: pgmap v381: 129 pgs: 129 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 MiB/s wr, 1.01k op/s 2023-10-08T09:16:40.641 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:16:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 -- 172.21.15.49:0/94374856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c106dc0 msgr2=0x7f808c1071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 --2- 172.21.15.49:0/94374856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c106dc0 0x7f808c1071e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f807c009990 tx=0x7f807c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 -- 172.21.15.49:0/94374856 shutdown_connections 2023-10-08T09:16:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 --2- 172.21.15.49:0/94374856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c106dc0 0x7f808c1071e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 --2- 172.21.15.49:0/94374856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c108950 0x7f808c10f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 --2- 172.21.15.49:0/94374856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c107fb0 0x7f808c108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.903 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 -- 172.21.15.49:0/94374856 >> 172.21.15.49:0/94374856 conn(0x7f808c075d60 msgr2=0x7f808c0781a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:40.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.901+0000 7f8090fbc700 1 -- 172.21.15.49:0/94374856 shutdown_connections 2023-10-08T09:16:40.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.902+0000 7f8090fbc700 1 -- 172.21.15.49:0/94374856 wait complete. 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.902+0000 7f8090fbc700 1 Processor -- start 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8090fbc700 1 -- start start 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8090fbc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c106dc0 0x7f808c070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8090fbc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c107fb0 0x7f808c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8090fbc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c108950 0x7f808c071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8090fbc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c071bf0 con 0x7f808c106dc0 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8090fbc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c071d60 con 0x7f808c108950 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8090fbc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c071f20 con 0x7f808c107fb0 2023-10-08T09:16:40.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f808a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c106dc0 0x7f808c070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8089d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c107fb0 0x7f808c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f808ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c108950 0x7f808c071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8089d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c107fb0 0x7f808c070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37902/0 (socket says 172.21.15.49:37902) 2023-10-08T09:16:40.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.903+0000 7f8089d9b700 1 -- 172.21.15.49:0/4072735724 learned_addr learned my addr 172.21.15.49:0/4072735724 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.904+0000 7f8089d9b700 1 -- 172.21.15.49:0/4072735724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c108950 msgr2=0x7f808c071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.904+0000 7f8089d9b700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c108950 0x7f808c071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.904+0000 7f8089d9b700 1 -- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c106dc0 msgr2=0x7f808c070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.904+0000 7f8089d9b700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c106dc0 0x7f808c070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.904+0000 7f8089d9b700 1 -- 172.21.15.49:0/4072735724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0095f0 con 0x7f808c107fb0 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.905+0000 7f8089d9b700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c107fb0 0x7f808c070680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f808000d950 tx=0x7f808000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.905+0000 7f807b7fe700 1 -- 172.21.15.49:0/4072735724 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000f890 con 0x7f808c107fb0 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.905+0000 7f807b7fe700 1 -- 172.21.15.49:0/4072735724 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8080010460 con 0x7f808c107fb0 2023-10-08T09:16:40.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.905+0000 7f807b7fe700 1 -- 172.21.15.49:0/4072735724 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808000e610 con 0x7f808c107fb0 2023-10-08T09:16:40.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.907+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c0721a0 con 0x7f808c107fb0 2023-10-08T09:16:40.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.907+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c072630 con 0x7f808c107fb0 2023-10-08T09:16:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.910+0000 7f807b7fe700 1 -- 172.21.15.49:0/4072735724 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f80800108d0 con 0x7f808c107fb0 2023-10-08T09:16:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.910+0000 7f807b7fe700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f80740a1500 0x7f80740a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.910+0000 7f807b7fe700 1 -- 172.21.15.49:0/4072735724 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f80800ccc30 con 0x7f808c107fb0 2023-10-08T09:16:40.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.910+0000 7f808a59c700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f80740a1500 0x7f80740a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.911+0000 7f808a59c700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f80740a1500 0x7f80740a39c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f807c00b680 tx=0x7f807c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:40.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.911+0000 7f809d1a5b80 1 Processor -- start 2023-10-08T09:16:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f809d1a5b80 1 -- start start 2023-10-08T09:16:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59790480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d59791270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59791c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d596f8460 con 0x561d59790060 2023-10-08T09:16:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d596e9c20 con 0x561d59791800 2023-10-08T09:16:40.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d596e7540 con 0x561d597909c0 2023-10-08T09:16:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.912+0000 7f8089d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d59791270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f8089d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d59791270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37908/0 (socket says 172.21.15.49:37908) 2023-10-08T09:16:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f8089d9b700 1 -- 172.21.15.49:0/931955058 learned_addr learned my addr 172.21.15.49:0/931955058 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f808ad9d700 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59791c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f808a59c700 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59790480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f808a59c700 1 -- 172.21.15.49:0/931955058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 msgr2=0x561d59791270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f808a59c700 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d59791270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f808a59c700 1 -- 172.21.15.49:0/931955058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 msgr2=0x561d59791c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.913+0000 7f808a59c700 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59791c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.914+0000 7f808a59c700 1 -- 172.21.15.49:0/931955058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d5950dde0 con 0x561d59790060 2023-10-08T09:16:40.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.914+0000 7f808a59c700 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59790480 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6f10c52bb3c1bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:40.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.914+0000 7f8078ff9700 1 -- 172.21.15.49:0/931955058 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c02b070 con 0x561d59790060 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.914+0000 7f8078ff9700 1 -- 172.21.15.49:0/931955058 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807c01e040 con 0x561d59790060 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.914+0000 7f8078ff9700 1 -- 172.21.15.49:0/931955058 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00f6a0 con 0x561d59790060 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.915+0000 7f809d1a5b80 1 -- 172.21.15.49:0/931955058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 msgr2=0x561d59790480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.915+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59790480 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f807c000c00 tx=0x7f807c020000 comp rx=0 tx=0).stop 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 -- 172.21.15.49:0/931955058 shutdown_connections 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59791c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59790480 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/931955058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d59791270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 -- 172.21.15.49:0/931955058 >> 172.21.15.49:0/931955058 conn(0x561d59704260 msgr2=0x561d5978ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:40.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 -- 172.21.15.49:0/931955058 shutdown_connections 2023-10-08T09:16:40.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 -- 172.21.15.49:0/931955058 wait complete. 2023-10-08T09:16:40.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.916+0000 7f809d1a5b80 1 -- start start 2023-10-08T09:16:40.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 1 Processor -- start 2023-10-08T09:16:40.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59799e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d5979a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d597a1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d596e2e30 con 0x561d59790060 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5967f710 con 0x561d59791800 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5967f3b0 con 0x561d597909c0 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f809d1a5b80 10 client.0 _ll_register_callbacks cb 0x561d594dc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f808ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d597a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f808ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d597a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40494/0 (socket says 172.21.15.49:40494) 2023-10-08T09:16:40.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f808ad9d700 1 -- 172.21.15.49:0/725310346 learned_addr learned my addr 172.21.15.49:0/725310346 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f8089d9b700 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d5979a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.917+0000 7f808a59c700 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59799e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8089d9b700 1 -- 172.21.15.49:0/725310346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 msgr2=0x561d597a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8089d9b700 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d597a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8089d9b700 1 -- 172.21.15.49:0/725310346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 msgr2=0x561d59799e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8089d9b700 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59799e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8089d9b700 1 -- 172.21.15.49:0/725310346 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c10fb50 con 0x561d597909c0 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8089d9b700 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d5979a340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80800998c0 tx=0x7f8080099610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5aa4eaf298154c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8071ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8071ffb700 1 -- 172.21.15.49:0/725310346 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800b1070 con 0x561d597909c0 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8071ffb700 1 -- 172.21.15.49:0/725310346 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f808009dd10 con 0x561d597909c0 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d596aaeb0 con 0x561d597909c0 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f8071ffb700 1 -- 172.21.15.49:0/725310346 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80800a1f50 con 0x561d597909c0 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f809d1a5b80 10 client.25693 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:40.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.918+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561d594f6c60 con 0x561d597909c0 2023-10-08T09:16:40.924 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.919+0000 7f8071ffb700 1 -- 172.21.15.49:0/725310346 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808007c080 con 0x561d597909c0 2023-10-08T09:16:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f8071ffb700 1 -- 172.21.15.49:0/725310346 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f80800b2320 con 0x561d597909c0 2023-10-08T09:16:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f809d1a5b80 10 client.25693 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f809d1a5b80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:40.925 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f809d1a5b80 20 client.25693 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d597a5cb0 0x561d597a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f809d1a5b80 4 client.25693 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d596dbe10 con 0x561d597a5cb0 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.921+0000 7f808a59c700 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d597a5cb0 0x561d597a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.922+0000 7f808a59c700 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d597a5cb0 0x561d597a80d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc87c56a99537cc server_cookie=89561676c0637a2b in_seq=0 out_seq=0 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.922+0000 7f8071ffb700 10 client.25693 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.922+0000 7f8071ffb700 1 -- 172.21.15.49:0/725310346 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561d596dbe10 con 0x561d597a5cb0 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.922+0000 7f8071ffb700 10 client.25693 handle_command_reply: tid=0 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.922+0000 7f809d1a5b80 1 client.25693 shutdown 2023-10-08T09:16:40.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 msgr2=0x561d5979a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d5979a340 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80800998c0 tx=0x7f8080099610 comp rx=0 tx=0).stop 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 shutdown_connections 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d597a5cb0 0x561d597a80d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d597a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59790060 0x561d59799e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/725310346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d597909c0 0x561d5979a340 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 >> 172.21.15.49:0/725310346 conn(0x561d59704260 msgr2=0x561d5978a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f8071ffb700 0 client.25693 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 shutdown_connections 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 1 -- 172.21.15.49:0/725310346 wait complete. 2023-10-08T09:16:40.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.923+0000 7f809d1a5b80 20 client.25693 trim_cache size 0 max 16384 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 Processor -- start 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 -- start start 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d597a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d597909c0 0x561d597a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59798fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59563920 con 0x561d597909c0 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d597a2500 con 0x561d59791800 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.943+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d59799970 con 0x561d59790060 2023-10-08T09:16:40.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d597a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59798fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59798fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40504/0 (socket says 172.21.15.49:40504) 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 -- 172.21.15.49:0/4037855805 learned_addr learned my addr 172.21.15.49:0/4037855805 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f8089d9b700 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d597909c0 0x561d597a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 -- 172.21.15.49:0/4037855805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 msgr2=0x561d597a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d597a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 -- 172.21.15.49:0/4037855805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d597909c0 msgr2=0x561d597a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d597909c0 0x561d597a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.944+0000 7f808ad9d700 1 -- 172.21.15.49:0/4037855805 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d59534250 con 0x561d59791800 2023-10-08T09:16:40.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.945+0000 7f808ad9d700 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59798fc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8084039890 tx=0x7f8084009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d429a658a6af2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:40.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.945+0000 7f80797fa700 1 -- 172.21.15.49:0/4037855805 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400e040 con 0x561d59791800 2023-10-08T09:16:40.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.945+0000 7f80797fa700 1 -- 172.21.15.49:0/4037855805 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f808400f040 con 0x561d59791800 2023-10-08T09:16:40.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.945+0000 7f80797fa700 1 -- 172.21.15.49:0/4037855805 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084004680 con 0x561d59791800 2023-10-08T09:16:40.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.946+0000 7f809d1a5b80 1 -- 172.21.15.49:0/4037855805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 msgr2=0x561d59798fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.946+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59798fc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8084039890 tx=0x7f8084009920 comp rx=0 tx=0).stop 2023-10-08T09:16:40.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.946+0000 7f809d1a5b80 1 -- 172.21.15.49:0/4037855805 shutdown_connections 2023-10-08T09:16:40.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.946+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d597a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.946+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d59791800 0x561d59798fc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.946+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/4037855805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d597909c0 0x561d597a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.946+0000 7f809d1a5b80 1 -- 172.21.15.49:0/4037855805 >> 172.21.15.49:0/4037855805 conn(0x561d59704260 msgr2=0x561d59797d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.947+0000 7f809d1a5b80 1 -- 172.21.15.49:0/4037855805 shutdown_connections 2023-10-08T09:16:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.947+0000 7f809d1a5b80 1 -- 172.21.15.49:0/4037855805 wait complete. 2023-10-08T09:16:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 -- start start 2023-10-08T09:16:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 Processor -- start 2023-10-08T09:16:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d5979f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d597909c0 0x561d5979ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59791800 0x561d597a0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d597a1640 con 0x561d59791800 2023-10-08T09:16:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d597f9ad0 con 0x561d597909c0 2023-10-08T09:16:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5980ac20 con 0x561d59790060 2023-10-08T09:16:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.948+0000 7f809d1a5b80 10 client.0 _ll_register_callbacks cb 0x561d594dc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d597909c0 0x561d5979ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d597909c0 0x561d5979ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40510/0 (socket says 172.21.15.49:40510) 2023-10-08T09:16:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 -- 172.21.15.49:0/2630394758 learned_addr learned my addr 172.21.15.49:0/2630394758 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:40.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f808ad9d700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59791800 0x561d597a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f808a59c700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d5979f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 -- 172.21.15.49:0/2630394758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 msgr2=0x561d5979f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d5979f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 -- 172.21.15.49:0/2630394758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59791800 msgr2=0x561d597a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59791800 0x561d597a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f8089d9b700 1 -- 172.21.15.49:0/2630394758 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8084039540 con 0x561d597909c0 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.949+0000 7f808ad9d700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59791800 0x561d597a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.950+0000 7f8089d9b700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d597909c0 0x561d5979ff00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80800ca3a0 tx=0x7f808009a3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=607192897ed2636b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.950+0000 7f8071ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.951+0000 7f8071ffb700 1 -- 172.21.15.49:0/2630394758 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800b1070 con 0x561d597909c0 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.951+0000 7f8071ffb700 1 -- 172.21.15.49:0/2630394758 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f80800a1ac0 con 0x561d597909c0 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.951+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f807c01e040 con 0x561d597909c0 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.951+0000 7f8071ffb700 1 -- 172.21.15.49:0/2630394758 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80800b2480 con 0x561d597909c0 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.951+0000 7f809d1a5b80 10 client.25682 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:40.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.951+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561d5950c2b0 con 0x561d597909c0 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.951+0000 7f8071ffb700 1 -- 172.21.15.49:0/2630394758 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807c01e040 con 0x561d597909c0 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f8071ffb700 1 -- 172.21.15.49:0/2630394758 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8080078ad0 con 0x561d597909c0 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f809d1a5b80 10 client.25682 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f809d1a5b80 10 client.25682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f809d1a5b80 20 client.25682 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d5980c2f0 0x561d5980e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f809d1a5b80 4 client.25682 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561d596dbb30 con 0x561d5980c2f0 2023-10-08T09:16:40.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.952+0000 7f808a59c700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d5980c2f0 0x561d5980e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f8071ffb700 10 client.25682 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f808a59c700 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d5980c2f0 0x561d5980e710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8297032186435dae server_cookie=3a0ea8cd489eb2c1 in_seq=0 out_seq=0 2023-10-08T09:16:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f8071ffb700 1 -- 172.21.15.49:0/2630394758 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561d596dbb30 con 0x561d5980c2f0 2023-10-08T09:16:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f8071ffb700 10 client.25682 handle_command_reply: tid=0 2023-10-08T09:16:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 client.25682 shutdown 2023-10-08T09:16:40.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d597909c0 msgr2=0x561d5979ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d597909c0 0x561d5979ff00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f80800ca3a0 tx=0x7f808009a3e0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 shutdown_connections 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d5980c2f0 0x561d5980e710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d59790060 0x561d5979f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f8071ffb700 0 client.25682 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d59791800 0x561d597a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 --2- 172.21.15.49:0/2630394758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d597909c0 0x561d5979ff00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 >> 172.21.15.49:0/2630394758 conn(0x561d59704260 msgr2=0x561d59798fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 shutdown_connections 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.954+0000 7f809d1a5b80 1 -- 172.21.15.49:0/2630394758 wait complete. 2023-10-08T09:16:40.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f809d1a5b80 20 client.25682 trim_cache size 0 max 16384 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:16:40.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:16:40.961 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:16:40.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f80740a1500 msgr2=0x7f80740a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f80740a1500 0x7f80740a39c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f807c00b680 tx=0x7f807c005fd0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c107fb0 msgr2=0x7f808c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c107fb0 0x7f808c070680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f808000d950 tx=0x7f808000dc60 comp rx=0 tx=0).stop 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 shutdown_connections 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f80740a1500 0x7f80740a39c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c108950 0x7f808c071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f808c106dc0 0x7f808c070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 --2- 172.21.15.49:0/4072735724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f808c107fb0 0x7f808c070680 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 >> 172.21.15.49:0/4072735724 conn(0x7f808c075d60 msgr2=0x7f808c10b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 shutdown_connections 2023-10-08T09:16:40.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:16:40.955+0000 7f8090fbc700 1 -- 172.21.15.49:0/4072735724 wait complete. 2023-10-08T09:16:40.978 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:16:41.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:41 smithi049 ceph-mon[99485]: pgmap v382: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.18k op/s 2023-10-08T09:16:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:41 smithi184 ceph-mon[103675]: pgmap v382: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.18k op/s 2023-10-08T09:16:42.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:41 smithi132 ceph-mon[103389]: pgmap v382: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 1.18k op/s 2023-10-08T09:16:43.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:43 smithi049 ceph-mon[99485]: pgmap v383: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2023-10-08T09:16:43.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:43 smithi184 ceph-mon[103675]: pgmap v383: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2023-10-08T09:16:44.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:43 smithi132 ceph-mon[103389]: pgmap v383: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2023-10-08T09:16:44.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 80.0 s, 1822.1 tps, lat 62.366 ms stddev 135.745 2023-10-08T09:16:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:45 smithi049 ceph-mon[99485]: pgmap v384: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.85k op/s 2023-10-08T09:16:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:45 smithi184 ceph-mon[103675]: pgmap v384: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.85k op/s 2023-10-08T09:16:46.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:45 smithi132 ceph-mon[103389]: pgmap v384: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 1.85k op/s 2023-10-08T09:16:47.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:47 smithi049 ceph-mon[99485]: pgmap v385: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.91k op/s 2023-10-08T09:16:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:47 smithi184 ceph-mon[103675]: pgmap v385: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.91k op/s 2023-10-08T09:16:48.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:47 smithi132 ceph-mon[103389]: pgmap v385: 129 pgs: 129 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 37 MiB/s wr, 1.91k op/s 2023-10-08T09:16:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 85.0 s, 2090.4 tps, lat 48.394 ms stddev 52.622 2023-10-08T09:16:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:49 smithi049 ceph-mon[99485]: pgmap v386: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.87k op/s 2023-10-08T09:16:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:49 smithi184 ceph-mon[103675]: pgmap v386: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.87k op/s 2023-10-08T09:16:50.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:49 smithi132 ceph-mon[103389]: pgmap v386: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.87k op/s 2023-10-08T09:16:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:51 smithi049 ceph-mon[99485]: pgmap v387: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.74k op/s 2023-10-08T09:16:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:51 smithi184 ceph-mon[103675]: pgmap v387: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.74k op/s 2023-10-08T09:16:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:51 smithi132 ceph-mon[103389]: pgmap v387: 129 pgs: 129 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 MiB/s wr, 1.74k op/s 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 -- 172.21.15.49:0/2143389067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c107f80 msgr2=0x7fd10c1083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 --2- 172.21.15.49:0/2143389067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c107f80 0x7fd10c1083e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009920 tx=0x7fd0f4009c30 comp rx=0 tx=0).stop 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 -- 172.21.15.49:0/2143389067 shutdown_connections 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 --2- 172.21.15.49:0/2143389067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c108920 0x7fd10c10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 --2- 172.21.15.49:0/2143389067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c106d90 0x7fd10c1071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 --2- 172.21.15.49:0/2143389067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c107f80 0x7fd10c1083e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 -- 172.21.15.49:0/2143389067 >> 172.21.15.49:0/2143389067 conn(0x7fd10c075d20 msgr2=0x7fd10c078160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 -- 172.21.15.49:0/2143389067 shutdown_connections 2023-10-08T09:16:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.715+0000 7fd111582700 1 -- 172.21.15.49:0/2143389067 wait complete. 2023-10-08T09:16:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 Processor -- start 2023-10-08T09:16:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 -- start start 2023-10-08T09:16:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c106d90 0x7fd10c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c107f80 0x7fd10c19d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c108920 0x7fd10c1a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c1a4890 con 0x7fd10c106d90 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c079070 con 0x7fd10c108920 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.716+0000 7fd111582700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c079230 con 0x7fd10c107f80 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c106d90 0x7fd10c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c106d90 0x7fd10c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50908/0 (socket says 172.21.15.49:50908) 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 -- 172.21.15.49:0/2155630010 learned_addr learned my addr 172.21.15.49:0/2155630010 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd1027fc700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c107f80 0x7fd10c19d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c108920 0x7fd10c1a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 -- 172.21.15.49:0/2155630010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c107f80 msgr2=0x7fd10c19d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c107f80 0x7fd10c19d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 -- 172.21.15.49:0/2155630010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c108920 msgr2=0x7fd10c1a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c108920 0x7fd10c1a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.717+0000 7fd10affd700 1 -- 172.21.15.49:0/2155630010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009520 con 0x7fd10c106d90 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.718+0000 7fd10affd700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c106d90 0x7fd10c19cda0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00d950 tx=0x7fd0fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.718+0000 7fd108ff9700 1 -- 172.21.15.49:0/2155630010 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00f890 con 0x7fd10c106d90 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.718+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c079510 con 0x7fd10c106d90 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.718+0000 7fd108ff9700 1 -- 172.21.15.49:0/2155630010 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0fc010460 con 0x7fd10c106d90 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.718+0000 7fd108ff9700 1 -- 172.21.15.49:0/2155630010 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00e610 con 0x7fd10c106d90 2023-10-08T09:16:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.718+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c079a70 con 0x7fd10c106d90 2023-10-08T09:16:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.722+0000 7fd108ff9700 1 -- 172.21.15.49:0/2155630010 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd0fc0105d0 con 0x7fd10c106d90 2023-10-08T09:16:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.723+0000 7fd11d76bb80 1 Processor -- start 2023-10-08T09:16:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.723+0000 7fd11d76bb80 1 -- start start 2023-10-08T09:16:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc697a0 0x559fdcc69cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc6a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcbd1400 con 0x559fdcc697a0 2023-10-08T09:16:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcbc2bc0 con 0x559fdcc68e40 2023-10-08T09:16:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcbc04e0 con 0x559fdcc6a2b0 2023-10-08T09:16:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd1027fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc6a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd10affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc697a0 0x559fdcc69cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd1027fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc6a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37952/0 (socket says 172.21.15.49:37952) 2023-10-08T09:16:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.724+0000 7fd1027fc700 1 -- 172.21.15.49:0/1977434467 learned_addr learned my addr 172.21.15.49:0/1977434467 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.725+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.725+0000 7fd108ff9700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f80a14d0 0x7fd0f80a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd108ff9700 1 -- 172.21.15.49:0/2155630010 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd0fc0cd080 con 0x7fd10c106d90 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd1027fc700 1 -- 172.21.15.49:0/1977434467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 msgr2=0x559fdcc69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd1027fc700 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd1027fc700 1 -- 172.21.15.49:0/1977434467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc697a0 msgr2=0x559fdcc69cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd1027fc700 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc697a0 0x559fdcc69cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd1027fc700 1 -- 172.21.15.49:0/1977434467 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fdc9e6d80 con 0x559fdcc6a2b0 2023-10-08T09:16:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:16:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd10affd700 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc697a0 0x559fdcc69cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:16:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f80a14d0 0x7fd0f80a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd1027fc700 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc6a730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400bfd0 tx=0x7fd0f400fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cced44b8a6d55b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.726+0000 7fd1017fa700 1 -- 172.21.15.49:0/1977434467 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f401d070 con 0x559fdcc6a2b0 2023-10-08T09:16:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.727+0000 7fd1017fa700 1 -- 172.21.15.49:0/1977434467 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f400f920 con 0x559fdcc6a2b0 2023-10-08T09:16:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.727+0000 7fd1017fa700 1 -- 172.21.15.49:0/1977434467 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400b990 con 0x559fdcc6a2b0 2023-10-08T09:16:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.727+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f80a14d0 0x7fd0f80a3990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd104009690 tx=0x7fd10400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.727+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1977434467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 msgr2=0x559fdcc6a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.727+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc6a730 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400bfd0 tx=0x7fd0f400fe60 comp rx=0 tx=0).stop 2023-10-08T09:16:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.728+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1977434467 shutdown_connections 2023-10-08T09:16:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.728+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc6a730 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.728+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.728+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1977434467 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc697a0 0x559fdcc69cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.728+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1977434467 >> 172.21.15.49:0/1977434467 conn(0x559fdcbdd200 msgr2=0x559fdcc638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.728+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1977434467 shutdown_connections 2023-10-08T09:16:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.728+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1977434467 wait complete. 2023-10-08T09:16:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 -- start start 2023-10-08T09:16:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 Processor -- start 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc72740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 0x559fdcc72c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcbbbdd0 con 0x559fdcc6a2b0 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcb586b0 con 0x559fdcc68e40 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcb58350 con 0x559fdcc697a0 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd11d76bb80 10 client.0 _ll_register_callbacks cb 0x559fdc9b57a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:52.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd10affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc72740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd10affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc72740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40536/0 (socket says 172.21.15.49:40536) 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.729+0000 7fd10affd700 1 -- 172.21.15.49:0/478929590 learned_addr learned my addr 172.21.15.49:0/478929590 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.730+0000 7fd1027fc700 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 0x559fdcc72c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.730+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd10affd700 1 -- 172.21.15.49:0/478929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 msgr2=0x559fdcc72c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd10affd700 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 0x559fdcc72c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd10affd700 1 -- 172.21.15.49:0/478929590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 msgr2=0x559fdcc79cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd10affd700 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc79cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd10affd700 1 -- 172.21.15.49:0/478929590 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c10fb20 con 0x559fdcc68e40 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd10affd700 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc72740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc096a80 tx=0x7fd0fc09a840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb4f43aa54a9238 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd0f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd0f27fc700 1 -- 172.21.15.49:0/478929590 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0a23a0 con 0x559fdcc68e40 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fdcb83e50 con 0x559fdcc68e40 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd0f27fc700 1 -- 172.21.15.49:0/478929590 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd0fc0108b0 con 0x559fdcc68e40 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.731+0000 7fd0f27fc700 1 -- 172.21.15.49:0/478929590 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0fc0cf0d0 con 0x559fdcc68e40 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 10 client.25694 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559fdc9cfc00 con 0x559fdcc68e40 2023-10-08T09:16:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd0f27fc700 1 -- 172.21.15.49:0/478929590 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559fdcb83e50 con 0x559fdcc68e40 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd0f27fc700 1 -- 172.21.15.49:0/478929590 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd0fc099620 con 0x559fdcc68e40 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 10 client.25694 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 10 client.25694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 20 client.25694 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcc7e4e0 0x559fdcc80900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 4 client.25694 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.732+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fdcbb4db0 con 0x559fdcc7e4e0 2023-10-08T09:16:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.733+0000 7fd1027fc700 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcc7e4e0 0x559fdcc80900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.734+0000 7fd0f27fc700 10 client.25694 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:52.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.734+0000 7fd1027fc700 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcc7e4e0 0x559fdcc80900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f65de56b4392c02 server_cookie=1753a3008a96b939 in_seq=0 out_seq=0 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd0f27fc700 1 -- 172.21.15.49:0/478929590 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559fdcbb4db0 con 0x559fdcc7e4e0 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd0f27fc700 10 client.25694 handle_command_reply: tid=0 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 client.25694 shutdown 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 msgr2=0x559fdcc72740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc72740 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc096a80 tx=0x7fd0fc09a840 comp rx=0 tx=0).stop 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 shutdown_connections 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcc7e4e0 0x559fdcc80900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd0f27fc700 0 client.25694 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc72740 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/478929590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 0x559fdcc72c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 >> 172.21.15.49:0/478929590 conn(0x559fdcbdd200 msgr2=0x559fdcc63010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 shutdown_connections 2023-10-08T09:16:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 1 -- 172.21.15.49:0/478929590 wait complete. 2023-10-08T09:16:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.735+0000 7fd11d76bb80 20 client.25694 trim_cache size 0 max 16384 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.763+0000 7fd11d76bb80 1 Processor -- start 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.764+0000 7fd11d76bb80 1 -- start start 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.764+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc7a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.764+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 0x559fdcc79c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.764+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc71c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.764+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdca3c8c0 con 0x559fdcc6a2b0 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.764+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcbb4db0 con 0x559fdcc68e40 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.764+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcc73790 con 0x559fdcc697a0 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc71c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc71c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50932/0 (socket says 172.21.15.49:50932) 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 -- 172.21.15.49:0/1346586560 learned_addr learned my addr 172.21.15.49:0/1346586560 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 -- 172.21.15.49:0/1346586560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 msgr2=0x559fdcc79c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/1346586560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 0x559fdcc79c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 -- 172.21.15.49:0/1346586560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 msgr2=0x559fdcc7a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/1346586560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc7a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 -- 172.21.15.49:0/1346586560 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fdca0d1f0 con 0x559fdcc6a2b0 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.765+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/1346586560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc71c50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd1040048a0 tx=0x7fd104015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e71a121724e1fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.766+0000 7fd101ffb700 1 -- 172.21.15.49:0/1346586560 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104017b10 con 0x559fdcc6a2b0 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.766+0000 7fd101ffb700 1 -- 172.21.15.49:0/1346586560 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd10400f040 con 0x559fdcc6a2b0 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.766+0000 7fd101ffb700 1 -- 172.21.15.49:0/1346586560 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104008460 con 0x559fdcc6a2b0 2023-10-08T09:16:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.767+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1346586560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 msgr2=0x559fdcc71c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.767+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1346586560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc71c50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd1040048a0 tx=0x7fd104015000 comp rx=0 tx=0).stop 2023-10-08T09:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.768+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1346586560 shutdown_connections 2023-10-08T09:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.768+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1346586560 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc6a2b0 0x559fdcc71c50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.768+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1346586560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc68e40 0x559fdcc7a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.768+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/1346586560 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc697a0 0x559fdcc79c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.768+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1346586560 >> 172.21.15.49:0/1346586560 conn(0x559fdcbdd200 msgr2=0x559fdcc85260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.770+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1346586560 shutdown_connections 2023-10-08T09:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.772+0000 7fd11d76bb80 1 -- 172.21.15.49:0/1346586560 wait complete. 2023-10-08T09:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 -- start start 2023-10-08T09:16:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 Processor -- start 2023-10-08T09:16:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc68e40 0x559fdcc73e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc74410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc79160 0x559fdcc795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcc79b20 con 0x559fdcc68e40 2023-10-08T09:16:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcc79c90 con 0x559fdcc79160 2023-10-08T09:16:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fdcc79e50 con 0x559fdcc6a2b0 2023-10-08T09:16:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.773+0000 7fd11d76bb80 10 client.0 _ll_register_callbacks cb 0x7fd0fc099c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:16:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc79160 0x559fdcc795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc79160 0x559fdcc795e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40548/0 (socket says 172.21.15.49:40548) 2023-10-08T09:16:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 -- 172.21.15.49:0/2192775801 learned_addr learned my addr 172.21.15.49:0/2192775801 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 -- 172.21.15.49:0/2192775801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 msgr2=0x559fdcc74410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc74410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 -- 172.21.15.49:0/2192775801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc68e40 msgr2=0x559fdcc73e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc68e40 0x559fdcc73e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.774+0000 7fd10b7fe700 1 -- 172.21.15.49:0/2192775801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd104004550 con 0x559fdcc79160 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd10b7fe700 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc79160 0x559fdcc795e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd104004870 tx=0x7fd1040044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98108b646c505ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd0f27fc700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd0f27fc700 1 -- 172.21.15.49:0/2192775801 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104017b10 con 0x559fdcc79160 2023-10-08T09:16:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0f400f920 con 0x559fdcc79160 2023-10-08T09:16:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd0f27fc700 1 -- 172.21.15.49:0/2192775801 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd10401c040 con 0x559fdcc79160 2023-10-08T09:16:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd0f27fc700 1 -- 172.21.15.49:0/2192775801 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd104017c70 con 0x559fdcc79160 2023-10-08T09:16:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd11d76bb80 10 client.25697 fetch_fsmap learned FSMap version 33 2023-10-08T09:16:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.775+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559fdc9e5250 con 0x559fdcc79160 2023-10-08T09:16:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.776+0000 7fd0f27fc700 1 -- 172.21.15.49:0/2192775801 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f400f920 con 0x559fdcc79160 2023-10-08T09:16:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.776+0000 7fd0f27fc700 1 -- 172.21.15.49:0/2192775801 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd10402e4e0 con 0x559fdcc79160 2023-10-08T09:16:52.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.776+0000 7fd11d76bb80 10 client.25697 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:16:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.776+0000 7fd11d76bb80 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:16:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.776+0000 7fd11d76bb80 20 client.25697 populate_metadata read hostname 'smithi049' 2023-10-08T09:16:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.776+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcce4a30 0x559fdcce6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:16:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.777+0000 7fd11d76bb80 4 client.25697 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:16:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.777+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559fdcbb4ad0 con 0x559fdcce4a30 2023-10-08T09:16:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.779+0000 7fd10affd700 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcce4a30 0x559fdcce6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:16:52.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.779+0000 7fd10affd700 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcce4a30 0x559fdcce6ef0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b20f8a8d6ca60e server_cookie=259cfb8f341b8e37 in_seq=0 out_seq=0 2023-10-08T09:16:52.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.779+0000 7fd0f27fc700 10 client.25697 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.809+0000 7fd0f27fc700 1 -- 172.21.15.49:0/2192775801 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559fdcbb4ad0 con 0x559fdcce4a30 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.809+0000 7fd0f27fc700 10 client.25697 handle_command_reply: tid=0 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.809+0000 7fd11d76bb80 1 client.25697 shutdown 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc79160 msgr2=0x559fdcc795e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc79160 0x559fdcc795e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd104004870 tx=0x7fd1040044b0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 shutdown_connections 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559fdcce4a30 0x559fdcce6ef0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559fdcc79160 0x559fdcc795e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd0f27fc700 0 client.25697 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:16:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559fdcc68e40 0x559fdcc73e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 --2- 172.21.15.49:0/2192775801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559fdcc6a2b0 0x559fdcc74410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 >> 172.21.15.49:0/2192775801 conn(0x559fdcbdd200 msgr2=0x559fdcc6d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 shutdown_connections 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 1 -- 172.21.15.49:0/2192775801 wait complete. 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.810+0000 7fd11d76bb80 20 client.25697 trim_cache size 0 max 16384 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:16:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:16:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f80a14d0 msgr2=0x7fd0f80a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f80a14d0 0x7fd0f80a3990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd104009690 tx=0x7fd10400b040 comp rx=0 tx=0).stop 2023-10-08T09:16:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c106d90 msgr2=0x7fd10c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:16:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c106d90 0x7fd10c19cda0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00d950 tx=0x7fd0fc00dc60 comp rx=0 tx=0).stop 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 shutdown_connections 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f80a14d0 0x7fd0f80a3990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd10c108920 0x7fd10c1a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd10c106d90 0x7fd10c19cda0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 --2- 172.21.15.49:0/2155630010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd10c107f80 0x7fd10c19d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 >> 172.21.15.49:0/2155630010 conn(0x7fd10c075d20 msgr2=0x7fd10c10b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 shutdown_connections 2023-10-08T09:16:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:16:52.811+0000 7fd111582700 1 -- 172.21.15.49:0/2155630010 wait complete. 2023-10-08T09:16:53.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:53 smithi049 ceph-mon[99485]: pgmap v388: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 MiB/s wr, 1.46k op/s 2023-10-08T09:16:53.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:53 smithi184 ceph-mon[103675]: pgmap v388: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 MiB/s wr, 1.46k op/s 2023-10-08T09:16:54.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:53 smithi132 ceph-mon[103389]: pgmap v388: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 MiB/s wr, 1.46k op/s 2023-10-08T09:16:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 90.0 s, 1991.6 tps, lat 50.135 ms stddev 58.587 2023-10-08T09:16:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:55 smithi184 ceph-mon[103675]: pgmap v389: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 984 op/s 2023-10-08T09:16:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:55 smithi132 ceph-mon[103389]: pgmap v389: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 984 op/s 2023-10-08T09:16:56.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:55 smithi049 ceph-mon[99485]: pgmap v389: 129 pgs: 129 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 19 MiB/s wr, 984 op/s 2023-10-08T09:16:57.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:57 smithi184 ceph-mon[103675]: pgmap v390: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 743 op/s 2023-10-08T09:16:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:57 smithi132 ceph-mon[103389]: pgmap v390: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 743 op/s 2023-10-08T09:16:58.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:57 smithi049 ceph-mon[99485]: pgmap v390: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 743 op/s 2023-10-08T09:16:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 95.0 s, 2165.1 tps, lat 45.540 ms stddev 55.693 2023-10-08T09:16:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:16:59 smithi184 ceph-mon[103675]: pgmap v391: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 734 op/s 2023-10-08T09:17:00.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:16:59 smithi132 ceph-mon[103389]: pgmap v391: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 734 op/s 2023-10-08T09:17:00.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:16:59 smithi049 ceph-mon[99485]: pgmap v391: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 734 op/s 2023-10-08T09:17:01.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:01 smithi184 ceph-mon[103675]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-10-08T09:17:02.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:01 smithi132 ceph-mon[103389]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-10-08T09:17:02.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:01 smithi049 ceph-mon[99485]: pgmap v392: 129 pgs: 129 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 12 MiB/s wr, 713 op/s 2023-10-08T09:17:03.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:03 smithi184 ceph-mon[103675]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-08T09:17:04.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:03 smithi132 ceph-mon[103389]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-08T09:17:04.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:03 smithi049 ceph-mon[99485]: pgmap v393: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 747 op/s 2023-10-08T09:17:04.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 100.0 s, 1906.8 tps, lat 52.918 ms stddev 62.314 2023-10-08T09:17:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.009+0000 7f2afbea3700 1 -- 172.21.15.49:0/36098981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 msgr2=0x7f2af410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.009+0000 7f2afbea3700 1 --2- 172.21.15.49:0/36098981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 0x7f2af410c5f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009990 tx=0x7f2ae4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.012+0000 7f2afbea3700 1 -- 172.21.15.49:0/36098981 shutdown_connections 2023-10-08T09:17:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.012+0000 7f2afbea3700 1 --2- 172.21.15.49:0/36098981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af410cb30 0x7f2af410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.012+0000 7f2afbea3700 1 --2- 172.21.15.49:0/36098981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af41077f0 0x7f2af4109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.012+0000 7f2afbea3700 1 --2- 172.21.15.49:0/36098981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 0x7f2af410c5f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.012+0000 7f2afbea3700 1 -- 172.21.15.49:0/36098981 >> 172.21.15.49:0/36098981 conn(0x7f2af406d770 msgr2=0x7f2af406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.013+0000 7f2afbea3700 1 -- 172.21.15.49:0/36098981 shutdown_connections 2023-10-08T09:17:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.016+0000 7f2afbea3700 1 -- 172.21.15.49:0/36098981 wait complete. 2023-10-08T09:17:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.016+0000 7f2afbea3700 1 Processor -- start 2023-10-08T09:17:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.017+0000 7f2afbea3700 1 -- start start 2023-10-08T09:17:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.017+0000 7f2afbea3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af41077f0 0x7f2af41a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.017+0000 7f2afbea3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 0x7f2af41a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.017+0000 7f2afbea3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af410cb30 0x7f2af41acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.017+0000 7f2afbea3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41ad150 con 0x7f2af410cb30 2023-10-08T09:17:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.017+0000 7f2afbea3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41ad290 con 0x7f2af410a160 2023-10-08T09:17:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.017+0000 7f2afbea3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41ad420 con 0x7f2af41077f0 2023-10-08T09:17:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af410cb30 0x7f2af41acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2af943e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 0x7f2af41a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af410cb30 0x7f2af41acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50944/0 (socket says 172.21.15.49:50944) 2023-10-08T09:17:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 -- 172.21.15.49:0/3416523804 learned_addr learned my addr 172.21.15.49:0/3416523804 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 -- 172.21.15.49:0/3416523804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af41077f0 msgr2=0x7f2af41a5620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:17:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af41077f0 0x7f2af41a5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 -- 172.21.15.49:0/3416523804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 msgr2=0x7f2af41a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 0x7f2af41a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.018+0000 7f2afa440700 1 -- 172.21.15.49:0/3416523804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae40095f0 con 0x7f2af410cb30 2023-10-08T09:17:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.021+0000 7f2afa440700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af410cb30 0x7f2af41acb80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b970 tx=0x7f2aec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.021+0000 7f2aeaffd700 1 -- 172.21.15.49:0/3416523804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00c930 con 0x7f2af410cb30 2023-10-08T09:17:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.021+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af41ad700 con 0x7f2af410cb30 2023-10-08T09:17:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.021+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af41adce0 con 0x7f2af410cb30 2023-10-08T09:17:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.021+0000 7f2aeaffd700 1 -- 172.21.15.49:0/3416523804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aec0098b0 con 0x7f2af410cb30 2023-10-08T09:17:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.021+0000 7f2aeaffd700 1 -- 172.21.15.49:0/3416523804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec01d3b0 con 0x7f2af410cb30 2023-10-08T09:17:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.024+0000 7f2aeaffd700 1 -- 172.21.15.49:0/3416523804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2aec01d530 con 0x7f2af410cb30 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.025+0000 7f2b0808cb80 1 Processor -- start 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.025+0000 7f2aeaffd700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ae00a1480 0x7f2ae00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2aeaffd700 1 -- 172.21.15.49:0/3416523804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2aec0cc630 con 0x7f2af410cb30 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2af943e700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ae00a1480 0x7f2ae00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2b0808cb80 1 -- start start 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 0x562351130260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x562351130cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x562351131730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562351098400 con 0x5623511307a0 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562351089bc0 con 0x56235112fe40 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623510874e0 con 0x5623511312b0 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2af943e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x562351130cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2afa440700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x562351131730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2af943e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x562351130cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50948/0 (socket says 172.21.15.49:50948) 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2af943e700 1 -- 172.21.15.49:0/2766745679 learned_addr learned my addr 172.21.15.49:0/2766745679 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.026+0000 7f2afa440700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x562351131730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37984/0 (socket says 172.21.15.49:37984) 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2af943e700 1 -- 172.21.15.49:0/2766745679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 msgr2=0x562351131730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2af943e700 1 --2- 172.21.15.49:0/2766745679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x562351131730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2af943e700 1 -- 172.21.15.49:0/2766745679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 msgr2=0x562351130260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2af943e700 1 --2- 172.21.15.49:0/2766745679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 0x562351130260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2af943e700 1 -- 172.21.15.49:0/2766745679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562350eadd80 con 0x5623511307a0 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2af943e700 1 --2- 172.21.15.49:0/2766745679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x562351130cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4027000 tx=0x7f2ae4028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4326ac84b376f747 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2af943e700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ae00a1480 0x7f2ae00a3940 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4003790 tx=0x7f2ae400bf20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:06.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.027+0000 7f2adffff700 1 -- 172.21.15.49:0/2766745679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40050c0 con 0x5623511307a0 2023-10-08T09:17:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.028+0000 7f2adffff700 1 -- 172.21.15.49:0/2766745679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae401e040 con 0x5623511307a0 2023-10-08T09:17:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.028+0000 7f2adffff700 1 -- 172.21.15.49:0/2766745679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4026d20 con 0x5623511307a0 2023-10-08T09:17:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.028+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2766745679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 msgr2=0x562351130cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.028+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2766745679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x562351130cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4027000 tx=0x7f2ae4028000 comp rx=0 tx=0).stop 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.030+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2766745679 shutdown_connections 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.030+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2766745679 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x562351131730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.031+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2766745679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 0x562351130260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.031+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2766745679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x562351130cd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.031+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2766745679 >> 172.21.15.49:0/2766745679 conn(0x5623510a4200 msgr2=0x56235112a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.031+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2766745679 shutdown_connections 2023-10-08T09:17:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.031+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2766745679 wait complete. 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.031+0000 7f2b0808cb80 1 -- start start 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.031+0000 7f2b0808cb80 1 Processor -- start 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56235112fe40 0x5623511396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511307a0 0x562351139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511312b0 0x562351140c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562351082dd0 con 0x56235112fe40 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235101f6b0 con 0x5623511312b0 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235101f350 con 0x5623511307a0 2023-10-08T09:17:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2b0808cb80 10 client.0 _ll_register_callbacks cb 0x562350e7c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2af9c3f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56235112fe40 0x5623511396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2af943e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511307a0 0x562351139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2af9c3f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56235112fe40 0x5623511396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50954/0 (socket says 172.21.15.49:50954) 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2af9c3f700 1 -- 172.21.15.49:0/4082049526 learned_addr learned my addr 172.21.15.49:0/4082049526 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.032+0000 7f2afa440700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511312b0 0x562351140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2af9c3f700 1 -- 172.21.15.49:0/4082049526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511307a0 msgr2=0x562351139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511307a0 0x562351139bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2af9c3f700 1 -- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511312b0 msgr2=0x562351140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511312b0 0x562351140c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2af9c3f700 1 -- 172.21.15.49:0/4082049526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af410da70 con 0x56235112fe40 2023-10-08T09:17:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2afa440700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511312b0 0x562351140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2af943e700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511307a0 0x562351139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56235112fe40 0x5623511396b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009ec0 tx=0x7f2af000b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbe8793ee77b1c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.033+0000 7f2add7fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.036+0000 7f2add7fa700 1 -- 172.21.15.49:0/4082049526 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000be70 con 0x56235112fe40 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.036+0000 7f2add7fa700 1 -- 172.21.15.49:0/4082049526 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2af0011420 con 0x56235112fe40 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.036+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56235104ae50 con 0x56235112fe40 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.036+0000 7f2add7fa700 1 -- 172.21.15.49:0/4082049526 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2af00115e0 con 0x56235112fe40 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.036+0000 7f2b0808cb80 10 client.16068 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.036+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562350e96c00 con 0x56235112fe40 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2add7fa700 1 -- 172.21.15.49:0/4082049526 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56235104ae50 con 0x56235112fe40 2023-10-08T09:17:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2add7fa700 1 -- 172.21.15.49:0/4082049526 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2af0019850 con 0x56235112fe40 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2b0808cb80 10 client.16068 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2b0808cb80 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2b0808cb80 20 client.16068 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351145720 0x562351147b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2b0808cb80 4 client.16068 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56235107bdb0 con 0x562351145720 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.037+0000 7f2af943e700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351145720 0x562351147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.038+0000 7f2af943e700 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351145720 0x562351147b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4fc6062f6494c74 server_cookie=b71b2f5fd71ea80 in_seq=0 out_seq=0 2023-10-08T09:17:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.038+0000 7f2add7fa700 10 client.16068 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.040+0000 7f2add7fa700 1 -- 172.21.15.49:0/4082049526 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56235107bdb0 con 0x562351145720 2023-10-08T09:17:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.040+0000 7f2add7fa700 10 client.16068 handle_command_reply: tid=0 2023-10-08T09:17:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.040+0000 7f2b0808cb80 1 client.16068 shutdown 2023-10-08T09:17:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.041+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56235112fe40 msgr2=0x5623511396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.041+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56235112fe40 0x5623511396b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009ec0 tx=0x7f2af000b3c0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 shutdown_connections 2023-10-08T09:17:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351145720 0x562351147b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511312b0 0x562351140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2add7fa700 0 client.16068 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56235112fe40 0x5623511396b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/4082049526 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511307a0 0x562351139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 >> 172.21.15.49:0/4082049526 conn(0x5623510a4200 msgr2=0x56235112a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 shutdown_connections 2023-10-08T09:17:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 1 -- 172.21.15.49:0/4082049526 wait complete. 2023-10-08T09:17:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.042+0000 7f2b0808cb80 20 client.16068 trim_cache size 0 max 16384 2023-10-08T09:17:06.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:05 smithi132 ceph-mon[103389]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 819 op/s 2023-10-08T09:17:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 Processor -- start 2023-10-08T09:17:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 -- start start 2023-10-08T09:17:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56235112fe40 0x562351147b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511307a0 0x562351140b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511312b0 0x562351138bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562350f038c0 con 0x5623511312b0 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235113a700 con 0x5623511307a0 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623511386c0 con 0x56235112fe40 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2af9c3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56235112fe40 0x562351147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2af9c3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56235112fe40 0x562351147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:37996/0 (socket says 172.21.15.49:37996) 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2af943e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511307a0 0x562351140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2af9c3f700 1 -- 172.21.15.49:0/519795842 learned_addr learned my addr 172.21.15.49:0/519795842 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.065+0000 7f2afa440700 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511312b0 0x562351138bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2af9c3f700 1 -- 172.21.15.49:0/519795842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511307a0 msgr2=0x562351140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511307a0 0x562351140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2af9c3f700 1 -- 172.21.15.49:0/519795842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511312b0 msgr2=0x562351138bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511312b0 0x562351138bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2af9c3f700 1 -- 172.21.15.49:0/519795842 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562350ed41f0 con 0x56235112fe40 2023-10-08T09:17:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56235112fe40 0x562351147b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2af0000c00 tx=0x7f2af0006c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc20225f5308066 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2ae8ff9700 1 -- 172.21.15.49:0/519795842 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000b790 con 0x56235112fe40 2023-10-08T09:17:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2ae8ff9700 1 -- 172.21.15.49:0/519795842 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2af0009180 con 0x56235112fe40 2023-10-08T09:17:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.066+0000 7f2ae8ff9700 1 -- 172.21.15.49:0/519795842 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0011840 con 0x56235112fe40 2023-10-08T09:17:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 -- 172.21.15.49:0/519795842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56235112fe40 msgr2=0x562351147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56235112fe40 0x562351147b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2af0000c00 tx=0x7f2af0006c70 comp rx=0 tx=0).stop 2023-10-08T09:17:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 -- 172.21.15.49:0/519795842 shutdown_connections 2023-10-08T09:17:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56235112fe40 0x562351147b10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511312b0 0x562351138bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/519795842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5623511307a0 0x562351140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 -- 172.21.15.49:0/519795842 >> 172.21.15.49:0/519795842 conn(0x5623510a4200 msgr2=0x56235114c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 -- 172.21.15.49:0/519795842 shutdown_connections 2023-10-08T09:17:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 -- 172.21.15.49:0/519795842 wait complete. 2023-10-08T09:17:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 -- start start 2023-10-08T09:17:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.067+0000 7f2b0808cb80 1 Processor -- start 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.068+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 0x56235113c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.068+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x56235113c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.068+0000 7f2b0808cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x56235113f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.068+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235113fdb0 con 0x5623511307a0 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.068+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235113ff20 con 0x56235112fe40 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.068+0000 7f2b0808cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623511400e0 con 0x5623511312b0 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.068+0000 7f2b0808cb80 10 client.0 _ll_register_callbacks cb 0x7f2af0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2afa440700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x56235113f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x56235113c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x56235113c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50970/0 (socket says 172.21.15.49:50970) 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 -- 172.21.15.49:0/2440436186 learned_addr learned my addr 172.21.15.49:0/2440436186 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 -- 172.21.15.49:0/2440436186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 msgr2=0x56235113f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x56235113f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 -- 172.21.15.49:0/2440436186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 msgr2=0x56235113c310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 0x56235113c310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.069+0000 7f2af943e700 1 -- 172.21.15.49:0/2440436186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562350e96c00 con 0x5623511307a0 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.070+0000 7f2af943e700 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x56235113c850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009990 tx=0x7f2ae4005120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a768ead31481164 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.070+0000 7f2add7fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.070+0000 7f2add7fa700 1 -- 172.21.15.49:0/2440436186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4042040 con 0x5623511307a0 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.070+0000 7f2add7fa700 1 -- 172.21.15.49:0/2440436186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2ae40265f0 con 0x5623511307a0 2023-10-08T09:17:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ae401e040 con 0x5623511307a0 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2add7fa700 1 -- 172.21.15.49:0/2440436186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ae40267b0 con 0x5623511307a0 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2b0808cb80 10 client.16074 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562350eac250 con 0x5623511307a0 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2add7fa700 1 -- 172.21.15.49:0/2440436186 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae401e040 con 0x5623511307a0 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2add7fa700 1 -- 172.21.15.49:0/2440436186 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2ae400b980 con 0x5623511307a0 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2b0808cb80 10 client.16074 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.071+0000 7f2b0808cb80 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.072+0000 7f2b0808cb80 20 client.16074 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.072+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351199bc0 0x56235119bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.072+0000 7f2b0808cb80 4 client.16074 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:17:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.072+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56235107bad0 con 0x562351199bc0 2023-10-08T09:17:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.072+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351199bc0 0x56235119bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.072+0000 7f2af9c3f700 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351199bc0 0x56235119bfe0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5948f7e525f37b06 server_cookie=3e79832bbd40321a in_seq=0 out_seq=0 2023-10-08T09:17:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.072+0000 7f2add7fa700 10 client.16074 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.090+0000 7f2add7fa700 1 -- 172.21.15.49:0/2440436186 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56235107bad0 con 0x562351199bc0 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.090+0000 7f2add7fa700 10 client.16074 handle_command_reply: tid=0 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 client.16074 shutdown 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 msgr2=0x56235113c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x56235113c850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009990 tx=0x7f2ae4005120 comp rx=0 tx=0).stop 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 shutdown_connections 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562351199bc0 0x56235119bfe0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5623511312b0 0x56235113f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56235112fe40 0x56235113c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 --2- 172.21.15.49:0/2440436186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5623511307a0 0x56235113c850 secure :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009990 tx=0x7f2ae4005120 comp rx=0 tx=0).stop 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 >> 172.21.15.49:0/2440436186 conn(0x5623510a4200 msgr2=0x562351138bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2add7fa700 0 client.16074 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 shutdown_connections 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 1 -- 172.21.15.49:0/2440436186 wait complete. 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.091+0000 7f2b0808cb80 20 client.16074 trim_cache size 0 max 16384 2023-10-08T09:17:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:17:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:17:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:17:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:17:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ae00a1480 msgr2=0x7f2ae00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ae00a1480 0x7f2ae00a3940 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4003790 tx=0x7f2ae400bf20 comp rx=0 tx=0).stop 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af410cb30 msgr2=0x7f2af41acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af410cb30 0x7f2af41acb80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b970 tx=0x7f2aec00bc80 comp rx=0 tx=0).stop 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 shutdown_connections 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2ae00a1480 0x7f2ae00a3940 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af41077f0 0x7f2af41a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2af410cb30 0x7f2af41acb80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 --2- 172.21.15.49:0/3416523804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2af410a160 0x7f2af41a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 >> 172.21.15.49:0/3416523804 conn(0x7f2af406d770 msgr2=0x7f2af4108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 shutdown_connections 2023-10-08T09:17:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:06.092+0000 7f2afbea3700 1 -- 172.21.15.49:0/3416523804 wait complete. 2023-10-08T09:17:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:05 smithi049 ceph-mon[99485]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 819 op/s 2023-10-08T09:17:06.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:05 smithi184 ceph-mon[103675]: pgmap v394: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 819 op/s 2023-10-08T09:17:08.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:07 smithi132 ceph-mon[103389]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 827 op/s 2023-10-08T09:17:08.139 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:07 smithi049 ceph-mon[99485]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 827 op/s 2023-10-08T09:17:08.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:07 smithi184 ceph-mon[103675]: pgmap v395: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 827 op/s 2023-10-08T09:17:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 105.0 s, 1850.0 tps, lat 54.014 ms stddev 64.033 2023-10-08T09:17:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:09 smithi049 ceph-mon[99485]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 791 op/s 2023-10-08T09:17:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:09 smithi184 ceph-mon[103675]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 791 op/s 2023-10-08T09:17:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:09 smithi132 ceph-mon[103389]: pgmap v396: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 12 MiB/s wr, 791 op/s 2023-10-08T09:17:10.979 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:17:11.093 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:10 smithi049 ceph-mon[99485]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-08T09:17:11.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:10 smithi184 ceph-mon[103675]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-08T09:17:11.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.281+0000 7fdc07b19700 1 -- 172.21.15.49:0/768021975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 msgr2=0x7fdc00100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.281+0000 7fdc07b19700 1 --2- 172.21.15.49:0/768021975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 0x7fdc00100410 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009990 tx=0x7fdbf0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.284+0000 7fdc07b19700 1 -- 172.21.15.49:0/768021975 shutdown_connections 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.284+0000 7fdc07b19700 1 --2- 172.21.15.49:0/768021975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 0x7fdc000ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.284+0000 7fdc07b19700 1 --2- 172.21.15.49:0/768021975 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00100950 0x7fdc00107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.284+0000 7fdc07b19700 1 --2- 172.21.15.49:0/768021975 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 0x7fdc00100410 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.284+0000 7fdc07b19700 1 -- 172.21.15.49:0/768021975 >> 172.21.15.49:0/768021975 conn(0x7fdc000f9f50 msgr2=0x7fdc000fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.284+0000 7fdc07b19700 1 -- 172.21.15.49:0/768021975 shutdown_connections 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.284+0000 7fdc07b19700 1 -- 172.21.15.49:0/768021975 wait complete. 2023-10-08T09:17:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 Processor -- start 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 -- start start 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 0x7fdc0019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 0x7fdc0019d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00100950 0x7fdc001a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001a4830 con 0x7fdc000fff90 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001a49a0 con 0x7fdc00100950 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc07b19700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001a4b60 con 0x7fdc000fef80 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.285+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 0x7fdc0019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 0x7fdc0019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38016/0 (socket says 172.21.15.49:38016) 2023-10-08T09:17:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 -- 172.21.15.49:0/2779415600 learned_addr learned my addr 172.21.15.49:0/2779415600 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc050b4700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 0x7fdc0019d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 -- 172.21.15.49:0/2779415600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00100950 msgr2=0x7fdc001a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00100950 0x7fdc001a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 -- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 msgr2=0x7fdc0019d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 0x7fdc0019d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 -- 172.21.15.49:0/2779415600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf00095f0 con 0x7fdc000fef80 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc058b5700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 0x7fdc0019cd10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e970 tx=0x7fdbfc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdbf6ffd700 1 -- 172.21.15.49:0/2779415600 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc00cbd0 con 0x7fdc000fef80 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc001a4e40 con 0x7fdc000fef80 2023-10-08T09:17:11.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.286+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc001a5310 con 0x7fdc000fef80 2023-10-08T09:17:11.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.287+0000 7fdbf6ffd700 1 -- 172.21.15.49:0/2779415600 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbfc00eec0 con 0x7fdc000fef80 2023-10-08T09:17:11.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.287+0000 7fdbf6ffd700 1 -- 172.21.15.49:0/2779415600 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc003cf0 con 0x7fdc000fef80 2023-10-08T09:17:11.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.290+0000 7fdbf6ffd700 1 -- 172.21.15.49:0/2779415600 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdbfc004020 con 0x7fdc000fef80 2023-10-08T09:17:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.291+0000 7fdc13d02b80 1 Processor -- start 2023-10-08T09:17:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.292+0000 7fdc13d02b80 1 -- start start 2023-10-08T09:17:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b618480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b619270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b619800 0x55f91b619c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b580460 con 0x55f91b618060 2023-10-08T09:17:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b571c20 con 0x55f91b619800 2023-10-08T09:17:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b56f540 con 0x55f91b6189c0 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdbf6ffd700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbec0a1480 0x7fdbec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b619270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b619270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38022/0 (socket says 172.21.15.49:38022) 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc058b5700 1 -- 172.21.15.49:0/968793686 learned_addr learned my addr 172.21.15.49:0/968793686 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc050b4700 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b619800 0x55f91b619c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdc060b6700 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b618480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.293+0000 7fdbf6ffd700 1 -- 172.21.15.49:0/2779415600 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdbfc014070 con 0x7fdc000fef80 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc058b5700 1 -- 172.21.15.49:0/968793686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b619800 msgr2=0x55f91b619c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:11.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc058b5700 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b619800 0x55f91b619c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc058b5700 1 -- 172.21.15.49:0/968793686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 msgr2=0x55f91b618480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc058b5700 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b618480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc058b5700 1 -- 172.21.15.49:0/968793686 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91b395de0 con 0x55f91b6189c0 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc050b4700 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b619800 0x55f91b619c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc060b6700 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b618480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc060b6700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbec0a1480 0x7fdbec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc058b5700 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b619270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc09a890 tx=0x7fdbfc096a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff572c436add2e10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdbebfff700 1 -- 172.21.15.49:0/968793686 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc099d70 con 0x55f91b6189c0 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdc060b6700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbec0a1480 0x7fdbec0a3940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009fd0 tx=0x7fdbf8009b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdbebfff700 1 -- 172.21.15.49:0/968793686 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbfc099b90 con 0x55f91b6189c0 2023-10-08T09:17:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.294+0000 7fdbebfff700 1 -- 172.21.15.49:0/968793686 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0a2630 con 0x55f91b6189c0 2023-10-08T09:17:11.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:10 smithi132 ceph-mon[103389]: pgmap v397: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 700 op/s 2023-10-08T09:17:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 -- 172.21.15.49:0/968793686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 msgr2=0x55f91b619270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b619270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc09a890 tx=0x7fdbfc096a70 comp rx=0 tx=0).stop 2023-10-08T09:17:11.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 -- 172.21.15.49:0/968793686 shutdown_connections 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b619800 0x55f91b619c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b618480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/968793686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b619270 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 -- 172.21.15.49:0/968793686 >> 172.21.15.49:0/968793686 conn(0x55f91b58c260 msgr2=0x55f91b612e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 -- 172.21.15.49:0/968793686 shutdown_connections 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.295+0000 7fdc13d02b80 1 -- 172.21.15.49:0/968793686 wait complete. 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.296+0000 7fdc13d02b80 1 -- start start 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.296+0000 7fdc13d02b80 1 Processor -- start 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b618060 0x55f91b621e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 0x55f91b622360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b6293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b56ae30 con 0x55f91b619800 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b507710 con 0x55f91b6189c0 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b5073b0 con 0x55f91b618060 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc13d02b80 10 client.0 _ll_register_callbacks cb 0x55f91b364800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b618060 0x55f91b621e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b618060 0x55f91b621e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38030/0 (socket says 172.21.15.49:38030) 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 -- 172.21.15.49:0/1539556623 learned_addr learned my addr 172.21.15.49:0/1539556623 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc060b6700 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b6293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc050b4700 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 0x55f91b622360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 -- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 msgr2=0x55f91b622360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 0x55f91b622360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 -- 172.21.15.49:0/1539556623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 msgr2=0x55f91b6293a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b6293a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 -- 172.21.15.49:0/1539556623 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc000fe6d0 con 0x55f91b618060 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdc058b5700 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b618060 0x55f91b621e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc096ca0 tx=0x7fdbfc009810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56f8138815e0b21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdbe97fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.297+0000 7fdbe97fa700 1 -- 172.21.15.49:0/1539556623 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc09a930 con 0x55f91b618060 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdbe97fa700 1 -- 172.21.15.49:0/1539556623 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdbfc09a230 con 0x55f91b618060 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f91b532eb0 con 0x55f91b618060 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdbe97fa700 1 -- 172.21.15.49:0/1539556623 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbfc00cd30 con 0x55f91b618060 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 10 client.25732 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fdc001a5310 con 0x55f91b618060 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdbe97fa700 1 -- 172.21.15.49:0/1539556623 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f91b532eb0 con 0x55f91b618060 2023-10-08T09:17:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdbe97fa700 1 -- 172.21.15.49:0/1539556623 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdbfc099840 con 0x55f91b618060 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 10 client.25732 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 20 client.25732 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b62d7f0 0x55f91b62fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 4 client.25732 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f91b563e10 con 0x55f91b62d7f0 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.298+0000 7fdc050b4700 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b62d7f0 0x55f91b62fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.299+0000 7fdbe97fa700 10 client.25732 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.299+0000 7fdc050b4700 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b62d7f0 0x55f91b62fc10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990fc3e34fce72cf server_cookie=6d82359ac91346bb in_seq=0 out_seq=0 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdbe97fa700 1 -- 172.21.15.49:0/1539556623 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f91b563e10 con 0x55f91b62d7f0 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdbe97fa700 10 client.25732 handle_command_reply: tid=0 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 client.25732 shutdown 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b618060 msgr2=0x55f91b621e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b618060 0x55f91b621e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc096ca0 tx=0x7fdbfc009810 comp rx=0 tx=0).stop 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 shutdown_connections 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b62d7f0 0x55f91b62fc10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b618060 0x55f91b621e20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdbe97fa700 0 client.25732 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b6293a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/1539556623 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 0x55f91b622360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 >> 172.21.15.49:0/1539556623 conn(0x55f91b58c260 msgr2=0x55f91b6124d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 shutdown_connections 2023-10-08T09:17:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 1 -- 172.21.15.49:0/1539556623 wait complete. 2023-10-08T09:17:11.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.301+0000 7fdc13d02b80 20 client.25732 trim_cache size 0 max 16384 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 Processor -- start 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 -- start start 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b618060 0x55f91b62fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b629360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b60ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b3eb920 con 0x55f91b619800 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b614910 con 0x55f91b618060 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b619060 con 0x55f91b6189c0 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc060b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b60ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc050b4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b629360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc060b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b60ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51006/0 (socket says 172.21.15.49:51006) 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc050b4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b629360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38038/0 (socket says 172.21.15.49:38038) 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc060b6700 1 -- 172.21.15.49:0/4064319689 learned_addr learned my addr 172.21.15.49:0/4064319689 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.330+0000 7fdc058b5700 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b618060 0x55f91b62fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdc050b4700 1 -- 172.21.15.49:0/4064319689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b618060 msgr2=0x55f91b62fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdc050b4700 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b618060 0x55f91b62fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdc050b4700 1 -- 172.21.15.49:0/4064319689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 msgr2=0x55f91b60ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdc050b4700 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b60ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdc050b4700 1 -- 172.21.15.49:0/4064319689 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91b37ec60 con 0x55f91b6189c0 2023-10-08T09:17:11.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdc060b6700 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b60ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdc050b4700 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b629360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000b680 tx=0x7fdbf000bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1167a659160060 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdbf4d82700 1 -- 172.21.15.49:0/4064319689 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf001d040 con 0x55f91b6189c0 2023-10-08T09:17:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdbf4d82700 1 -- 172.21.15.49:0/4064319689 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf001e040 con 0x55f91b6189c0 2023-10-08T09:17:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.331+0000 7fdbf4d82700 1 -- 172.21.15.49:0/4064319689 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00274a0 con 0x55f91b6189c0 2023-10-08T09:17:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.332+0000 7fdc13d02b80 1 -- 172.21.15.49:0/4064319689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 msgr2=0x55f91b629360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.332+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b629360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000b680 tx=0x7fdbf000bdf0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.332+0000 7fdc13d02b80 1 -- 172.21.15.49:0/4064319689 shutdown_connections 2023-10-08T09:17:11.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.332+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b619800 0x55f91b60ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.332+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b618060 0x55f91b62fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.332+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/4064319689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b6189c0 0x55f91b629360 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.332+0000 7fdc13d02b80 1 -- 172.21.15.49:0/4064319689 >> 172.21.15.49:0/4064319689 conn(0x55f91b58c260 msgr2=0x55f91b61fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:11.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.335+0000 7fdc13d02b80 1 -- 172.21.15.49:0/4064319689 shutdown_connections 2023-10-08T09:17:11.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.335+0000 7fdc13d02b80 1 -- 172.21.15.49:0/4064319689 wait complete. 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.335+0000 7fdc13d02b80 1 -- start start 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.335+0000 7fdc13d02b80 1 Processor -- start 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b628bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 0x55f91b629130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc13d02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b619800 0x55f91b627200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b627820 con 0x55f91b618060 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b627960 con 0x55f91b6189c0 2023-10-08T09:17:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc13d02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91b627b20 con 0x55f91b619800 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc13d02b80 10 client.0 _ll_register_callbacks cb 0x7fdbf80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b628bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b628bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51008/0 (socket says 172.21.15.49:51008) 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 -- 172.21.15.49:0/3911050895 learned_addr learned my addr 172.21.15.49:0/3911050895 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc060b6700 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b619800 0x55f91b627200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 -- 172.21.15.49:0/3911050895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b619800 msgr2=0x55f91b627200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b619800 0x55f91b627200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 -- 172.21.15.49:0/3911050895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 msgr2=0x55f91b629130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 0x55f91b629130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.336+0000 7fdc058b5700 1 -- 172.21.15.49:0/3911050895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf8011040 con 0x55f91b618060 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc058b5700 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b628bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009230 tx=0x7fdbfc005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4385ed6d0d5e324 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:11.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdbe97fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdbe97fa700 1 -- 172.21.15.49:0/3911050895 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc09a690 con 0x55f91b618060 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdbe97fa700 1 -- 172.21.15.49:0/3911050895 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdbfc094e40 con 0x55f91b618060 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbfc099b90 con 0x55f91b618060 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdbe97fa700 1 -- 172.21.15.49:0/3911050895 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbfc0a0720 con 0x55f91b618060 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 10 client.16089 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fdbf000bbb0 con 0x55f91b618060 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdbe97fa700 1 -- 172.21.15.49:0/3911050895 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbfc07a020 con 0x55f91b618060 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdbe97fa700 1 -- 172.21.15.49:0/3911050895 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdbfc099190 con 0x55f91b618060 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 20 client.16089 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b682670 0x55f91b684ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 4 client.16089 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:17:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.337+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f91b563b30 con 0x55f91b682670 2023-10-08T09:17:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.338+0000 7fdc050b4700 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b682670 0x55f91b684ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.338+0000 7fdc050b4700 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b682670 0x55f91b684ad0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea56127fc85c59dc server_cookie=155eba5426bb2fda in_seq=0 out_seq=0 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.338+0000 7fdbe97fa700 10 client.16089 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.339+0000 7fdbe97fa700 1 -- 172.21.15.49:0/3911050895 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f91b563b30 con 0x55f91b682670 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.339+0000 7fdbe97fa700 10 client.16089 handle_command_reply: tid=0 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "scrub active (2 inodes in the stack)", 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:17:11.343 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:17:11.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:17:11.344 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:17:11.344 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:17:11.344 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:17:11.344 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.339+0000 7fdc13d02b80 1 client.16089 shutdown 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 msgr2=0x55f91b628bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b628bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009230 tx=0x7fdbfc005b40 comp rx=0 tx=0).stop 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 shutdown_connections 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f91b682670 0x55f91b684ad0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f91b619800 0x55f91b627200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f91b618060 0x55f91b628bf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 --2- 172.21.15.49:0/3911050895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f91b6189c0 0x55f91b629130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 >> 172.21.15.49:0/3911050895 conn(0x55f91b58c260 msgr2=0x55f91b61c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdbe97fa700 0 client.16089 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 shutdown_connections 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 1 -- 172.21.15.49:0/3911050895 wait complete. 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.341+0000 7fdc13d02b80 20 client.16089 trim_cache size 0 max 16384 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbec0a1480 msgr2=0x7fdbec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbec0a1480 0x7fdbec0a3940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009fd0 tx=0x7fdbf8009b60 comp rx=0 tx=0).stop 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 msgr2=0x7fdc0019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 0x7fdc0019cd10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00e970 tx=0x7fdbfc00ec80 comp rx=0 tx=0).stop 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 shutdown_connections 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbec0a1480 0x7fdbec0a3940 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc000fef80 0x7fdc0019cd10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc00100950 0x7fdc001a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.342+0000 7fdc07b19700 1 --2- 172.21.15.49:0/2779415600 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc000fff90 0x7fdc0019d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:11.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.343+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 >> 172.21.15.49:0/2779415600 conn(0x7fdc000f9f50 msgr2=0x7fdc0010cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:11.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.343+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 shutdown_connections 2023-10-08T09:17:11.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:11.343+0000 7fdc07b19700 1 -- 172.21.15.49:0/2779415600 wait complete. 2023-10-08T09:17:11.381 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:17:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:13 smithi132 ceph-mon[103389]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 703 op/s 2023-10-08T09:17:13.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:13 smithi049 ceph-mon[99485]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 703 op/s 2023-10-08T09:17:13.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:13 smithi184 ceph-mon[103675]: pgmap v398: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 703 op/s 2023-10-08T09:17:14.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 110.0 s, 1211.6 tps, lat 81.591 ms stddev 105.079 2023-10-08T09:17:15.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:15 smithi132 ceph-mon[103389]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 751 op/s 2023-10-08T09:17:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:15 smithi049 ceph-mon[99485]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 751 op/s 2023-10-08T09:17:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:15 smithi184 ceph-mon[103675]: pgmap v399: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 751 op/s 2023-10-08T09:17:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:17 smithi132 ceph-mon[103389]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 744 op/s 2023-10-08T09:17:17.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:17 smithi049 ceph-mon[99485]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 744 op/s 2023-10-08T09:17:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:17 smithi184 ceph-mon[103675]: pgmap v400: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 744 op/s 2023-10-08T09:17:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 -- 172.21.15.49:0/436164157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1981063f0 msgr2=0x7fb198108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 --2- 172.21.15.49:0/436164157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1981063f0 0x7fb198108890 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb19000b0a0 tx=0x7fb19000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 -- 172.21.15.49:0/436164157 shutdown_connections 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 --2- 172.21.15.49:0/436164157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1981063f0 0x7fb198108890 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 --2- 172.21.15.49:0/436164157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1980fe4c0 0x7fb1980fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 --2- 172.21.15.49:0/436164157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1980fee20 0x7fb198105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 -- 172.21.15.49:0/436164157 >> 172.21.15.49:0/436164157 conn(0x7fb1980f9ef0 msgr2=0x7fb1980fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 -- 172.21.15.49:0/436164157 shutdown_connections 2023-10-08T09:17:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.238+0000 7fb19eef9700 1 -- 172.21.15.49:0/436164157 wait complete. 2023-10-08T09:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 Processor -- start 2023-10-08T09:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 -- start start 2023-10-08T09:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1980fe4c0 0x7fb198194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1980fee20 0x7fb198195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1981063f0 0x7fb19819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19819c0c0 con 0x7fb1981063f0 2023-10-08T09:17:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19819c230 con 0x7fb1980fee20 2023-10-08T09:17:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.239+0000 7fb19eef9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19819c3f0 con 0x7fb1980fe4c0 2023-10-08T09:17:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.245+0000 7fb197fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1980fee20 0x7fb198195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.245+0000 7fb197fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1980fee20 0x7fb198195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40638/0 (socket says 172.21.15.49:40638) 2023-10-08T09:17:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.245+0000 7fb197fff700 1 -- 172.21.15.49:0/1340684214 learned_addr learned my addr 172.21.15.49:0/1340684214 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.246+0000 7fb19cc95700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1980fe4c0 0x7fb198194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.249+0000 7fb19cc95700 1 -- 172.21.15.49:0/1340684214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1980fee20 msgr2=0x7fb198195260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.249+0000 7fb19cc95700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1980fee20 0x7fb198195260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.249+0000 7fb19cc95700 1 -- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1981063f0 msgr2=0x7fb19819bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.249+0000 7fb19cc95700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1981063f0 0x7fb19819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.249+0000 7fb19cc95700 1 -- 172.21.15.49:0/1340684214 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb190009d00 con 0x7fb1980fe4c0 2023-10-08T09:17:19.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.250+0000 7fb19cc95700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1980fe4c0 0x7fb198194d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00e9f0 tx=0x7fb18c00edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.259+0000 7fb195ffb700 1 -- 172.21.15.49:0/1340684214 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00cbd0 con 0x7fb1980fe4c0 2023-10-08T09:17:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.259+0000 7fb195ffb700 1 -- 172.21.15.49:0/1340684214 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c00cd30 con 0x7fb1980fe4c0 2023-10-08T09:17:19.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.259+0000 7fb195ffb700 1 -- 172.21.15.49:0/1340684214 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c018680 con 0x7fb1980fe4c0 2023-10-08T09:17:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.260+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19819c6d0 con 0x7fb1980fe4c0 2023-10-08T09:17:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.260+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19819cb60 con 0x7fb1980fe4c0 2023-10-08T09:17:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.264+0000 7fb1ab0e2b80 1 Processor -- start 2023-10-08T09:17:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb1ab0e2b80 1 -- start start 2023-10-08T09:17:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b6a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b6acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1ad2400 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1ac3bc0 con 0x55cfc1b69e40 2023-10-08T09:17:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1ac14e0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb19cc95700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b6acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb19cc95700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b6acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38062/0 (socket says 172.21.15.49:38062) 2023-10-08T09:17:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.265+0000 7fb19cc95700 1 -- 172.21.15.49:0/2167770353 learned_addr learned my addr 172.21.15.49:0/2167770353 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.266+0000 7fb19d496700 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b6a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.270+0000 7fb19cc95700 1 -- 172.21.15.49:0/2167770353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 msgr2=0x55cfc1b6a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.270+0000 7fb19cc95700 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b6a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.270+0000 7fb19cc95700 1 -- 172.21.15.49:0/2167770353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 msgr2=0x55cfc1b6b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.270+0000 7fb19cc95700 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.270+0000 7fb19cc95700 1 -- 172.21.15.49:0/2167770353 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfc18e7d80 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.271+0000 7fb195ffb700 1 -- 172.21.15.49:0/1340684214 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb18c005520 con 0x7fb1980fe4c0 2023-10-08T09:17:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.272+0000 7fb195ffb700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb1880a1650 0x7fb1880a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.273+0000 7fb195ffb700 1 -- 172.21.15.49:0/1340684214 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb18c014070 con 0x7fb1980fe4c0 2023-10-08T09:17:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.273+0000 7fb19d496700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb1880a1650 0x7fb1880a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.274+0000 7fb19d496700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb1880a1650 0x7fb1880a3b10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb1900062a0 tx=0x7fb1900061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.277+0000 7fb19cc95700 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b6acd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00cf50 tx=0x7fb18c0108a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d888cb7d17af37bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.279+0000 7fb182ffd700 1 -- 172.21.15.49:0/2167770353 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c097bd0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.279+0000 7fb182ffd700 1 -- 172.21.15.49:0/2167770353 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c09b3d0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.279+0000 7fb182ffd700 1 -- 172.21.15.49:0/2167770353 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0ce3c0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.280+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/2167770353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 msgr2=0x55cfc1b6acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.280+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b6acd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00cf50 tx=0x7fb18c0108a0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.282+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/2167770353 shutdown_connections 2023-10-08T09:17:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.282+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.282+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b6a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.282+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/2167770353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b6acd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.282+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/2167770353 >> 172.21.15.49:0/2167770353 conn(0x55cfc1ade200 msgr2=0x55cfc1b648a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.284+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/2167770353 shutdown_connections 2023-10-08T09:17:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.289+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/2167770353 wait complete. 2023-10-08T09:17:19.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.290+0000 7fb1ab0e2b80 1 -- start start 2023-10-08T09:17:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.290+0000 7fb1ab0e2b80 1 Processor -- start 2023-10-08T09:17:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.294+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b69e40 0x55cfc1b737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.294+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.294+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b7ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.294+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1abcdd0 con 0x55cfc1b69e40 2023-10-08T09:17:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.294+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1a596b0 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.294+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1a59350 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.294+0000 7fb1ab0e2b80 10 client.0 _ll_register_callbacks cb 0x55cfc18b67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b73d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38068/0 (socket says 172.21.15.49:38068) 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 -- 172.21.15.49:0/3151115917 learned_addr learned my addr 172.21.15.49:0/3151115917 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 -- 172.21.15.49:0/3151115917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b6b2b0 msgr2=0x55cfc1b7ad40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b7ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 -- 172.21.15.49:0/3151115917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b69e40 msgr2=0x55cfc1b737c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b69e40 0x55cfc1b737c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.295+0000 7fb197fff700 1 -- 172.21.15.49:0/3151115917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19819cb60 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.298+0000 7fb197fff700 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b73d00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb184011690 tx=0x7fb184011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4c39eaefaecd44 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.298+0000 7fb177fff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.300+0000 7fb177fff700 1 -- 172.21.15.49:0/3151115917 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184012860 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.300+0000 7fb177fff700 1 -- 172.21.15.49:0/3151115917 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb1840129c0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.300+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfc1a84e50 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.301+0000 7fb177fff700 1 -- 172.21.15.49:0/3151115917 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb184012b80 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 10 client.25750 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:19.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cfc18d0c00 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb177fff700 1 -- 172.21.15.49:0/3151115917 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cfc1a84e50 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb177fff700 1 -- 172.21.15.49:0/3151115917 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb18401a3f0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 10 client.25750 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 20 client.25750 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1b90820 0x55cfc1b92c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 4 client.25750 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:17:19.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.302+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfc1ab5db0 con 0x55cfc1b90820 2023-10-08T09:17:19.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 115.0 s, 1245.0 tps, lat 80.074 ms stddev 96.056 2023-10-08T09:17:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.312+0000 7fb19cc95700 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1b90820 0x55cfc1b92c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.314+0000 7fb177fff700 10 client.25750 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.314+0000 7fb19cc95700 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1b90820 0x55cfc1b92c40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d23b29c80b7ed3f2 server_cookie=b509eda4697ca5f1 in_seq=0 out_seq=0 2023-10-08T09:17:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.319+0000 7fb177fff700 1 -- 172.21.15.49:0/3151115917 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cfc1ab5db0 con 0x55cfc1b90820 2023-10-08T09:17:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.319+0000 7fb177fff700 10 client.25750 handle_command_reply: tid=0 2023-10-08T09:17:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.319+0000 7fb1ab0e2b80 1 client.25750 shutdown 2023-10-08T09:17:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.319+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 msgr2=0x55cfc1b73d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.319+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b73d00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb184011690 tx=0x7fb184011a50 comp rx=0 tx=0).stop 2023-10-08T09:17:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.324+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 shutdown_connections 2023-10-08T09:17:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.324+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1b90820 0x55cfc1b92c40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.324+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b7ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.324+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b69e40 0x55cfc1b737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.324+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3151115917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b73d00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.324+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 >> 172.21.15.49:0/3151115917 conn(0x55cfc1ade200 msgr2=0x55cfc1b64010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.325+0000 7fb177fff700 0 client.25750 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.327+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 shutdown_connections 2023-10-08T09:17:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.332+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3151115917 wait complete. 2023-10-08T09:17:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.333+0000 7fb1ab0e2b80 20 client.25750 trim_cache size 0 max 16384 2023-10-08T09:17:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 Processor -- start 2023-10-08T09:17:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 -- start start 2023-10-08T09:17:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b7ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc193d8c0 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1b8da60 con 0x55cfc1b69e40 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1b66030 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb19d496700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb19d496700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51040/0 (socket says 172.21.15.49:51040) 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb19d496700 1 -- 172.21.15.49:0/965176487 learned_addr learned my addr 172.21.15.49:0/965176487 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.356+0000 7fb197fff700 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b7ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb19d496700 1 -- 172.21.15.49:0/965176487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 msgr2=0x55cfc1b7ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb19d496700 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b7ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb19d496700 1 -- 172.21.15.49:0/965176487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 msgr2=0x55cfc1b8c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb19d496700 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb19d496700 1 -- 172.21.15.49:0/965176487 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfc190e1f0 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb19d496700 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6e120 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb190015000 tx=0x7fb19001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3528837d1937171 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb1837fe700 1 -- 172.21.15.49:0/965176487 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19000bcb0 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb1837fe700 1 -- 172.21.15.49:0/965176487 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb19000be10 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.357+0000 7fb1837fe700 1 -- 172.21.15.49:0/965176487 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190005900 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/965176487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 msgr2=0x55cfc1b6e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6e120 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb190015000 tx=0x7fb19001b000 comp rx=0 tx=0).stop 2023-10-08T09:17:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/965176487 shutdown_connections 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1b6e120 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b8c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/965176487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1b7ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/965176487 >> 172.21.15.49:0/965176487 conn(0x55cfc1ade200 msgr2=0x55cfc1b97560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/965176487 shutdown_connections 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/965176487 wait complete. 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 -- start start 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.358+0000 7fb1ab0e2b80 1 Processor -- start 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b78cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1be05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb1ab0e2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1be2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1be3100 con 0x55cfc1b6b2b0 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1be3270 con 0x55cfc1b69e40 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb1ab0e2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1be3430 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb1ab0e2b80 10 client.0 _ll_register_callbacks cb 0x7fb184009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb19d496700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1be2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb19d496700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1be2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51048/0 (socket says 172.21.15.49:51048) 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb19cc95700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b78cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb19d496700 1 -- 172.21.15.49:0/3605839750 learned_addr learned my addr 172.21.15.49:0/3605839750 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb197fff700 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1be05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb197fff700 1 -- 172.21.15.49:0/3605839750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 msgr2=0x55cfc1b78cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb197fff700 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b78cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb197fff700 1 -- 172.21.15.49:0/3605839750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 msgr2=0x55cfc1be2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb197fff700 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1be2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb197fff700 1 -- 172.21.15.49:0/3605839750 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1981094b0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.359+0000 7fb19d496700 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1be2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb197fff700 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1be05e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb18401a7d0 tx=0x7fb18401abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f40d8752782c21d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb177fff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb177fff700 1 -- 172.21.15.49:0/3605839750 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184005040 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb18c09b3d0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb177fff700 1 -- 172.21.15.49:0/3605839750 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb184000e40 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb177fff700 1 -- 172.21.15.49:0/3605839750 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1840051a0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 10 client.25756 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55cfc18e6250 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb177fff700 1 -- 172.21.15.49:0/3605839750 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb18c09b3d0 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb177fff700 1 -- 172.21.15.49:0/3605839750 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb18402b070 con 0x55cfc1b6a7a0 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 10 client.25756 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 20 client.25756 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1be4fd0 0x55cfc1be73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 4 client.25756 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.360+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cfc1ab5ad0 con 0x55cfc1be4fd0 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.361+0000 7fb19cc95700 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1be4fd0 0x55cfc1be73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.362+0000 7fb19cc95700 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1be4fd0 0x55cfc1be73f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4a76d60e5620e21 server_cookie=e7a23aa4f8eb226e in_seq=0 out_seq=0 2023-10-08T09:17:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.362+0000 7fb177fff700 10 client.25756 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb177fff700 1 -- 172.21.15.49:0/3605839750 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cfc1ab5ad0 con 0x55cfc1be4fd0 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb177fff700 10 client.25756 handle_command_reply: tid=0 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 client.25756 shutdown 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 msgr2=0x55cfc1be05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1be05e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb18401a7d0 tx=0x7fb18401abf0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 shutdown_connections 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cfc1be4fd0 0x55cfc1be73f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cfc1b6b2b0 0x55cfc1be2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cfc1b69e40 0x55cfc1b78cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb177fff700 0 client.25756 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 --2- 172.21.15.49:0/3605839750 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cfc1b6a7a0 0x55cfc1be05e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 >> 172.21.15.49:0/3605839750 conn(0x55cfc1ade200 msgr2=0x55cfc1b6e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 shutdown_connections 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 1 -- 172.21.15.49:0/3605839750 wait complete. 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.386+0000 7fb1ab0e2b80 20 client.25756 trim_cache size 0 max 16384 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:17:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:17:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:17:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb1880a1650 msgr2=0x7fb1880a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb1880a1650 0x7fb1880a3b10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb1900062a0 tx=0x7fb1900061f0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1980fe4c0 msgr2=0x7fb198194d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1980fe4c0 0x7fb198194d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00e9f0 tx=0x7fb18c00edb0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 shutdown_connections 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb1880a1650 0x7fb1880a3b10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1980fe4c0 0x7fb198194d20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb1981063f0 0x7fb19819bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 --2- 172.21.15.49:0/1340684214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1980fee20 0x7fb198195260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 >> 172.21.15.49:0/1340684214 conn(0x7fb1980f9ef0 msgr2=0x7fb19810b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 shutdown_connections 2023-10-08T09:17:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:19.387+0000 7fb19eef9700 1 -- 172.21.15.49:0/1340684214 wait complete. 2023-10-08T09:17:19.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:19 smithi132 ceph-mon[103389]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 775 op/s 2023-10-08T09:17:19.886 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:19 smithi049 ceph-mon[99485]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 775 op/s 2023-10-08T09:17:19.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:19 smithi184 ceph-mon[103675]: pgmap v401: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 MiB/s wr, 775 op/s 2023-10-08T09:17:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 749 op/s 2023-10-08T09:17:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:17:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:21 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 749 op/s 2023-10-08T09:17:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:17:21.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:21 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: pgmap v402: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 749 op/s 2023-10-08T09:17:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:17:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:21 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:22.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:22 smithi049 ceph-mon[99485]: scrub summary: active paths [/] 2023-10-08T09:17:22.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:22 smithi049 ceph-mon[99485]: scrub summary: idle+waiting paths [/] 2023-10-08T09:17:22.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:17:22.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:17:22.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:22.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:22 smithi184 ceph-mon[103675]: scrub summary: active paths [/] 2023-10-08T09:17:22.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:22 smithi184 ceph-mon[103675]: scrub summary: idle+waiting paths [/] 2023-10-08T09:17:22.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:17:22.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:17:22.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:23.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:22 smithi132 ceph-mon[103389]: scrub summary: active paths [/] 2023-10-08T09:17:23.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:22 smithi132 ceph-mon[103389]: scrub summary: idle+waiting paths [/] 2023-10-08T09:17:23.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:22 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:17:23.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:22 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:17:23.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:22 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:17:23.892 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:23 smithi049 ceph-mon[99485]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 788 op/s 2023-10-08T09:17:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:23 smithi184 ceph-mon[103675]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 788 op/s 2023-10-08T09:17:24.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:23 smithi132 ceph-mon[103389]: pgmap v403: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 14 MiB/s wr, 788 op/s 2023-10-08T09:17:24.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 120.0 s, 1405.2 tps, lat 71.587 ms stddev 81.262 2023-10-08T09:17:25.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:25 smithi049 ceph-mon[99485]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 810 op/s 2023-10-08T09:17:25.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:25 smithi184 ceph-mon[103675]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 810 op/s 2023-10-08T09:17:26.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:25 smithi132 ceph-mon[103389]: pgmap v404: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 810 op/s 2023-10-08T09:17:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:27 smithi049 ceph-mon[99485]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 811 op/s 2023-10-08T09:17:27.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:27 smithi184 ceph-mon[103675]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 811 op/s 2023-10-08T09:17:28.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:27 smithi132 ceph-mon[103389]: pgmap v405: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 811 op/s 2023-10-08T09:17:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 125.0 s, 1436.2 tps, lat 68.518 ms stddev 82.193 2023-10-08T09:17:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:29 smithi184 ceph-mon[103675]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 786 op/s 2023-10-08T09:17:30.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:29 smithi132 ceph-mon[103389]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 786 op/s 2023-10-08T09:17:30.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:29 smithi049 ceph-mon[99485]: pgmap v406: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 16 MiB/s wr, 786 op/s 2023-10-08T09:17:31.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:31 smithi184 ceph-mon[103675]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 698 op/s 2023-10-08T09:17:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:31 smithi132 ceph-mon[103389]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 698 op/s 2023-10-08T09:17:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:31 smithi049 ceph-mon[99485]: pgmap v407: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 698 op/s 2023-10-08T09:17:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.577+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/2321605237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 msgr2=0x7fc2cc10ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.577+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/2321605237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc10ff90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009f90 tx=0x7fc2c400b330 comp rx=0 tx=0).stop 2023-10-08T09:17:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.577+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/2321605237 shutdown_connections 2023-10-08T09:17:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.577+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/2321605237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc108900 0x7fc2cc10acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.577+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/2321605237 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc10ff90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.577+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/2321605237 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2cc10b230 0x7fc2cc10d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.577+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/2321605237 >> 172.21.15.49:0/2321605237 conn(0x7fc2cc06d770 msgr2=0x7fc2cc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.578+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/2321605237 shutdown_connections 2023-10-08T09:17:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.578+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/2321605237 wait complete. 2023-10-08T09:17:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 Processor -- start 2023-10-08T09:17:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 -- start start 2023-10-08T09:17:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2cc10b230 0x7fc2cc19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc1a2360 0x7fc2cc1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc1a4d30 con 0x7fc2cc10b230 2023-10-08T09:17:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc1a4ea0 con 0x7fc2cc10db60 2023-10-08T09:17:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.579+0000 7fc2d2b0e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc1a5060 con 0x7fc2cc1a2360 2023-10-08T09:17:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc19d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40670/0 (socket says 172.21.15.49:40670) 2023-10-08T09:17:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 -- 172.21.15.49:0/1292464820 learned_addr learned my addr 172.21.15.49:0/1292464820 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2cc10b230 0x7fc2cc19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 -- 172.21.15.49:0/1292464820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc1a2360 msgr2=0x7fc2cc1a47f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:17:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc1a2360 0x7fc2cc1a47f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 -- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2cc10b230 msgr2=0x7fc2cc19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2cc10b230 0x7fc2cc19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.580+0000 7fc2cbfff700 1 -- 172.21.15.49:0/1292464820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c4009c40 con 0x7fc2cc10db60 2023-10-08T09:17:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.581+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc19d350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9a0 tx=0x7fc2bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.581+0000 7fc2c9ffb700 1 -- 172.21.15.49:0/1292464820 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00cbd0 con 0x7fc2cc10db60 2023-10-08T09:17:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.581+0000 7fc2c9ffb700 1 -- 172.21.15.49:0/1292464820 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2bc00cd30 con 0x7fc2cc10db60 2023-10-08T09:17:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.581+0000 7fc2c9ffb700 1 -- 172.21.15.49:0/1292464820 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc018680 con 0x7fc2cc10db60 2023-10-08T09:17:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.581+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2cc1a5340 con 0x7fc2cc10db60 2023-10-08T09:17:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.582+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc1a5800 con 0x7fc2cc10db60 2023-10-08T09:17:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.586+0000 7fc2c9ffb700 1 -- 172.21.15.49:0/1292464820 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc2bc010430 con 0x7fc2cc10db60 2023-10-08T09:17:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.586+0000 7fc2c9ffb700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc2b40a1580 0x7fc2b40a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.587+0000 7fc2c9ffb700 1 -- 172.21.15.49:0/1292464820 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2bc014070 con 0x7fc2cc10db60 2023-10-08T09:17:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.587+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc2b40a1580 0x7fc2b40a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.588+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc2b40a1580 0x7fc2b40a3a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0005950 tx=0x7fc2c000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 Processor -- start 2023-10-08T09:17:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 -- start start 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d200260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d200cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d201730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d168400 con 0x564a5d2012b0 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d159bc0 con 0x564a5d2007a0 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d1574e0 con 0x564a5d1ffe40 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2d10ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d201730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2d10ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d201730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51066/0 (socket says 172.21.15.49:51066) 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2d08aa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d200260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2d10ab700 1 -- 172.21.15.49:0/811516733 learned_addr learned my addr 172.21.15.49:0/811516733 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.589+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d200cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2d10ab700 1 -- 172.21.15.49:0/811516733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 msgr2=0x564a5d200260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d200260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2d10ab700 1 -- 172.21.15.49:0/811516733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 msgr2=0x564a5d200cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d200cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2d10ab700 1 -- 172.21.15.49:0/811516733 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a5cf7dd80 con 0x564a5d2012b0 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d201730 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009510 tx=0x7fc2c4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820a1a53c266a791 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2b2ffd700 1 -- 172.21.15.49:0/811516733 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400e040 con 0x564a5d2012b0 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2b2ffd700 1 -- 172.21.15.49:0/811516733 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c4008e70 con 0x564a5d2012b0 2023-10-08T09:17:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.590+0000 7fc2b2ffd700 1 -- 172.21.15.49:0/811516733 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400bde0 con 0x564a5d2012b0 2023-10-08T09:17:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.591+0000 7fc2decf7b80 1 -- 172.21.15.49:0/811516733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 msgr2=0x564a5d201730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.591+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d201730 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009510 tx=0x7fc2c4009ac0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.592+0000 7fc2decf7b80 1 -- 172.21.15.49:0/811516733 shutdown_connections 2023-10-08T09:17:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.592+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d200260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.592+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d201730 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.592+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/811516733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d200cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.592+0000 7fc2decf7b80 1 -- 172.21.15.49:0/811516733 >> 172.21.15.49:0/811516733 conn(0x564a5d174200 msgr2=0x564a5d1fa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.593+0000 7fc2decf7b80 1 -- 172.21.15.49:0/811516733 shutdown_connections 2023-10-08T09:17:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.593+0000 7fc2decf7b80 1 -- 172.21.15.49:0/811516733 wait complete. 2023-10-08T09:17:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.593+0000 7fc2decf7b80 1 -- start start 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.593+0000 7fc2decf7b80 1 Processor -- start 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d209790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d209cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d210d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d152dd0 con 0x564a5d2012b0 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d0ef6b0 con 0x564a5d2007a0 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d0ef350 con 0x564a5d1ffe40 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2decf7b80 10 client.0 _ll_register_callbacks cb 0x564a5cf4c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d209790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d209790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38102/0 (socket says 172.21.15.49:38102) 2023-10-08T09:17:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 -- 172.21.15.49:0/4281900094 learned_addr learned my addr 172.21.15.49:0/4281900094 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d210d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d209cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 -- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 msgr2=0x564a5d209cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d209cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 -- 172.21.15.49:0/4281900094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 msgr2=0x564a5d210d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d210d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.594+0000 7fc2d08aa700 1 -- 172.21.15.49:0/4281900094 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc110b80 con 0x564a5d1ffe40 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d209790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b640 tx=0x7fc2c00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41f6d79061d4584 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2abfff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2abfff700 1 -- 172.21.15.49:0/4281900094 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011ad0 con 0x564a5d1ffe40 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2abfff700 1 -- 172.21.15.49:0/4281900094 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2c0017070 con 0x564a5d1ffe40 2023-10-08T09:17:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a5d11ae50 con 0x564a5d1ffe40 2023-10-08T09:17:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2abfff700 1 -- 172.21.15.49:0/4281900094 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2c0011c30 con 0x564a5d1ffe40 2023-10-08T09:17:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2decf7b80 10 client.25765 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.595+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564a5cf66c00 con 0x564a5d1ffe40 2023-10-08T09:17:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.596+0000 7fc2abfff700 1 -- 172.21.15.49:0/4281900094 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a5d11ae50 con 0x564a5d1ffe40 2023-10-08T09:17:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.597+0000 7fc2abfff700 1 -- 172.21.15.49:0/4281900094 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc2c001f430 con 0x564a5d1ffe40 2023-10-08T09:17:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.597+0000 7fc2decf7b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.597+0000 7fc2decf7b80 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.597+0000 7fc2decf7b80 20 client.25765 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.597+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d2156a0 0x564a5d217ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.597+0000 7fc2decf7b80 4 client.25765 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.597+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a5d14bdb0 con 0x564a5d2156a0 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.598+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d2156a0 0x564a5d217ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.599+0000 7fc2abfff700 10 client.25765 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.599+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d2156a0 0x564a5d217ac0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a4c14fee8d32a6 server_cookie=a6d5d73d26f726f4 in_seq=0 out_seq=0 2023-10-08T09:17:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.599+0000 7fc2abfff700 1 -- 172.21.15.49:0/4281900094 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a5d14bdb0 con 0x564a5d2156a0 2023-10-08T09:17:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.599+0000 7fc2abfff700 10 client.25765 handle_command_reply: tid=0 2023-10-08T09:17:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.599+0000 7fc2decf7b80 1 client.25765 shutdown 2023-10-08T09:17:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.600+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 msgr2=0x564a5d209790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.600+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d209790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000b640 tx=0x7fc2c00047c0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 shutdown_connections 2023-10-08T09:17:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d2156a0 0x564a5d217ac0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d1ffe40 0x564a5d209790 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d210d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/4281900094 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2007a0 0x564a5d209cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 >> 172.21.15.49:0/4281900094 conn(0x564a5d174200 msgr2=0x564a5d1fa010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2abfff700 0 client.25765 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 shutdown_connections 2023-10-08T09:17:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.601+0000 7fc2decf7b80 1 -- 172.21.15.49:0/4281900094 wait complete. 2023-10-08T09:17:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.602+0000 7fc2decf7b80 20 client.25765 trim_cache size 0 max 16384 2023-10-08T09:17:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 Processor -- start 2023-10-08T09:17:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 -- start start 2023-10-08T09:17:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d1ffe40 0x564a5d2113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 0x564a5d210ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d208ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5cfd38c0 con 0x564a5d2012b0 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d201120 con 0x564a5d1ffe40 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d208780 con 0x564a5d2007a0 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.627+0000 7fc2d10ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d208ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d08aa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d1ffe40 0x564a5d2113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d10ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d208ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51078/0 (socket says 172.21.15.49:51078) 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d10ab700 1 -- 172.21.15.49:0/2386695672 learned_addr learned my addr 172.21.15.49:0/2386695672 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 0x564a5d210ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d10ab700 1 -- 172.21.15.49:0/2386695672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 msgr2=0x564a5d210ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 0x564a5d210ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d10ab700 1 -- 172.21.15.49:0/2386695672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d1ffe40 msgr2=0x564a5d2113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d1ffe40 0x564a5d2113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d10ab700 1 -- 172.21.15.49:0/2386695672 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a5cfa41f0 con 0x564a5d2012b0 2023-10-08T09:17:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.628+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d1ffe40 0x564a5d2113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.629+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d208ca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009510 tx=0x7fc2c400af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fc1e7e367ca51d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.629+0000 7fc2b37fe700 1 -- 172.21.15.49:0/2386695672 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400e040 con 0x564a5d2012b0 2023-10-08T09:17:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.629+0000 7fc2b37fe700 1 -- 172.21.15.49:0/2386695672 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c4003a00 con 0x564a5d2012b0 2023-10-08T09:17:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.629+0000 7fc2b37fe700 1 -- 172.21.15.49:0/2386695672 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c401d470 con 0x564a5d2012b0 2023-10-08T09:17:32.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 -- 172.21.15.49:0/2386695672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 msgr2=0x564a5d208ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d208ca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009510 tx=0x7fc2c400af80 comp rx=0 tx=0).stop 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 -- 172.21.15.49:0/2386695672 shutdown_connections 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d2012b0 0x564a5d208ca0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d1ffe40 0x564a5d2113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/2386695672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 0x564a5d210ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 -- 172.21.15.49:0/2386695672 >> 172.21.15.49:0/2386695672 conn(0x564a5d174200 msgr2=0x564a5d21c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 -- 172.21.15.49:0/2386695672 shutdown_connections 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.630+0000 7fc2decf7b80 1 -- 172.21.15.49:0/2386695672 wait complete. 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.631+0000 7fc2decf7b80 1 -- start start 2023-10-08T09:17:32.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.631+0000 7fc2decf7b80 1 Processor -- start 2023-10-08T09:17:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d1ffe40 0x564a5d20c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 0x564a5d20c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2decf7b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2012b0 0x564a5d20f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d20fa50 con 0x564a5d1ffe40 2023-10-08T09:17:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d20fbc0 con 0x564a5d2012b0 2023-10-08T09:17:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2decf7b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5d20fd80 con 0x564a5d2007a0 2023-10-08T09:17:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2decf7b80 10 client.0 _ll_register_callbacks cb 0x7fc2c0003900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d1ffe40 0x564a5d20c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d1ffe40 0x564a5d20c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51082/0 (socket says 172.21.15.49:51082) 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 -- 172.21.15.49:0/1384218213 learned_addr learned my addr 172.21.15.49:0/1384218213 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2012b0 0x564a5d20f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 -- 172.21.15.49:0/1384218213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 msgr2=0x564a5d20c590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 0x564a5d20c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 -- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2012b0 msgr2=0x564a5d20f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2012b0 0x564a5d20f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d08aa700 1 -- 172.21.15.49:0/1384218213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a5cf66c00 con 0x564a5d1ffe40 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.632+0000 7fc2d10ab700 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2012b0 0x564a5d20f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.633+0000 7fc2d08aa700 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d1ffe40 0x564a5d20c050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0004790 tx=0x7fc2c001f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54778888956cae32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:32.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.633+0000 7fc2abfff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:17:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.633+0000 7fc2abfff700 1 -- 172.21.15.49:0/1384218213 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011920 con 0x564a5d1ffe40 2023-10-08T09:17:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.633+0000 7fc2abfff700 1 -- 172.21.15.49:0/1384218213 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc2c0017040 con 0x564a5d1ffe40 2023-10-08T09:17:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.633+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2c4008e70 con 0x564a5d1ffe40 2023-10-08T09:17:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.634+0000 7fc2abfff700 1 -- 172.21.15.49:0/1384218213 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2c0011a80 con 0x564a5d1ffe40 2023-10-08T09:17:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.634+0000 7fc2decf7b80 10 client.16116 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:32.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.634+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564a5cf7dd80 con 0x564a5d1ffe40 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.634+0000 7fc2abfff700 1 -- 172.21.15.49:0/1384218213 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c4008e70 con 0x564a5d1ffe40 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.634+0000 7fc2abfff700 1 -- 172.21.15.49:0/1384218213 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc2c0047ca0 con 0x564a5d1ffe40 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2decf7b80 10 client.16116 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2decf7b80 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2decf7b80 20 client.16116 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d269a70 0x564a5d26be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2decf7b80 4 client.16116 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:17:32.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a5d14bad0 con 0x564a5d269a70 2023-10-08T09:17:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d269a70 0x564a5d26be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2abfff700 10 client.16116 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:32.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.635+0000 7fc2cbfff700 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d269a70 0x564a5d26be90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20aba5954759c0e1 server_cookie=5569ea8be3a4ceae in_seq=0 out_seq=0 2023-10-08T09:17:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.700+0000 7fc2abfff700 1 -- 172.21.15.49:0/1384218213 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a5d14bad0 con 0x564a5d269a70 2023-10-08T09:17:32.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.700+0000 7fc2abfff700 10 client.16116 handle_command_reply: tid=0 2023-10-08T09:17:32.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.701+0000 7fc2decf7b80 1 client.16116 shutdown 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d1ffe40 msgr2=0x564a5d20c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d1ffe40 0x564a5d20c050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0004790 tx=0x7fc2c001f6e0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 shutdown_connections 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564a5d269a70 0x564a5d26be90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564a5d2012b0 0x564a5d20f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564a5d1ffe40 0x564a5d20c050 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2abfff700 0 client.16116 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 --2- 172.21.15.49:0/1384218213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564a5d2007a0 0x564a5d20c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 >> 172.21.15.49:0/1384218213 conn(0x564a5d174200 msgr2=0x564a5d2176b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 shutdown_connections 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 1 -- 172.21.15.49:0/1384218213 wait complete. 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.703+0000 7fc2decf7b80 20 client.16116 trim_cache size 0 max 16384 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:17:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:17:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc2b40a1580 msgr2=0x7fc2b40a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc2b40a1580 0x7fc2b40a3a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0005950 tx=0x7fc2c000b390 comp rx=0 tx=0).stop 2023-10-08T09:17:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 msgr2=0x7fc2cc19d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc19d350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9a0 tx=0x7fc2bc00ecb0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 shutdown_connections 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc2b40a1580 0x7fc2b40a3a40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2cc1a2360 0x7fc2cc1a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2cc10b230 0x7fc2cc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 --2- 172.21.15.49:0/1292464820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2cc10db60 0x7fc2cc19d350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 >> 172.21.15.49:0/1292464820 conn(0x7fc2cc06d770 msgr2=0x7fc2cc10baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 shutdown_connections 2023-10-08T09:17:32.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:32.704+0000 7fc2d2b0e700 1 -- 172.21.15.49:0/1292464820 wait complete. 2023-10-08T09:17:34.003 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:33 smithi184 ceph-mon[103675]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 720 op/s 2023-10-08T09:17:34.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:33 smithi132 ceph-mon[103389]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 720 op/s 2023-10-08T09:17:34.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:33 smithi049 ceph-mon[99485]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 14 MiB/s wr, 720 op/s 2023-10-08T09:17:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 130.0 s, 1160.6 tps, lat 76.990 ms stddev 88.180 2023-10-08T09:17:35.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:35 smithi184 ceph-mon[103675]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-10-08T09:17:36.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:35 smithi132 ceph-mon[103389]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-10-08T09:17:36.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:35 smithi049 ceph-mon[99485]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 16 MiB/s wr, 785 op/s 2023-10-08T09:17:37.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:37 smithi184 ceph-mon[103675]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.37k op/s 2023-10-08T09:17:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:37 smithi132 ceph-mon[103389]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.37k op/s 2023-10-08T09:17:38.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:37 smithi049 ceph-mon[99485]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.37k op/s 2023-10-08T09:17:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 135.0 s, 1242.8 tps, lat 90.753 ms stddev 188.124 2023-10-08T09:17:40.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:39 smithi132 ceph-mon[103389]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.78k op/s 2023-10-08T09:17:40.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:39 smithi049 ceph-mon[99485]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.78k op/s 2023-10-08T09:17:40.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:39 smithi184 ceph-mon[103675]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 28 MiB/s wr, 1.78k op/s 2023-10-08T09:17:41.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:17:41.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.583+0000 7fba40986700 1 -- 172.21.15.49:0/1088358763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3c10a160 msgr2=0x7fba3c10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.583+0000 7fba40986700 1 --2- 172.21.15.49:0/1088358763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3c10a160 0x7fba3c10c5f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fba2c01a9a0 tx=0x7fba2c01acb0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.585+0000 7fba40986700 1 -- 172.21.15.49:0/1088358763 shutdown_connections 2023-10-08T09:17:41.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.585+0000 7fba40986700 1 --2- 172.21.15.49:0/1088358763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba3c1077f0 0x7fba3c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.585+0000 7fba40986700 1 --2- 172.21.15.49:0/1088358763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c10cb30 0x7fba3c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.595 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.585+0000 7fba40986700 1 --2- 172.21.15.49:0/1088358763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3c10a160 0x7fba3c10c5f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.585+0000 7fba40986700 1 -- 172.21.15.49:0/1088358763 >> 172.21.15.49:0/1088358763 conn(0x7fba3c06d790 msgr2=0x7fba3c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.590+0000 7fba40986700 1 -- 172.21.15.49:0/1088358763 shutdown_connections 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.590+0000 7fba40986700 1 -- 172.21.15.49:0/1088358763 wait complete. 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.591+0000 7fba40986700 1 Processor -- start 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.591+0000 7fba40986700 1 -- start start 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba40986700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c1077f0 0x7fba3c118160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba40986700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3c10a160 0x7fba3c1186a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba40986700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba3c10cb30 0x7fba3c1c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba40986700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c1c69d0 con 0x7fba3c10a160 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba40986700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c1c6b10 con 0x7fba3c1077f0 2023-10-08T09:17:41.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba40986700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c1c6c50 con 0x7fba3c10cb30 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c1077f0 0x7fba3c118160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c1077f0 0x7fba3c118160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40704/0 (socket says 172.21.15.49:40704) 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba3a59c700 1 -- 172.21.15.49:0/2152586494 learned_addr learned my addr 172.21.15.49:0/2152586494 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.592+0000 7fba3ad9d700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba3c10cb30 0x7fba3c1c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba3a59c700 1 -- 172.21.15.49:0/2152586494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba3c10cb30 msgr2=0x7fba3c1c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba3a59c700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba3c10cb30 0x7fba3c1c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba3a59c700 1 -- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3c10a160 msgr2=0x7fba3c1186a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba3a59c700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3c10a160 0x7fba3c1186a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba3a59c700 1 -- 172.21.15.49:0/2152586494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c01a600 con 0x7fba3c1077f0 2023-10-08T09:17:41.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba3a59c700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c1077f0 0x7fba3c118160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b550 tx=0x7fba3000b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:41.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba2b7fe700 1 -- 172.21.15.49:0/2152586494 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30011830 con 0x7fba3c1077f0 2023-10-08T09:17:41.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba2b7fe700 1 -- 172.21.15.49:0/2152586494 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba30011e50 con 0x7fba3c1077f0 2023-10-08T09:17:41.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba2b7fe700 1 -- 172.21.15.49:0/2152586494 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000f2e0 con 0x7fba3c1077f0 2023-10-08T09:17:41.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c1c6f30 con 0x7fba3c1077f0 2023-10-08T09:17:41.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.593+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c1c7300 con 0x7fba3c1077f0 2023-10-08T09:17:41.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.597+0000 7fba2b7fe700 1 -- 172.21.15.49:0/2152586494 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fba30011990 con 0x7fba3c1077f0 2023-10-08T09:17:41.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.598+0000 7fba2b7fe700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fba240a1600 0x7fba240a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.598+0000 7fba4cb6fb80 1 Processor -- start 2023-10-08T09:17:41.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.598+0000 7fba2b7fe700 1 -- 172.21.15.49:0/2152586494 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fba300cc910 con 0x7fba3c1077f0 2023-10-08T09:17:41.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.598+0000 7fba4cb6fb80 1 -- start start 2023-10-08T09:17:41.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.599+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c4060 0x559ecc9c4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.599+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c49c0 0x559ecc9c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.599+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.599+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc92c460 con 0x559ecc9c5800 2023-10-08T09:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.599+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc91dc20 con 0x559ecc9c4060 2023-10-08T09:17:41.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.599+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc91b540 con 0x559ecc9c49c0 2023-10-08T09:17:41.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.599+0000 7fba3ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba3ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51094/0 (socket says 172.21.15.49:51094) 2023-10-08T09:17:41.603 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba3ad9d700 1 -- 172.21.15.49:0/2624879273 learned_addr learned my addr 172.21.15.49:0/2624879273 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:41.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba39d9b700 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c49c0 0x559ecc9c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba39d9b700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fba240a1600 0x7fba240a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba3ad9d700 1 -- 172.21.15.49:0/2624879273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c49c0 msgr2=0x559ecc9c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:41.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba3ad9d700 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c49c0 0x559ecc9c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.604 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba3ad9d700 1 -- 172.21.15.49:0/2624879273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c4060 msgr2=0x559ecc9c4480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:17:41.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba3ad9d700 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c4060 0x559ecc9c4480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.600+0000 7fba3ad9d700 1 -- 172.21.15.49:0/2624879273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ecc741de0 con 0x559ecc9c5800 2023-10-08T09:17:41.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.601+0000 7fba3ad9d700 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9c5c90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fba3400a430 tx=0x7fba34009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dcda9115b32f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:41.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.601+0000 7fba28ff9700 1 -- 172.21.15.49:0/2624879273 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34010040 con 0x559ecc9c5800 2023-10-08T09:17:41.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.601+0000 7fba28ff9700 1 -- 172.21.15.49:0/2624879273 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba34014ae0 con 0x559ecc9c5800 2023-10-08T09:17:41.605 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.601+0000 7fba28ff9700 1 -- 172.21.15.49:0/2624879273 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba34013ac0 con 0x559ecc9c5800 2023-10-08T09:17:41.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba39d9b700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fba240a1600 0x7fba240a3ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x559ecc9c4e00 tx=0x7fba2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:41.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2624879273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 msgr2=0x559ecc9c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9c5c90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fba3400a430 tx=0x7fba34009650 comp rx=0 tx=0).stop 2023-10-08T09:17:41.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2624879273 shutdown_connections 2023-10-08T09:17:41.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9c5c90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c4060 0x559ecc9c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2624879273 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c49c0 0x559ecc9c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2624879273 >> 172.21.15.49:0/2624879273 conn(0x559ecc938260 msgr2=0x559ecc9bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:41.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2624879273 shutdown_connections 2023-10-08T09:17:41.607 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2624879273 wait complete. 2023-10-08T09:17:41.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 -- start start 2023-10-08T09:17:41.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.602+0000 7fba4cb6fb80 1 Processor -- start 2023-10-08T09:17:41.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.615 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 0x559ecc9ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc916e30 con 0x559ecc9c5800 2023-10-08T09:17:41.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc8b3710 con 0x559ecc9c49c0 2023-10-08T09:17:41.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc8b33b0 con 0x559ecc9c4060 2023-10-08T09:17:41.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba4cb6fb80 10 client.0 _ll_register_callbacks cb 0x559ecc710800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38134/0 (socket says 172.21.15.49:38134) 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 -- 172.21.15.49:0/2715702785 learned_addr learned my addr 172.21.15.49:0/2715702785 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3ad9d700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba39d9b700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 0x559ecc9ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 -- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 msgr2=0x559ecc9ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 0x559ecc9ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 -- 172.21.15.49:0/2715702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 msgr2=0x559ecc9d53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:41.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9d53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3a59c700 1 -- 172.21.15.49:0/2715702785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba34009c60 con 0x559ecc9c4060 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.603+0000 7fba3ad9d700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba3a59c700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9cde60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba30010ac0 tx=0x7fba3009a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b8214596428245 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba21ffb700 1 -- 172.21.15.49:0/2715702785 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3009da50 con 0x559ecc9c4060 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba21ffb700 1 -- 172.21.15.49:0/2715702785 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fba300a1c90 con 0x559ecc9c4060 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ecc8deeb0 con 0x559ecc9c4060 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba21ffb700 1 -- 172.21.15.49:0/2715702785 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba30097ca0 con 0x559ecc9c4060 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 10 client.25777 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:41.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559ecc72ac60 con 0x559ecc9c4060 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba21ffb700 1 -- 172.21.15.49:0/2715702785 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ecc8deeb0 con 0x559ecc9c4060 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba21ffb700 1 -- 172.21.15.49:0/2715702785 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fba300a1800 con 0x559ecc9c4060 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 10 client.25777 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 10 client.25777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 20 client.25777 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecc9d9da0 0x559ecc9dc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 4 client.25777 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.604+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ecc90fe10 con 0x559ecc9d9da0 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.607+0000 7fba39d9b700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecc9d9da0 0x559ecc9dc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.608+0000 7fba39d9b700 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecc9d9da0 0x559ecc9dc1c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1c998f02db5e7b server_cookie=bc6b3b07d636f6c6 in_seq=0 out_seq=0 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.608+0000 7fba21ffb700 10 client.25777 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba21ffb700 1 -- 172.21.15.49:0/2715702785 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559ecc90fe10 con 0x559ecc9d9da0 2023-10-08T09:17:41.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba21ffb700 10 client.25777 handle_command_reply: tid=0 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 client.25777 shutdown 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 msgr2=0x559ecc9cde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9cde60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba30010ac0 tx=0x7fba3009a0d0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 shutdown_connections 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecc9d9da0 0x559ecc9dc1c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9cde60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba21ffb700 0 client.25777 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/2715702785 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 0x559ecc9ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 >> 172.21.15.49:0/2715702785 conn(0x559ecc938260 msgr2=0x559ecc9be4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 shutdown_connections 2023-10-08T09:17:41.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/2715702785 wait complete. 2023-10-08T09:17:41.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.609+0000 7fba4cb6fb80 20 client.25777 trim_cache size 0 max 16384 2023-10-08T09:17:41.632 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.630+0000 7fba4cb6fb80 1 Processor -- start 2023-10-08T09:17:41.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba4cb6fb80 1 -- start start 2023-10-08T09:17:41.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 0x559ecc9d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.634 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc797920 con 0x559ecc9c5800 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc9cc990 con 0x559ecc9c49c0 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc9b95f0 con 0x559ecc9c4060 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38142/0 (socket says 172.21.15.49:38142) 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.631+0000 7fba3a59c700 1 -- 172.21.15.49:0/1099069714 learned_addr learned my addr 172.21.15.49:0/1099069714 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba3a59c700 1 -- 172.21.15.49:0/1099069714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 msgr2=0x559ecc9d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba3a59c700 1 --2- 172.21.15.49:0/1099069714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 0x559ecc9d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba3a59c700 1 -- 172.21.15.49:0/1099069714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 msgr2=0x559ecc9cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba3a59c700 1 --2- 172.21.15.49:0/1099069714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba3a59c700 1 -- 172.21.15.49:0/1099069714 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ecc768250 con 0x559ecc9c4060 2023-10-08T09:17:41.635 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba3a59c700 1 --2- 172.21.15.49:0/1099069714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9c4440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba30095e00 tx=0x7fba30098860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf78dc78d1db9c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:41.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba297fa700 1 -- 172.21.15.49:0/1099069714 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba300a1c90 con 0x559ecc9c4060 2023-10-08T09:17:41.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba297fa700 1 -- 172.21.15.49:0/1099069714 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba3009da50 con 0x559ecc9c4060 2023-10-08T09:17:41.636 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.632+0000 7fba297fa700 1 -- 172.21.15.49:0/1099069714 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30098df0 con 0x559ecc9c4060 2023-10-08T09:17:41.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/1099069714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 msgr2=0x559ecc9c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/1099069714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9c4440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba30095e00 tx=0x7fba30098860 comp rx=0 tx=0).stop 2023-10-08T09:17:41.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/1099069714 shutdown_connections 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/1099069714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9c4440 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/1099069714 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c5800 0x559ecc9cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/1099069714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c49c0 0x559ecc9d53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/1099069714 >> 172.21.15.49:0/1099069714 conn(0x559ecc938260 msgr2=0x559ecc9bedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/1099069714 shutdown_connections 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.633+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/1099069714 wait complete. 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 -- start start 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 Processor -- start 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c49c0 0x559ecc9d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c5800 0x559ecc9d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc9d4d70 con 0x559ecc9c49c0 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc9d4ee0 con 0x559ecc9c5800 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ecc9d50a0 con 0x559ecc9c4060 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba4cb6fb80 10 client.0 _ll_register_callbacks cb 0x7fba30099480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba39d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c49c0 0x559ecc9d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba3ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c5800 0x559ecc9d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba3a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9d3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38148/0 (socket says 172.21.15.49:38148) 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.634+0000 7fba3a59c700 1 -- 172.21.15.49:0/3925903099 learned_addr learned my addr 172.21.15.49:0/3925903099 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:41.642 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba3a59c700 1 -- 172.21.15.49:0/3925903099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c5800 msgr2=0x559ecc9d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba3a59c700 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c5800 0x559ecc9d4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba3a59c700 1 -- 172.21.15.49:0/3925903099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c49c0 msgr2=0x559ecc9d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba3a59c700 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c49c0 0x559ecc9d3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba3a59c700 1 -- 172.21.15.49:0/3925903099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ecc72ac60 con 0x559ecc9c4060 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba39d9b700 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c49c0 0x559ecc9d3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba3a59c700 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9d3420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba30099890 tx=0x7fba300ca160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef3beb4e50bef7b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba21ffb700 1 -- 172.21.15.49:0/3925903099 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30099000 con 0x559ecc9c4060 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba21ffb700 1 -- 172.21.15.49:0/3925903099 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fba300a1c90 con 0x559ecc9c4060 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba34014ae0 con 0x559ecc9c4060 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba21ffb700 1 -- 172.21.15.49:0/3925903099 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba300b90b0 con 0x559ecc9c4060 2023-10-08T09:17:41.643 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba4cb6fb80 10 client.25783 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x559ecc7402b0 con 0x559ecc9c4060 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.635+0000 7fba21ffb700 1 -- 172.21.15.49:0/3925903099 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba34014ae0 con 0x559ecc9c4060 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba21ffb700 1 -- 172.21.15.49:0/3925903099 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fba30005a20 con 0x559ecc9c4060 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba4cb6fb80 10 client.25783 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba4cb6fb80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba4cb6fb80 20 client.25783 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecca2e8b0 0x559ecca30cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba4cb6fb80 4 client.25783 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ecc90fb30 con 0x559ecca2e8b0 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba39d9b700 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecca2e8b0 0x559ecca30cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba39d9b700 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecca2e8b0 0x559ecca30cd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f95be0a81a234a server_cookie=8683e92fe2770bc0 in_seq=0 out_seq=0 2023-10-08T09:17:41.644 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.636+0000 7fba21ffb700 10 client.25783 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba21ffb700 1 -- 172.21.15.49:0/3925903099 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559ecc90fb30 con 0x559ecca2e8b0 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba21ffb700 10 client.25783 handle_command_reply: tid=0 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 client.25783 shutdown 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 msgr2=0x559ecc9d3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9d3420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba30099890 tx=0x7fba300ca160 comp rx=0 tx=0).stop 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 shutdown_connections 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559ecca2e8b0 0x559ecca30cd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559ecc9c4060 0x559ecc9d3420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559ecc9c5800 0x559ecc9d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.645 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba21ffb700 0 client.25783 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 --2- 172.21.15.49:0/3925903099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559ecc9c49c0 0x559ecc9d3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 >> 172.21.15.49:0/3925903099 conn(0x559ecc938260 msgr2=0x559ecc9cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 shutdown_connections 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 1 -- 172.21.15.49:0/3925903099 wait complete. 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba4cb6fb80 20 client.25783 trim_cache size 0 max 16384 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:17:41.646 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:17:41.647 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:17:41.647 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:17:41.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:17:41.647 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:17:41.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:17:41.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:17:41.647 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:17:41.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fba240a1600 msgr2=0x7fba240a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba40986700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fba240a1600 0x7fba240a3ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x559ecc9c4e00 tx=0x7fba2c000bc0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c1077f0 msgr2=0x7fba3c118160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:41.648 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.637+0000 7fba40986700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c1077f0 0x7fba3c118160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b550 tx=0x7fba3000b860 comp rx=0 tx=0).stop 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 shutdown_connections 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fba240a1600 0x7fba240a3ac0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba3c10cb30 0x7fba3c1c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba3c1077f0 0x7fba3c118160 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 --2- 172.21.15.49:0/2152586494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fba3c10a160 0x7fba3c1186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 >> 172.21.15.49:0/2152586494 conn(0x7fba3c06d790 msgr2=0x7fba3c10a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 shutdown_connections 2023-10-08T09:17:41.649 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:17:41.638+0000 7fba40986700 1 -- 172.21.15.49:0/2152586494 wait complete. 2023-10-08T09:17:41.654 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:17:41.787 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:41 smithi049 ceph-mon[99485]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.70k op/s 2023-10-08T09:17:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:41 smithi132 ceph-mon[103389]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.70k op/s 2023-10-08T09:17:42.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:41 smithi184 ceph-mon[103675]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.70k op/s 2023-10-08T09:17:43.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:42 smithi049 ceph-mon[99485]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2023-10-08T09:17:43.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:42 smithi184 ceph-mon[103675]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2023-10-08T09:17:43.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:42 smithi132 ceph-mon[103389]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2023-10-08T09:17:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 140.0 s, 1309.4 tps, lat 76.693 ms stddev 92.645 2023-10-08T09:17:45.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:45 smithi049 ceph-mon[99485]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2023-10-08T09:17:45.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:45 smithi184 ceph-mon[103675]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2023-10-08T09:17:45.546 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:45 smithi132 ceph-mon[103389]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 MiB/s wr, 1.72k op/s 2023-10-08T09:17:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.866+0000 7f018745c700 1 -- 172.21.15.49:0/350921697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f018010b230 msgr2=0x7f018010d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.866+0000 7f018745c700 1 --2- 172.21.15.49:0/350921697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f018010b230 0x7f018010d620 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0170009990 tx=0x7f0170009ca0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.866+0000 7f018745c700 1 -- 172.21.15.49:0/350921697 shutdown_connections 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.866+0000 7f018745c700 1 --2- 172.21.15.49:0/350921697 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0180108900 0x7f018010acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.866+0000 7f018745c700 1 --2- 172.21.15.49:0/350921697 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010db60 0x7f018010ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.866+0000 7f018745c700 1 --2- 172.21.15.49:0/350921697 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f018010b230 0x7f018010d620 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.866+0000 7f018745c700 1 -- 172.21.15.49:0/350921697 >> 172.21.15.49:0/350921697 conn(0x7f018006d930 msgr2=0x7f018006fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.867+0000 7f018745c700 1 -- 172.21.15.49:0/350921697 shutdown_connections 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.867+0000 7f018745c700 1 -- 172.21.15.49:0/350921697 wait complete. 2023-10-08T09:17:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.868+0000 7f018745c700 1 Processor -- start 2023-10-08T09:17:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f018745c700 1 -- start start 2023-10-08T09:17:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f018745c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010b230 0x7f018019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f018745c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f018010db60 0x7f018019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f018745c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01801a3520 0x7f01801a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f018745c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01801a5ee0 con 0x7f01801a3520 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f018745c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01801a6020 con 0x7f018010b230 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f018745c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01801a61b0 con 0x7f018010db60 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010b230 0x7f018019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010b230 0x7f018019e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40740/0 (socket says 172.21.15.49:40740) 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 -- 172.21.15.49:0/748265583 learned_addr learned my addr 172.21.15.49:0/748265583 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01849f7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f018010db60 0x7f018019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01859f9700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01801a3520 0x7f01801a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 -- 172.21.15.49:0/748265583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f018010db60 msgr2=0x7f018019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f018010db60 0x7f018019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 -- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01801a3520 msgr2=0x7f01801a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01801a3520 0x7f01801a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.870+0000 7f01851f8700 1 -- 172.21.15.49:0/748265583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01700095f0 con 0x7f018010b230 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.871+0000 7f01851f8700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010b230 0x7f018019e070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f017c006fd0 tx=0x7f017c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.871+0000 7f01767fc700 1 -- 172.21.15.49:0/748265583 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c010880 con 0x7f018010b230 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.871+0000 7f01767fc700 1 -- 172.21.15.49:0/748265583 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017c00be30 con 0x7f018010b230 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.871+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0180107850 con 0x7f018010b230 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.871+0000 7f01767fc700 1 -- 172.21.15.49:0/748265583 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c0183b0 con 0x7f018010b230 2023-10-08T09:17:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.871+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0180107d60 con 0x7f018010b230 2023-10-08T09:17:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.874+0000 7f01767fc700 1 -- 172.21.15.49:0/748265583 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f017c00f4d0 con 0x7f018010b230 2023-10-08T09:17:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.875+0000 7f01767fc700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f016c0a1480 0x7f016c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.875+0000 7f01767fc700 1 -- 172.21.15.49:0/748265583 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f017c0ce410 con 0x7f018010b230 2023-10-08T09:17:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.876+0000 7f0193645b80 1 Processor -- start 2023-10-08T09:17:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.876+0000 7f0193645b80 1 -- start start 2023-10-08T09:17:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.876+0000 7f0193645b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b17b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f0193645b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17b7a0 0x56044b17bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f0193645b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b17c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f0193645b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b0e3400 con 0x56044b17c2b0 2023-10-08T09:17:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f0193645b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b0d4bc0 con 0x56044b17b7a0 2023-10-08T09:17:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f0193645b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b0d24e0 con 0x56044b17ae40 2023-10-08T09:17:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f01851f8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b17b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f01851f8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b17b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38166/0 (socket says 172.21.15.49:38166) 2023-10-08T09:17:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.877+0000 7f01851f8700 1 -- 172.21.15.49:0/3692008396 learned_addr learned my addr 172.21.15.49:0/3692008396 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f01851f8700 1 -- 172.21.15.49:0/3692008396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17b7a0 msgr2=0x56044b17bcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f01851f8700 1 --2- 172.21.15.49:0/3692008396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17b7a0 0x56044b17bcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f01851f8700 1 -- 172.21.15.49:0/3692008396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 msgr2=0x56044b17c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f01851f8700 1 --2- 172.21.15.49:0/3692008396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b17c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f01851f8700 1 -- 172.21.15.49:0/3692008396 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044aef8d80 con 0x56044b17ae40 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f01851f8700 1 --2- 172.21.15.49:0/3692008396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b17b260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f017c09b390 tx=0x7f017c09af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b12b5a0065b1653 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f016b7fe700 1 -- 172.21.15.49:0/3692008396 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c09bb00 con 0x56044b17ae40 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f016b7fe700 1 -- 172.21.15.49:0/3692008396 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017c0ce050 con 0x56044b17ae40 2023-10-08T09:17:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.878+0000 7f016b7fe700 1 -- 172.21.15.49:0/3692008396 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c09b660 con 0x56044b17ae40 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.879+0000 7f0193645b80 1 -- 172.21.15.49:0/3692008396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 msgr2=0x56044b17b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.879+0000 7f0193645b80 1 --2- 172.21.15.49:0/3692008396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b17b260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f017c09b390 tx=0x7f017c09af40 comp rx=0 tx=0).stop 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.879+0000 7f0193645b80 1 -- 172.21.15.49:0/3692008396 shutdown_connections 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.879+0000 7f0193645b80 1 --2- 172.21.15.49:0/3692008396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b17b260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.879+0000 7f0193645b80 1 --2- 172.21.15.49:0/3692008396 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b17c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.879+0000 7f0193645b80 1 --2- 172.21.15.49:0/3692008396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17b7a0 0x56044b17bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.879+0000 7f0193645b80 1 -- 172.21.15.49:0/3692008396 >> 172.21.15.49:0/3692008396 conn(0x56044b0ef200 msgr2=0x56044b1758a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.880+0000 7f01849f7700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f016c0a1480 0x7f016c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.880+0000 7f0193645b80 1 -- 172.21.15.49:0/3692008396 shutdown_connections 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.880+0000 7f0193645b80 1 -- 172.21.15.49:0/3692008396 wait complete. 2023-10-08T09:17:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.880+0000 7f01849f7700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f016c0a1480 0x7f016c0a3940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f017000b790 tx=0x7f0170005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.880+0000 7f0193645b80 1 -- start start 2023-10-08T09:17:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.880+0000 7f0193645b80 1 Processor -- start 2023-10-08T09:17:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f0193645b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b186a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f0193645b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b186fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f0193645b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b1839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f0193645b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b0cddd0 con 0x56044b17c2b0 2023-10-08T09:17:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f0193645b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b06a6b0 con 0x56044b17ae40 2023-10-08T09:17:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f0193645b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b06a350 con 0x56044b183550 2023-10-08T09:17:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f0193645b80 10 client.0 _ll_register_callbacks cb 0x56044aec77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f01851f8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b186a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f01851f8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b186a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40754/0 (socket says 172.21.15.49:40754) 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f01851f8700 1 -- 172.21.15.49:0/3305157638 learned_addr learned my addr 172.21.15.49:0/3305157638 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f01849f7700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b186fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.881+0000 7f01859f9700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b1839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f01851f8700 1 -- 172.21.15.49:0/3305157638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 msgr2=0x56044b1839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f01851f8700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b1839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f01851f8700 1 -- 172.21.15.49:0/3305157638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 msgr2=0x56044b186fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f01851f8700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b186fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f01851f8700 1 -- 172.21.15.49:0/3305157638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0180107d60 con 0x56044b17ae40 2023-10-08T09:17:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f01859f9700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b1839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f01851f8700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b186a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f017c09b340 tx=0x7f017c09bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48b1b8bdf526772 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f0168ff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f0168ff9700 1 -- 172.21.15.49:0/3305157638 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c09b660 con 0x56044b17ae40 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f0168ff9700 1 -- 172.21.15.49:0/3305157638 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f017c09b7c0 con 0x56044b17ae40 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.882+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56044b095e50 con 0x56044b17ae40 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0168ff9700 1 -- 172.21.15.49:0/3305157638 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f017c0a3300 con 0x56044b17ae40 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 10 client.25769 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56044aee1c00 con 0x56044b17ae40 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0168ff9700 1 -- 172.21.15.49:0/3305157638 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56044b095e50 con 0x56044b17ae40 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0168ff9700 1 -- 172.21.15.49:0/3305157638 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f017c09a360 con 0x56044b17ae40 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 10 client.25769 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:45.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 20 client.25769 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b18c7d0 0x56044b18ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 4 client.25769 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:17:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56044b0c6db0 con 0x56044b18c7d0 2023-10-08T09:17:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.883+0000 7f01849f7700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b18c7d0 0x56044b18ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.884+0000 7f0168ff9700 10 client.25769 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.884+0000 7f01849f7700 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b18c7d0 0x56044b18ebf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327e83ded60df28e server_cookie=d11d341b641e8970 in_seq=0 out_seq=0 2023-10-08T09:17:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0168ff9700 1 -- 172.21.15.49:0/3305157638 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56044b0c6db0 con 0x56044b18c7d0 2023-10-08T09:17:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0168ff9700 10 client.25769 handle_command_reply: tid=0 2023-10-08T09:17:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 client.25769 shutdown 2023-10-08T09:17:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 msgr2=0x56044b186a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b186a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f017c09b340 tx=0x7f017c09bb00 comp rx=0 tx=0).stop 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 shutdown_connections 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b18c7d0 0x56044b18ebf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b1839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b186a60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 --2- 172.21.15.49:0/3305157638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b186fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 >> 172.21.15.49:0/3305157638 conn(0x56044b0ef200 msgr2=0x56044b175010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0168ff9700 0 client.25769 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 shutdown_connections 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 1 -- 172.21.15.49:0/3305157638 wait complete. 2023-10-08T09:17:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.885+0000 7f0193645b80 20 client.25769 trim_cache size 0 max 16384 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.908+0000 7f0193645b80 1 Processor -- start 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.908+0000 7f0193645b80 1 -- start start 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f0193645b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b1839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f0193645b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b171c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f0193645b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b17f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f0193645b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044af4e8c0 con 0x56044b17c2b0 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f0193645b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b183080 con 0x56044b17ae40 2023-10-08T09:17:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f0193645b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b184000 con 0x56044b183550 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b1839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b1839b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40762/0 (socket says 172.21.15.49:40762) 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 -- 172.21.15.49:0/2747962580 learned_addr learned my addr 172.21.15.49:0/2747962580 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01849f7700 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b171c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01859f9700 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b17f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 -- 172.21.15.49:0/2747962580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 msgr2=0x56044b17f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b17f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 -- 172.21.15.49:0/2747962580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 msgr2=0x56044b171c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b171c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 -- 172.21.15.49:0/2747962580 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56044af1f1f0 con 0x56044b17ae40 2023-10-08T09:17:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01849f7700 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b171c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:17:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.909+0000 7f01851f8700 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b1839b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f017c097900 tx=0x7f017c00f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92555887f924da6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:45.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.910+0000 7f016bfff700 1 -- 172.21.15.49:0/2747962580 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c0092a0 con 0x56044b17ae40 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.910+0000 7f016bfff700 1 -- 172.21.15.49:0/2747962580 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017c09f550 con 0x56044b17ae40 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.910+0000 7f016bfff700 1 -- 172.21.15.49:0/2747962580 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c0998a0 con 0x56044b17ae40 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 -- 172.21.15.49:0/2747962580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 msgr2=0x56044b1839b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b1839b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f017c097900 tx=0x7f017c00f8d0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 -- 172.21.15.49:0/2747962580 shutdown_connections 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b183550 0x56044b17f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b17ae40 0x56044b1839b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 --2- 172.21.15.49:0/2747962580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b171c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 -- 172.21.15.49:0/2747962580 >> 172.21.15.49:0/2747962580 conn(0x56044b0ef200 msgr2=0x56044b1aa240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 -- 172.21.15.49:0/2747962580 shutdown_connections 2023-10-08T09:17:45.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 -- 172.21.15.49:0/2747962580 wait complete. 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 -- start start 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.911+0000 7f0193645b80 1 Processor -- start 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f0193645b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b1923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f0193645b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b1928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f0193645b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b183550 0x56044b1938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f0193645b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b193de0 con 0x56044b17c2b0 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f0193645b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b193f50 con 0x56044b183550 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f0193645b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56044b194110 con 0x56044b17ae40 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f0193645b80 10 client.0 _ll_register_callbacks cb 0x56044aec77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01849f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b1928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01849f7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b1928e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51154/0 (socket says 172.21.15.49:51154) 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01849f7700 1 -- 172.21.15.49:0/803113371 learned_addr learned my addr 172.21.15.49:0/803113371 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01851f8700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b1923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01859f9700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b183550 0x56044b1938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01851f8700 1 -- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b183550 msgr2=0x56044b1938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01851f8700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b183550 0x56044b1938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01851f8700 1 -- 172.21.15.49:0/803113371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 msgr2=0x56044b1928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01851f8700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b1928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01851f8700 1 -- 172.21.15.49:0/803113371 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56044aee1c00 con 0x56044b17ae40 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.912+0000 7f01849f7700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b1928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f01851f8700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b1923a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f017c09b390 tx=0x7f017c09a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b458b45185ec9a3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0168ff9700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:45.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0168ff9700 1 -- 172.21.15.49:0/803113371 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c0092a0 con 0x56044b17ae40 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0168ff9700 1 -- 172.21.15.49:0/803113371 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f017c09b400 con 0x56044b17ae40 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f017c0ce050 con 0x56044b17ae40 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0168ff9700 1 -- 172.21.15.49:0/803113371 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f017c099880 con 0x56044b17ae40 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0193645b80 10 client.25798 fetch_fsmap learned FSMap version 33 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56044aef7250 con 0x56044b17ae40 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.913+0000 7f0168ff9700 1 -- 172.21.15.49:0/803113371 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017c0ce050 con 0x56044b17ae40 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0168ff9700 1 -- 172.21.15.49:0/803113371 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 33) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f017c0a0290 con 0x56044b17ae40 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0193645b80 10 client.25798 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0193645b80 10 client.25798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0193645b80 20 client.25798 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:45.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0193645b80 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b1f5870 0x56044b1f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0193645b80 4 client.25798 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:17:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56044b0c6ad0 con 0x56044b1f5870 2023-10-08T09:17:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f01849f7700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b1f5870 0x56044b1f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f0168ff9700 10 client.25798 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:45.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.914+0000 7f01849f7700 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b1f5870 0x56044b1f7c90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d1131e6b3ad998 server_cookie=f2047ec0c22d312d in_seq=0 out_seq=0 2023-10-08T09:17:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0168ff9700 1 -- 172.21.15.49:0/803113371 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56044b0c6ad0 con 0x56044b1f5870 2023-10-08T09:17:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0168ff9700 10 client.25798 handle_command_reply: tid=0 2023-10-08T09:17:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 client.25798 shutdown 2023-10-08T09:17:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 msgr2=0x56044b1923a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b1923a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f017c09b390 tx=0x7f017c09a590 comp rx=0 tx=0).stop 2023-10-08T09:17:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 shutdown_connections 2023-10-08T09:17:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56044b1f5870 0x56044b1f7c90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56044b17ae40 0x56044b1923a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0168ff9700 0 client.25798 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56044b183550 0x56044b1938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 --2- 172.21.15.49:0/803113371 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56044b17c2b0 0x56044b1928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 >> 172.21.15.49:0/803113371 conn(0x56044b0ef200 msgr2=0x56044b171c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 shutdown_connections 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 1 -- 172.21.15.49:0/803113371 wait complete. 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.929+0000 7f0193645b80 20 client.25798 trim_cache size 0 max 16384 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:17:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:17:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:17:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:17:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f016c0a1480 msgr2=0x7f016c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f016c0a1480 0x7f016c0a3940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f017000b790 tx=0x7f0170005e80 comp rx=0 tx=0).stop 2023-10-08T09:17:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010b230 msgr2=0x7f018019e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010b230 0x7f018019e070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f017c006fd0 tx=0x7f017c00b730 comp rx=0 tx=0).stop 2023-10-08T09:17:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 shutdown_connections 2023-10-08T09:17:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f016c0a1480 0x7f016c0a3940 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f01801a3520 0x7f01801a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f018010b230 0x7f018019e070 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f017c006fd0 tx=0x7f017c00b730 comp rx=0 tx=0).stop 2023-10-08T09:17:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 --2- 172.21.15.49:0/748265583 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f018010db60 0x7f018019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.931+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 >> 172.21.15.49:0/748265583 conn(0x7f018006d930 msgr2=0x7f0180109320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.932+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 shutdown_connections 2023-10-08T09:17:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:45.932+0000 7f018745c700 1 -- 172.21.15.49:0/748265583 wait complete. 2023-10-08T09:17:47.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:47 smithi132 ceph-mon[103389]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2023-10-08T09:17:47.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:47 smithi049 ceph-mon[99485]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2023-10-08T09:17:47.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:47 smithi184 ceph-mon[103675]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2023-10-08T09:17:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 145.0 s, 1530.4 tps, lat 64.951 ms stddev 79.508 2023-10-08T09:17:49.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:49 smithi132 ceph-mon[103389]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2023-10-08T09:17:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:49 smithi049 ceph-mon[99485]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2023-10-08T09:17:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:49 smithi184 ceph-mon[103675]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 1.09k op/s 2023-10-08T09:17:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:50 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:17:50.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:50 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:17:50.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:50 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:17:51.796 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:51 smithi132 ceph-mon[103389]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 584 op/s 2023-10-08T09:17:51.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:51 smithi049 ceph-mon[99485]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 584 op/s 2023-10-08T09:17:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:51 smithi184 ceph-mon[103675]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 584 op/s 2023-10-08T09:17:53.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:53 smithi132 ceph-mon[103389]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 640 op/s 2023-10-08T09:17:53.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:53 smithi049 ceph-mon[99485]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 640 op/s 2023-10-08T09:17:53.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:53 smithi184 ceph-mon[103675]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 640 op/s 2023-10-08T09:17:54.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 150.0 s, 1206.8 tps, lat 82.712 ms stddev 91.621 2023-10-08T09:17:55.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:55 smithi132 ceph-mon[103389]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 615 op/s 2023-10-08T09:17:55.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:55 smithi049 ceph-mon[99485]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 615 op/s 2023-10-08T09:17:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:55 smithi184 ceph-mon[103675]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 615 op/s 2023-10-08T09:17:57.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:57 smithi049 ceph-mon[99485]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 672 op/s 2023-10-08T09:17:57.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:57 smithi184 ceph-mon[103675]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 672 op/s 2023-10-08T09:17:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:57 smithi132 ceph-mon[103389]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 672 op/s 2023-10-08T09:17:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.092+0000 7f53f26c4700 1 -- 172.21.15.49:0/2209116799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec1063f0 msgr2=0x7f53ec108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.092+0000 7f53f26c4700 1 --2- 172.21.15.49:0/2209116799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec1063f0 0x7f53ec108890 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f53e400b0a0 tx=0x7f53e400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 -- 172.21.15.49:0/2209116799 shutdown_connections 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 --2- 172.21.15.49:0/2209116799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec1063f0 0x7f53ec108890 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 --2- 172.21.15.49:0/2209116799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53ec0fe4c0 0x7f53ec0fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 --2- 172.21.15.49:0/2209116799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec0fee20 0x7f53ec105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 -- 172.21.15.49:0/2209116799 >> 172.21.15.49:0/2209116799 conn(0x7f53ec0f9ef0 msgr2=0x7f53ec0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 -- 172.21.15.49:0/2209116799 shutdown_connections 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 -- 172.21.15.49:0/2209116799 wait complete. 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 Processor -- start 2023-10-08T09:17:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.093+0000 7f53f26c4700 1 -- start start 2023-10-08T09:17:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f26c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec0fe4c0 0x7f53ec101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f26c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53ec0fee20 0x7f53ec0ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f26c4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec1063f0 0x7f53ec1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f26c4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec100a90 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f26c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec100c00 con 0x7f53ec0fee20 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f26c4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1a07e0 con 0x7f53ec1063f0 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec0fe4c0 0x7f53ec101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec0fe4c0 0x7f53ec101720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51166/0 (socket says 172.21.15.49:51166) 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53ec0fee20 0x7f53ec0ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 -- 172.21.15.49:0/3023741786 learned_addr learned my addr 172.21.15.49:0/3023741786 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f0c61700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec1063f0 0x7f53ec1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 -- 172.21.15.49:0/3023741786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec1063f0 msgr2=0x7f53ec1002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec1063f0 0x7f53ec1002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 -- 172.21.15.49:0/3023741786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53ec0fee20 msgr2=0x7f53ec0ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53ec0fee20 0x7f53ec0ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 -- 172.21.15.49:0/3023741786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e4009d00 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53f0c61700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec1063f0 0x7f53ec1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.094+0000 7f53ebfff700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec0fe4c0 0x7f53ec101720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e9a0 tx=0x7f53dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.095+0000 7f53e97fa700 1 -- 172.21.15.49:0/3023741786 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc009790 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.095+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec1a0a70 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.095+0000 7f53e97fa700 1 -- 172.21.15.49:0/3023741786 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53dc00ce70 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.095+0000 7f53e97fa700 1 -- 172.21.15.49:0/3023741786 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc018420 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.095+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec1a0fd0 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.099+0000 7f53fe8adb80 1 Processor -- start 2023-10-08T09:17:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.100+0000 7f53e97fa700 1 -- 172.21.15.49:0/3023741786 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f53dc0106f0 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.100+0000 7f53fe8adb80 1 -- start start 2023-10-08T09:17:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.100+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83dee40 0x5635f83df260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83e0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f8347400 con 0x5635f83e02b0 2023-10-08T09:17:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f8338bc0 con 0x5635f83df7a0 2023-10-08T09:17:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f83364e0 con 0x5635f83dee40 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83e0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51172/0 (socket says 172.21.15.49:51172) 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53eb7fe700 1 -- 172.21.15.49:0/2662397617 learned_addr learned my addr 172.21.15.49:0/2662397617 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.101+0000 7f53ebfff700 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.102+0000 7f53eb7fe700 1 -- 172.21.15.49:0/2662397617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83dee40 msgr2=0x5635f83df260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.102+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83dee40 0x5635f83df260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.102+0000 7f53eb7fe700 1 -- 172.21.15.49:0/2662397617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 msgr2=0x5635f83dfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.102+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83dfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.102+0000 7f53eb7fe700 1 -- 172.21.15.49:0/2662397617 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635f815cd80 con 0x5635f83e02b0 2023-10-08T09:17:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.102+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83e0730 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009d20 tx=0x7f53e0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab98a3f0b6063239 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.104+0000 7f53d27fc700 1 -- 172.21.15.49:0/2662397617 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0004a30 con 0x5635f83e02b0 2023-10-08T09:17:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.105+0000 7f53d27fc700 1 -- 172.21.15.49:0/2662397617 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e00054e0 con 0x5635f83e02b0 2023-10-08T09:17:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.105+0000 7f53d27fc700 1 -- 172.21.15.49:0/2662397617 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0019860 con 0x5635f83e02b0 2023-10-08T09:17:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.110+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2662397617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 msgr2=0x5635f83e0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.110+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83e0730 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009d20 tx=0x7f53e0011720 comp rx=0 tx=0).stop 2023-10-08T09:17:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.110+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2662397617 shutdown_connections 2023-10-08T09:17:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.110+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83dee40 0x5635f83df260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.110+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83e0730 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.110+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2662397617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83dfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.110+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2662397617 >> 172.21.15.49:0/2662397617 conn(0x5635f8353200 msgr2=0x5635f83d98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.112+0000 7f53e97fa700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d40a1480 0x7f53d40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.112+0000 7f53e97fa700 1 -- 172.21.15.49:0/3023741786 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f53dc014070 con 0x7f53ec0fe4c0 2023-10-08T09:17:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.115+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2662397617 shutdown_connections 2023-10-08T09:17:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.123+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d40a1480 0x7f53d40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.123+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2662397617 wait complete. 2023-10-08T09:17:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.124+0000 7f53fe8adb80 1 -- start start 2023-10-08T09:17:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.124+0000 7f53fe8adb80 1 Processor -- start 2023-10-08T09:17:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.125+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83dee40 0x5635f83f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.125+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.125+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83f8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.125+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f8331dd0 con 0x5635f83e02b0 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.125+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f82ce6b0 con 0x5635f83dee40 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.125+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f82ce350 con 0x5635f83df7a0 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.125+0000 7f53fe8adb80 10 client.0 _ll_register_callbacks cb 0x5635f812b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53f0c61700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53f0c61700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83f8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51178/0 (socket says 172.21.15.49:51178) 2023-10-08T09:17:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53f0c61700 1 -- 172.21.15.49:0/4065004293 learned_addr learned my addr 172.21.15.49:0/4065004293 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53ebfff700 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83dee40 0x5635f83f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d40a1480 0x7f53d40a3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53e0004f30 tx=0x7f53e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53ebfff700 1 -- 172.21.15.49:0/4065004293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 msgr2=0x5635f83f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53ebfff700 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53ebfff700 1 -- 172.21.15.49:0/4065004293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 msgr2=0x5635f83f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53ebfff700 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.126+0000 7f53ebfff700 1 -- 172.21.15.49:0/4065004293 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e0009980 con 0x5635f83dee40 2023-10-08T09:17:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.130+0000 7f53ebfff700 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83dee40 0x5635f83f11b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0ca9f0 tx=0x7f53dc09a700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e24be70c448f9a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.133+0000 7f53cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:17:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.133+0000 7f53cb7fe700 1 -- 172.21.15.49:0/4065004293 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc09e150 con 0x5635f83dee40 2023-10-08T09:17:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.133+0000 7f53cb7fe700 1 -- 172.21.15.49:0/4065004293 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f53dc09a8c0 con 0x5635f83dee40 2023-10-08T09:17:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.133+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635f82f9e50 con 0x5635f83dee40 2023-10-08T09:17:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.136+0000 7f53cb7fe700 1 -- 172.21.15.49:0/4065004293 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53dc09a1c0 con 0x5635f83dee40 2023-10-08T09:17:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.138+0000 7f53fe8adb80 10 client.25787 fetch_fsmap learned FSMap version 34 2023-10-08T09:17:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.138+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5635f8145c00 con 0x5635f83dee40 2023-10-08T09:17:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.148+0000 7f53cb7fe700 1 -- 172.21.15.49:0/4065004293 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635f82f9e50 con 0x5635f83dee40 2023-10-08T09:17:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.148+0000 7f53cb7fe700 1 -- 172.21.15.49:0/4065004293 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f53dc0a20c0 con 0x5635f83dee40 2023-10-08T09:17:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.152+0000 7f53fe8adb80 10 client.25787 fetch_fsmap finished waiting for FSMap version 34 2023-10-08T09:17:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.152+0000 7f53fe8adb80 10 client.25787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.152+0000 7f53fe8adb80 20 client.25787 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.152+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83fcea0 0x5635f83ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.152+0000 7f53fe8adb80 4 client.25787 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:17:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.152+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635f832adb0 con 0x5635f83fcea0 2023-10-08T09:17:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.153+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83fcea0 0x5635f83ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.153+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83fcea0 0x5635f83ff2c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fb77283cab9c77 server_cookie=b1c0fc912309f86f in_seq=0 out_seq=0 2023-10-08T09:17:59.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.153+0000 7f53cb7fe700 10 client.25787 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.154+0000 7f53cb7fe700 1 -- 172.21.15.49:0/4065004293 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635f832adb0 con 0x5635f83fcea0 2023-10-08T09:17:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.154+0000 7f53cb7fe700 10 client.25787 handle_command_reply: tid=0 2023-10-08T09:17:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.154+0000 7f53fe8adb80 1 client.25787 shutdown 2023-10-08T09:17:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.154+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83dee40 msgr2=0x5635f83f11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.154+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83dee40 0x5635f83f11b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0ca9f0 tx=0x7f53dc09a700 comp rx=0 tx=0).stop 2023-10-08T09:17:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 shutdown_connections 2023-10-08T09:17:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83fcea0 0x5635f83ff2c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83e02b0 0x5635f83f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83dee40 0x5635f83f11b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/4065004293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 >> 172.21.15.49:0/4065004293 conn(0x5635f8353200 msgr2=0x5635f83d9010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53cb7fe700 0 client.25787 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.155+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 shutdown_connections 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.156+0000 7f53fe8adb80 1 -- 172.21.15.49:0/4065004293 wait complete. 2023-10-08T09:17:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.156+0000 7f53fe8adb80 20 client.25787 trim_cache size 0 max 16384 2023-10-08T09:17:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.181+0000 7f53fe8adb80 1 Processor -- start 2023-10-08T09:17:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53fe8adb80 1 -- start start 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83e6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83f8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83e02b0 0x5635f83e2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f81b28c0 con 0x5635f83dee40 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f82f9e50 con 0x5635f83df7a0 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f83d6830 con 0x5635f83e02b0 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53f0c61700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83e02b0 0x5635f83e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83f8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40802/0 (socket says 172.21.15.49:40802) 2023-10-08T09:17:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.182+0000 7f53eb7fe700 1 -- 172.21.15.49:0/1055736631 learned_addr learned my addr 172.21.15.49:0/1055736631 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.183+0000 7f53f0c61700 1 -- 172.21.15.49:0/1055736631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 msgr2=0x5635f83f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.183+0000 7f53f0c61700 1 --2- 172.21.15.49:0/1055736631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.183+0000 7f53f0c61700 1 -- 172.21.15.49:0/1055736631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 msgr2=0x5635f83e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.183+0000 7f53f0c61700 1 --2- 172.21.15.49:0/1055736631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.183+0000 7f53f0c61700 1 -- 172.21.15.49:0/1055736631 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635f81831f0 con 0x5635f83e02b0 2023-10-08T09:17:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.183+0000 7f53f0c61700 1 --2- 172.21.15.49:0/1055736631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83e02b0 0x5635f83e2800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009fd0 tx=0x7f53e4009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55b3b871bd8113d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53d2ffd700 1 -- 172.21.15.49:0/1055736631 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400e040 con 0x5635f83e02b0 2023-10-08T09:17:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53d2ffd700 1 -- 172.21.15.49:0/1055736631 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e4021070 con 0x5635f83e02b0 2023-10-08T09:17:59.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53d2ffd700 1 -- 172.21.15.49:0/1055736631 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40266d0 con 0x5635f83e02b0 2023-10-08T09:17:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53fe8adb80 1 -- 172.21.15.49:0/1055736631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83e02b0 msgr2=0x5635f83e2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/1055736631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83e02b0 0x5635f83e2800 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009fd0 tx=0x7f53e4009c70 comp rx=0 tx=0).stop 2023-10-08T09:17:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53fe8adb80 1 -- 172.21.15.49:0/1055736631 shutdown_connections 2023-10-08T09:17:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/1055736631 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83e02b0 0x5635f83e2800 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/1055736631 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/1055736631 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83df7a0 0x5635f83f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.185+0000 7f53fe8adb80 1 -- 172.21.15.49:0/1055736631 >> 172.21.15.49:0/1055736631 conn(0x5635f8353200 msgr2=0x5635f8403c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 -- 172.21.15.49:0/1055736631 shutdown_connections 2023-10-08T09:17:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 -- 172.21.15.49:0/1055736631 wait complete. 2023-10-08T09:17:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 -- start start 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 Processor -- start 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83e02b0 0x5635f83f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f83f8020 con 0x5635f83dee40 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f83f8190 con 0x5635f83e02b0 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.186+0000 7f53fe8adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f83f8300 con 0x5635f83df7a0 2023-10-08T09:17:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53fe8adb80 10 client.0 _ll_register_callbacks cb 0x7f53dc096500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51192/0 (socket says 172.21.15.49:51192) 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53ebfff700 1 -- 172.21.15.49:0/2158281891 learned_addr learned my addr 172.21.15.49:0/2158281891 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53f0c61700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83e02b0 0x5635f83f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53eb7fe700 1 -- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83e02b0 msgr2=0x5635f83f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83e02b0 0x5635f83f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53eb7fe700 1 -- 172.21.15.49:0/2158281891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 msgr2=0x5635f83f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53eb7fe700 1 -- 172.21.15.49:0/2158281891 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec1094b0 con 0x5635f83df7a0 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53ebfff700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:17:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53eb7fe700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f2d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5635f83efbe0 tx=0x7f53e00098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b2ddc481326182 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.187+0000 7f53cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53cb7fe700 1 -- 172.21.15.49:0/2158281891 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00342b0 con 0x5635f83df7a0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53cb7fe700 1 -- 172.21.15.49:0/2158281891 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f53e002a040 con 0x5635f83df7a0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e00054e0 con 0x5635f83df7a0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53cb7fe700 1 -- 172.21.15.49:0/2158281891 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e0034410 con 0x5635f83df7a0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 10 client.25810 fetch_fsmap learned FSMap version 34 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5635f815b250 con 0x5635f83df7a0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53cb7fe700 1 -- 172.21.15.49:0/2158281891 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e00054e0 con 0x5635f83df7a0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53cb7fe700 1 -- 172.21.15.49:0/2158281891 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 34) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f53e003d470 con 0x5635f83df7a0 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 10 client.25810 fetch_fsmap finished waiting for FSMap version 34 2023-10-08T09:17:59.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 10 client.25810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:17:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 20 client.25810 populate_metadata read hostname 'smithi049' 2023-10-08T09:17:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83ea340 0x5635f83ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:17:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 4 client.25810 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:17:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.188+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635f832aad0 con 0x5635f83ea340 2023-10-08T09:17:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.189+0000 7f53ebfff700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83ea340 0x5635f83ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:17:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.189+0000 7f53ebfff700 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83ea340 0x5635f83ec760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f21bfa7c504f7 server_cookie=8b1d05ab104c7eab in_seq=0 out_seq=0 2023-10-08T09:17:59.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.189+0000 7f53cb7fe700 10 client.25810 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53cb7fe700 1 -- 172.21.15.49:0/2158281891 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635f832aad0 con 0x5635f83ea340 2023-10-08T09:17:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53cb7fe700 10 client.25810 handle_command_reply: tid=0 2023-10-08T09:17:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 client.25810 shutdown 2023-10-08T09:17:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 msgr2=0x5635f83f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f2d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x5635f83efbe0 tx=0x7f53e00098e0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 shutdown_connections 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635f83ea340 0x5635f83ec760 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635f83e02b0 0x5635f83f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53cb7fe700 0 client.25810 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635f83dee40 0x5635f83f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 --2- 172.21.15.49:0/2158281891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635f83df7a0 0x5635f83f2d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 >> 172.21.15.49:0/2158281891 conn(0x5635f8353200 msgr2=0x5635f83e6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 shutdown_connections 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 1 -- 172.21.15.49:0/2158281891 wait complete. 2023-10-08T09:17:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.201+0000 7f53fe8adb80 20 client.25810 trim_cache size 0 max 16384 2023-10-08T09:17:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:17:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:17:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:17:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d40a1480 msgr2=0x7f53d40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d40a1480 0x7f53d40a3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f53e0004f30 tx=0x7f53e0000bc0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec0fe4c0 msgr2=0x7f53ec101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec0fe4c0 0x7f53ec101720 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00e9a0 tx=0x7f53dc00ecb0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 shutdown_connections 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53d40a1480 0x7f53d40a3940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53ec1063f0 0x7f53ec1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53ec0fe4c0 0x7f53ec101720 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 --2- 172.21.15.49:0/3023741786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53ec0fee20 0x7f53ec0ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 >> 172.21.15.49:0/3023741786 conn(0x7f53ec0f9ef0 msgr2=0x7f53ec101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:17:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 shutdown_connections 2023-10-08T09:17:59.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:17:59.202+0000 7f53f26c4700 1 -- 172.21.15.49:0/3023741786 wait complete. 2023-10-08T09:17:59.313 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 155.0 s, 1383.7 tps, lat 72.571 ms stddev 92.005 2023-10-08T09:17:59.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:17:59 smithi049 ceph-mon[99485]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 670 op/s 2023-10-08T09:17:59.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:17:59 smithi184 ceph-mon[103675]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 670 op/s 2023-10-08T09:18:00.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:17:59 smithi132 ceph-mon[103389]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 670 op/s 2023-10-08T09:18:01.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:01 smithi049 ceph-mon[99485]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 615 op/s 2023-10-08T09:18:01.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:01 smithi184 ceph-mon[103675]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 615 op/s 2023-10-08T09:18:02.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:01 smithi132 ceph-mon[103389]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 615 op/s 2023-10-08T09:18:03.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:03 smithi049 ceph-mon[99485]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 695 op/s 2023-10-08T09:18:03.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:03 smithi184 ceph-mon[103675]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 695 op/s 2023-10-08T09:18:04.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:03 smithi132 ceph-mon[103389]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.4 MiB/s wr, 695 op/s 2023-10-08T09:18:04.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 160.0 s, 1209.2 tps, lat 82.832 ms stddev 97.814 2023-10-08T09:18:05.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:05 smithi049 ceph-mon[99485]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 MiB/s wr, 697 op/s 2023-10-08T09:18:05.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:05 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:05 smithi184 ceph-mon[103675]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 MiB/s wr, 697 op/s 2023-10-08T09:18:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:05 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:05 smithi132 ceph-mon[103389]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 9.3 MiB/s wr, 697 op/s 2023-10-08T09:18:06.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:05 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:07.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:07 smithi184 ceph-mon[103675]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 763 op/s 2023-10-08T09:18:08.054 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:07 smithi132 ceph-mon[103389]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 763 op/s 2023-10-08T09:18:08.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:07 smithi049 ceph-mon[99485]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 763 op/s 2023-10-08T09:18:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 165.0 s, 1180.0 tps, lat 83.999 ms stddev 97.633 2023-10-08T09:18:10.003 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:09 smithi184 ceph-mon[103675]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 MiB/s wr, 646 op/s 2023-10-08T09:18:10.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:09 smithi132 ceph-mon[103389]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 MiB/s wr, 646 op/s 2023-10-08T09:18:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:09 smithi049 ceph-mon[99485]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 MiB/s wr, 646 op/s 2023-10-08T09:18:11.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:18:11.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.907+0000 7fa43fd19700 1 -- 172.21.15.49:0/2990819770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4381077f0 msgr2=0x7fa438109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:11.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.907+0000 7fa43fd19700 1 --2- 172.21.15.49:0/2990819770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4381077f0 0x7fa438109c20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa434009990 tx=0x7fa434009ca0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.909+0000 7fa43fd19700 1 -- 172.21.15.49:0/2990819770 shutdown_connections 2023-10-08T09:18:11.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.909+0000 7fa43fd19700 1 --2- 172.21.15.49:0/2990819770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4381077f0 0x7fa438109c20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.909+0000 7fa43fd19700 1 --2- 172.21.15.49:0/2990819770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa43810cb30 0x7fa43810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.909+0000 7fa43fd19700 1 --2- 172.21.15.49:0/2990819770 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa43810a160 0x7fa43810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.909+0000 7fa43fd19700 1 -- 172.21.15.49:0/2990819770 >> 172.21.15.49:0/2990819770 conn(0x7fa43806d790 msgr2=0x7fa43806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:11.911 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.911+0000 7fa43fd19700 1 -- 172.21.15.49:0/2990819770 shutdown_connections 2023-10-08T09:18:11.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.911+0000 7fa43fd19700 1 -- 172.21.15.49:0/2990819770 wait complete. 2023-10-08T09:18:11.912 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.911+0000 7fa43fd19700 1 Processor -- start 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43fd19700 1 -- start start 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43fd19700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4381077f0 0x7fa4381a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43fd19700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa43810a160 0x7fa4381a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43fd19700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa43810cb30 0x7fa4381acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43fd19700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4381ad070 con 0x7fa43810cb30 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43fd19700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43810ef20 con 0x7fa4381077f0 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43fd19700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43810f0b0 con 0x7fa43810a160 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43dab5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4381077f0 0x7fa4381a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43dab5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4381077f0 0x7fa4381a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40820/0 (socket says 172.21.15.49:40820) 2023-10-08T09:18:11.913 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43dab5700 1 -- 172.21.15.49:0/772343920 learned_addr learned my addr 172.21.15.49:0/772343920 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.912+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa43810a160 0x7fa4381a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.913+0000 7fa43dab5700 1 -- 172.21.15.49:0/772343920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa43810a160 msgr2=0x7fa4381a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.913+0000 7fa43dab5700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa43810a160 0x7fa4381a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.913+0000 7fa43dab5700 1 -- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa43810cb30 msgr2=0x7fa4381acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:11.914 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.913+0000 7fa43dab5700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa43810cb30 0x7fa4381acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.913+0000 7fa43dab5700 1 -- 172.21.15.49:0/772343920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340095f0 con 0x7fa4381077f0 2023-10-08T09:18:11.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.914+0000 7fa43dab5700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4381077f0 0x7fa4381a5540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa43400b5c0 tx=0x7fa434003b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:11.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.921+0000 7fa42effd700 1 -- 172.21.15.49:0/772343920 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43401d070 con 0x7fa4381077f0 2023-10-08T09:18:11.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.921+0000 7fa42effd700 1 -- 172.21.15.49:0/772343920 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa434003e60 con 0x7fa4381077f0 2023-10-08T09:18:11.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.921+0000 7fa42effd700 1 -- 172.21.15.49:0/772343920 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400fdb0 con 0x7fa4381077f0 2023-10-08T09:18:11.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.922+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43810f330 con 0x7fa4381077f0 2023-10-08T09:18:11.923 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.922+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43810f760 con 0x7fa4381077f0 2023-10-08T09:18:11.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.926+0000 7fa42effd700 1 -- 172.21.15.49:0/772343920 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa43400f460 con 0x7fa4381077f0 2023-10-08T09:18:11.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.927+0000 7fa42effd700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4240a1600 0x7fa4240a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.927+0000 7fa42effd700 1 -- 172.21.15.49:0/772343920 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa4340ce1a0 con 0x7fa4381077f0 2023-10-08T09:18:11.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.928+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4240a1600 0x7fa4240a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.931+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4240a1600 0x7fa4240a3ac0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa428009a10 tx=0x7fa428009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:11.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.936+0000 7fa44bf02b80 1 Processor -- start 2023-10-08T09:18:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa44bf02b80 1 -- start start 2023-10-08T09:18:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 0x55af24352480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24353270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24353c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af242ba460 con 0x55af24353800 2023-10-08T09:18:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af242abc20 con 0x55af243529c0 2023-10-08T09:18:11.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af242a9540 con 0x55af24352060 2023-10-08T09:18:11.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa43d2b4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24353270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa43d2b4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24353270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40828/0 (socket says 172.21.15.49:40828) 2023-10-08T09:18:11.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.943+0000 7fa43d2b4700 1 -- 172.21.15.49:0/2527462933 learned_addr learned my addr 172.21.15.49:0/2527462933 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.944+0000 7fa43d2b4700 1 -- 172.21.15.49:0/2527462933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 msgr2=0x55af24352480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.944+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/2527462933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 0x55af24352480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.944+0000 7fa43d2b4700 1 -- 172.21.15.49:0/2527462933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 msgr2=0x55af24353c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.944+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/2527462933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24353c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.944+0000 7fa43d2b4700 1 -- 172.21.15.49:0/2527462933 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af240cfde0 con 0x55af243529c0 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.945+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/2527462933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24353270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa42800ea00 tx=0x7fa428011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8902cd8389fc0a98 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.945+0000 7fa423fff700 1 -- 172.21.15.49:0/2527462933 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428015480 con 0x55af243529c0 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.945+0000 7fa423fff700 1 -- 172.21.15.49:0/2527462933 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa428015aa0 con 0x55af243529c0 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.945+0000 7fa423fff700 1 -- 172.21.15.49:0/2527462933 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428013a90 con 0x55af243529c0 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.946+0000 7fa44bf02b80 1 -- 172.21.15.49:0/2527462933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 msgr2=0x55af24353270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:11.947 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.946+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/2527462933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24353270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa42800ea00 tx=0x7fa428011000 comp rx=0 tx=0).stop 2023-10-08T09:18:11.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.946+0000 7fa44bf02b80 1 -- 172.21.15.49:0/2527462933 shutdown_connections 2023-10-08T09:18:11.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.946+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/2527462933 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 0x55af24352480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.946+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/2527462933 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24353c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.946+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/2527462933 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24353270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.946+0000 7fa44bf02b80 1 -- 172.21.15.49:0/2527462933 >> 172.21.15.49:0/2527462933 conn(0x55af242c6260 msgr2=0x55af2434ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:11.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.947+0000 7fa44bf02b80 1 -- 172.21.15.49:0/2527462933 shutdown_connections 2023-10-08T09:18:11.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.947+0000 7fa44bf02b80 1 -- 172.21.15.49:0/2527462933 wait complete. 2023-10-08T09:18:11.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.947+0000 7fa44bf02b80 1 -- start start 2023-10-08T09:18:11.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.947+0000 7fa44bf02b80 1 Processor -- start 2023-10-08T09:18:11.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24352060 0x55af2435bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af243529c0 0x55af2435c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24363440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af242a4e30 con 0x55af24353800 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af24241710 con 0x55af24352060 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af242413b0 con 0x55af243529c0 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa44bf02b80 10 client.0 _ll_register_callbacks cb 0x55af2409e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa43dab5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24352060 0x55af2435bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa43d2b4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af243529c0 0x55af2435c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa43dab5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24352060 0x55af2435bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40834/0 (socket says 172.21.15.49:40834) 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa43dab5700 1 -- 172.21.15.49:0/1151548223 learned_addr learned my addr 172.21.15.49:0/1151548223 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:11.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa43d2b4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af243529c0 0x55af2435c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38256/0 (socket says 172.21.15.49:38256) 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.948+0000 7fa43e2b6700 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24363440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa43d2b4700 1 -- 172.21.15.49:0/1151548223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24352060 msgr2=0x55af2435bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24352060 0x55af2435bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa43d2b4700 1 -- 172.21.15.49:0/1151548223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 msgr2=0x55af24363440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24363440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa43d2b4700 1 -- 172.21.15.49:0/1151548223 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42800e6e0 con 0x55af243529c0 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa43e2b6700 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24363440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af243529c0 0x55af2435c400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa42800edc0 tx=0x7fa4280156b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605f7abb3768d627 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa4217fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa4217fa700 1 -- 172.21.15.49:0/1151548223 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428013c20 con 0x55af243529c0 2023-10-08T09:18:11.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.949+0000 7fa4217fa700 1 -- 172.21.15.49:0/1151548223 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa428013d80 con 0x55af243529c0 2023-10-08T09:18:11.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.950+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af2426ceb0 con 0x55af243529c0 2023-10-08T09:18:11.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.950+0000 7fa4217fa700 1 -- 172.21.15.49:0/1151548223 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa428032400 con 0x55af243529c0 2023-10-08T09:18:11.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.954+0000 7fa44bf02b80 10 client.25819 fetch_fsmap learned FSMap version 35 2023-10-08T09:18:11.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.954+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55af240b8c60 con 0x55af243529c0 2023-10-08T09:18:11.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa4217fa700 1 -- 172.21.15.49:0/1151548223 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af2426ceb0 con 0x55af243529c0 2023-10-08T09:18:11.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa4217fa700 1 -- 172.21.15.49:0/1151548223 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa428039050 con 0x55af243529c0 2023-10-08T09:18:11.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa44bf02b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 35 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa44bf02b80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa44bf02b80 20 client.25819 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af24367ca0 0x55af2436a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa44bf02b80 4 client.25819 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.956+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af2429de10 con 0x55af24367ca0 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.957+0000 7fa43dab5700 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af24367ca0 0x55af2436a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.957+0000 7fa4217fa700 10 client.25819 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:11.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.957+0000 7fa43dab5700 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af24367ca0 0x55af2436a0c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b2e9cc677641 server_cookie=1445f286a75344d3 in_seq=0 out_seq=0 2023-10-08T09:18:11.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.958+0000 7fa4217fa700 1 -- 172.21.15.49:0/1151548223 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af2429de10 con 0x55af24367ca0 2023-10-08T09:18:11.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.958+0000 7fa4217fa700 10 client.25819 handle_command_reply: tid=0 2023-10-08T09:18:11.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.959+0000 7fa44bf02b80 1 client.25819 shutdown 2023-10-08T09:18:11.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af243529c0 msgr2=0x55af2435c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:11.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af243529c0 0x55af2435c400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa42800edc0 tx=0x7fa4280156b0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 shutdown_connections 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af24367ca0 0x55af2436a0c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24353800 0x55af24363440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24352060 0x55af2435bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa4217fa700 0 client.25819 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1151548223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af243529c0 0x55af2435c400 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 >> 172.21.15.49:0/1151548223 conn(0x55af242c6260 msgr2=0x55af2434c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:11.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 shutdown_connections 2023-10-08T09:18:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1151548223 wait complete. 2023-10-08T09:18:11.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.960+0000 7fa44bf02b80 20 client.25819 trim_cache size 0 max 16384 2023-10-08T09:18:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.987+0000 7fa44bf02b80 1 Processor -- start 2023-10-08T09:18:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.987+0000 7fa44bf02b80 1 -- start start 2023-10-08T09:18:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 0x55af24353be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af243529c0 0x55af24363360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24353800 0x55af2435b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af24125920 con 0x55af243529c0 2023-10-08T09:18:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af2435a770 con 0x55af24353800 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af2435ba80 con 0x55af24352060 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af243529c0 0x55af24363360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af243529c0 0x55af24363360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51228/0 (socket says 172.21.15.49:51228) 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 -- 172.21.15.49:0/1404804297 learned_addr learned my addr 172.21.15.49:0/1404804297 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43e2b6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24353800 0x55af2435b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 -- 172.21.15.49:0/1404804297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 msgr2=0x55af24353be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/1404804297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 0x55af24353be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 -- 172.21.15.49:0/1404804297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24353800 msgr2=0x55af2435b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/1404804297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24353800 0x55af2435b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 -- 172.21.15.49:0/1404804297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af240f6250 con 0x55af243529c0 2023-10-08T09:18:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa43d2b4700 1 --2- 172.21.15.49:0/1404804297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af243529c0 0x55af24363360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa42803c8a0 tx=0x7fa428006b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affc8a4f19c9298b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.988+0000 7fa42cff9700 1 -- 172.21.15.49:0/1404804297 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428009200 con 0x55af243529c0 2023-10-08T09:18:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa42cff9700 1 -- 172.21.15.49:0/1404804297 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa42802c070 con 0x55af243529c0 2023-10-08T09:18:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa42cff9700 1 -- 172.21.15.49:0/1404804297 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428013a20 con 0x55af243529c0 2023-10-08T09:18:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1404804297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af243529c0 msgr2=0x55af24363360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1404804297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af243529c0 0x55af24363360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa42803c8a0 tx=0x7fa428006b60 comp rx=0 tx=0).stop 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1404804297 shutdown_connections 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1404804297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24352060 0x55af24353be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1404804297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af24353800 0x55af2435b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1404804297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af243529c0 0x55af24363360 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1404804297 >> 172.21.15.49:0/1404804297 conn(0x55af242c6260 msgr2=0x55af24348f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1404804297 shutdown_connections 2023-10-08T09:18:11.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.989+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1404804297 wait complete. 2023-10-08T09:18:11.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.990+0000 7fa44bf02b80 1 -- start start 2023-10-08T09:18:11.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.990+0000 7fa44bf02b80 1 Processor -- start 2023-10-08T09:18:11.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.991+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24352060 0x55af243615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.991+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24361b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.991+0000 7fa44bf02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24353800 0x55af24362860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:11.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.991+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af24362e60 con 0x55af24352060 2023-10-08T09:18:11.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.991+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af24362fd0 con 0x55af243529c0 2023-10-08T09:18:11.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.991+0000 7fa44bf02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af24363190 con 0x55af24353800 2023-10-08T09:18:11.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.991+0000 7fa44bf02b80 10 client.0 _ll_register_callbacks cb 0x7fa42800e900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:11.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24353800 0x55af24362860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43d2b4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24361b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:11.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24353800 0x55af24362860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38264/0 (socket says 172.21.15.49:38264) 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 -- 172.21.15.49:0/1931873992 learned_addr learned my addr 172.21.15.49:0/1931873992 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 -- 172.21.15.49:0/1931873992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 msgr2=0x55af24361b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24361b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 -- 172.21.15.49:0/1931873992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24352060 msgr2=0x55af243615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24352060 0x55af243615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.992+0000 7fa43e2b6700 1 -- 172.21.15.49:0/1931873992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af240b8c60 con 0x55af24353800 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.993+0000 7fa43e2b6700 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24353800 0x55af24362860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa430007b20 tx=0x7fa430007e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76575e05ea60015e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.993+0000 7fa4217fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.993+0000 7fa4217fa700 1 -- 172.21.15.49:0/1931873992 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430003c10 con 0x55af24353800 2023-10-08T09:18:11.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.993+0000 7fa4217fa700 1 -- 172.21.15.49:0/1931873992 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa430003d70 con 0x55af24353800 2023-10-08T09:18:12.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.993+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa428015aa0 con 0x55af24353800 2023-10-08T09:18:12.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.994+0000 7fa4217fa700 1 -- 172.21.15.49:0/1931873992 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4300346c0 con 0x55af24353800 2023-10-08T09:18:12.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.994+0000 7fa44bf02b80 10 client.25822 fetch_fsmap learned FSMap version 35 2023-10-08T09:18:12.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.994+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55af240ce2b0 con 0x55af24353800 2023-10-08T09:18:12.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.994+0000 7fa4217fa700 1 -- 172.21.15.49:0/1931873992 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa428015aa0 con 0x55af24353800 2023-10-08T09:18:12.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.994+0000 7fa4217fa700 1 -- 172.21.15.49:0/1931873992 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fa4300044e0 con 0x55af24353800 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa44bf02b80 10 client.25822 fetch_fsmap finished waiting for FSMap version 35 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa44bf02b80 10 client.25822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa44bf02b80 20 client.25822 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af243bcf20 0x55af243bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa44bf02b80 4 client.25822 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af2429db30 con 0x55af243bcf20 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa43dab5700 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af243bcf20 0x55af243bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.001 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa4217fa700 10 client.25822 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.995+0000 7fa43dab5700 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af243bcf20 0x55af243bf340 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92848033a7e9cb38 server_cookie=bc0fd5aff6efa2f8 in_seq=0 out_seq=0 2023-10-08T09:18:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa4217fa700 1 -- 172.21.15.49:0/1931873992 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55af2429db30 con 0x55af243bcf20 2023-10-08T09:18:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa4217fa700 10 client.25822 handle_command_reply: tid=0 2023-10-08T09:18:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 client.25822 shutdown 2023-10-08T09:18:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24353800 msgr2=0x55af24362860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24353800 0x55af24362860 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa430007b20 tx=0x7fa430007e30 comp rx=0 tx=0).stop 2023-10-08T09:18:12.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 shutdown_connections 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55af243bcf20 0x55af243bf340 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55af24353800 0x55af24362860 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55af24352060 0x55af243615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 --2- 172.21.15.49:0/1931873992 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55af243529c0 0x55af24361b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa4217fa700 0 client.25822 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 >> 172.21.15.49:0/1931873992 conn(0x55af242c6260 msgr2=0x55af24348f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 shutdown_connections 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 1 -- 172.21.15.49:0/1931873992 wait complete. 2023-10-08T09:18:12.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.996+0000 7fa44bf02b80 20 client.25822 trim_cache size 0 max 16384 2023-10-08T09:18:12.004 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:18:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:18:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:18:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:18:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:18:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:18:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:18:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:18:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:18:12.005 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:18:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.997+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4240a1600 msgr2=0x7fa4240a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.997+0000 7fa43fd19700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4240a1600 0x7fa4240a3ac0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa428009a10 tx=0x7fa428009190 comp rx=0 tx=0).stop 2023-10-08T09:18:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.997+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4381077f0 msgr2=0x7fa4381a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.997+0000 7fa43fd19700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4381077f0 0x7fa4381a5540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa43400b5c0 tx=0x7fa434003b40 comp rx=0 tx=0).stop 2023-10-08T09:18:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.997+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 shutdown_connections 2023-10-08T09:18:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.997+0000 7fa43fd19700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4240a1600 0x7fa4240a3ac0 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa428009a10 tx=0x7fa428009190 comp rx=0 tx=0).stop 2023-10-08T09:18:12.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.997+0000 7fa43fd19700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa43810cb30 0x7fa4381acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.998+0000 7fa43fd19700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4381077f0 0x7fa4381a5540 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.998+0000 7fa43fd19700 1 --2- 172.21.15.49:0/772343920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa43810a160 0x7fa4381a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.998+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 >> 172.21.15.49:0/772343920 conn(0x7fa43806d790 msgr2=0x7fa43810b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.999+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 shutdown_connections 2023-10-08T09:18:12.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:11.999+0000 7fa43fd19700 1 -- 172.21.15.49:0/772343920 wait complete. 2023-10-08T09:18:12.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:11 smithi184 ceph-mon[103675]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 610 op/s 2023-10-08T09:18:12.032 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:11 smithi049 ceph-mon[99485]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 610 op/s 2023-10-08T09:18:12.033 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:18:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:11 smithi132 ceph-mon[103389]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 610 op/s 2023-10-08T09:18:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.383+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2458995544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44107a60 msgr2=0x7f8c44107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.383+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2458995544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44107a60 0x7f8c44107e80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b0a0 tx=0x7f8c3c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.384+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2458995544 shutdown_connections 2023-10-08T09:18:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.384+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2458995544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c44109340 0x7f8c4410fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.384+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2458995544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44107a60 0x7f8c44107e80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.384+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2458995544 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c44108980 0x7f8c44108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.384+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2458995544 >> 172.21.15.49:0/2458995544 conn(0x7f8c4406d5f0 msgr2=0x7f8c4406fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.385+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2458995544 shutdown_connections 2023-10-08T09:18:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.387+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2458995544 wait complete. 2023-10-08T09:18:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 Processor -- start 2023-10-08T09:18:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 -- start start 2023-10-08T09:18:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c44107a60 0x7f8c4419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44108980 0x7f8c4419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c44109340 0x7f8c441a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c441a6a90 con 0x7f8c44107a60 2023-10-08T09:18:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c441a6bd0 con 0x7f8c44108980 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.388+0000 7f8c4af9e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c441a6d60 con 0x7f8c44109340 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4979b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44108980 0x7f8c4419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4a79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c44109340 0x7f8c441a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c49f9c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c44107a60 0x7f8c4419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4979b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44108980 0x7f8c4419f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40860/0 (socket says 172.21.15.49:40860) 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4979b700 1 -- 172.21.15.49:0/2171178450 learned_addr learned my addr 172.21.15.49:0/2171178450 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4a79d700 1 -- 172.21.15.49:0/2171178450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44108980 msgr2=0x7f8c4419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4a79d700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44108980 0x7f8c4419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4a79d700 1 -- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c44107a60 msgr2=0x7f8c4419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4a79d700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c44107a60 0x7f8c4419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4a79d700 1 -- 172.21.15.49:0/2171178450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c009d00 con 0x7f8c44109340 2023-10-08T09:18:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4979b700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44108980 0x7f8c4419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:18:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.389+0000 7f8c4a79d700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c44109340 0x7f8c441a64c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b3f0 tx=0x7f8c4000b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.391+0000 7f8c3affd700 1 -- 172.21.15.49:0/2171178450 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40011830 con 0x7f8c44109340 2023-10-08T09:18:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.391+0000 7f8c3affd700 1 -- 172.21.15.49:0/2171178450 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c40011e50 con 0x7f8c44109340 2023-10-08T09:18:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.391+0000 7f8c3affd700 1 -- 172.21.15.49:0/2171178450 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40010db0 con 0x7f8c44109340 2023-10-08T09:18:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.391+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c44111280 con 0x7f8c44109340 2023-10-08T09:18:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.391+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c44111740 con 0x7f8c44109340 2023-10-08T09:18:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.395+0000 7f8c3affd700 1 -- 172.21.15.49:0/2171178450 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8c4001a400 con 0x7f8c44109340 2023-10-08T09:18:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.395+0000 7f8c3affd700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c300a59e0 0x7f8c300a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.396+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c300a59e0 0x7f8c300a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.396+0000 7f8c3affd700 1 -- 172.21.15.49:0/2171178450 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8c400cc810 con 0x7f8c44109340 2023-10-08T09:18:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.396+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c300a59e0 0x7f8c300a7ea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0062a0 tx=0x7f8c3c0061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.397+0000 7f8c583e9b80 1 Processor -- start 2023-10-08T09:18:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.397+0000 7f8c583e9b80 1 -- start start 2023-10-08T09:18:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e74131dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131e200 0x55e74131e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131ecc0 0x55e74131f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74127d400 con 0x55e74131ecc0 2023-10-08T09:18:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74126ebc0 con 0x55e74131e200 2023-10-08T09:18:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74126c4e0 con 0x55e74131d8a0 2023-10-08T09:18:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c49f9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e74131dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c49f9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e74131dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38288/0 (socket says 172.21.15.49:38288) 2023-10-08T09:18:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c49f9c700 1 -- 172.21.15.49:0/502795604 learned_addr learned my addr 172.21.15.49:0/502795604 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c49f9c700 1 -- 172.21.15.49:0/502795604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131e200 msgr2=0x55e74131e6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:18:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/502795604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131e200 0x55e74131e6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c49f9c700 1 -- 172.21.15.49:0/502795604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131ecc0 msgr2=0x55e74131f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.398+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/502795604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131ecc0 0x55e74131f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.399+0000 7f8c49f9c700 1 -- 172.21.15.49:0/502795604 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e741092d80 con 0x55e74131d8a0 2023-10-08T09:18:12.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.399+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/502795604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e74131dcc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c015000 tx=0x7f8c3c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2793ee0423c24b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.401+0000 7f8c2ffff700 1 -- 172.21.15.49:0/502795604 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00e040 con 0x55e74131d8a0 2023-10-08T09:18:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.401+0000 7f8c2ffff700 1 -- 172.21.15.49:0/502795604 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c005840 con 0x55e74131d8a0 2023-10-08T09:18:12.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.401+0000 7f8c2ffff700 1 -- 172.21.15.49:0/502795604 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c01cbe0 con 0x55e74131d8a0 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 -- 172.21.15.49:0/502795604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 msgr2=0x55e74131dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/502795604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e74131dcc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c015000 tx=0x7f8c3c01c000 comp rx=0 tx=0).stop 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 -- 172.21.15.49:0/502795604 shutdown_connections 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/502795604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e74131dcc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/502795604 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131ecc0 0x55e74131f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/502795604 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131e200 0x55e74131e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 -- 172.21.15.49:0/502795604 >> 172.21.15.49:0/502795604 conn(0x55e741289200 msgr2=0x55e74130fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 -- 172.21.15.49:0/502795604 shutdown_connections 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 -- 172.21.15.49:0/502795604 wait complete. 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 -- start start 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.402+0000 7f8c583e9b80 1 Processor -- start 2023-10-08T09:18:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e741327370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131e200 0x55e7413278b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741267dd0 con 0x55e74131e200 2023-10-08T09:18:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7412046b0 con 0x55e74131d8a0 2023-10-08T09:18:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741204350 con 0x55e74131ecc0 2023-10-08T09:18:12.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c583e9b80 10 client.0 _ll_register_callbacks cb 0x55e7410617a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:12.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c4a79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c4a79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38292/0 (socket says 172.21.15.49:38292) 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c4a79d700 1 -- 172.21.15.49:0/1150615629 learned_addr learned my addr 172.21.15.49:0/1150615629 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.403+0000 7f8c4979b700 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131e200 0x55e7413278b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e741327370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c4979b700 1 -- 172.21.15.49:0/1150615629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 msgr2=0x55e74132e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c4979b700 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c4979b700 1 -- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 msgr2=0x55e741327370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c4979b700 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e741327370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c4979b700 1 -- 172.21.15.49:0/1150615629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c44071f50 con 0x55e74131e200 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c4979b700 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131e200 0x55e7413278b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8c34010620 tx=0x7f8c34010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7d6647d19a2e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:12.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.404+0000 7f8c2d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.405+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1150615629 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34012860 con 0x55e74131e200 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.405+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1150615629 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8c34016070 con 0x55e74131e200 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.405+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e74122fe50 con 0x55e74131e200 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.405+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1150615629 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c340129c0 con 0x55e74131e200 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.405+0000 7f8c583e9b80 10 client.16173 fetch_fsmap learned FSMap version 35 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e74107bc00 con 0x55e74131e200 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1150615629 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e74122fe50 con 0x55e74131e200 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1150615629 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8c3401bd00 con 0x55e74131e200 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c583e9b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 35 2023-10-08T09:18:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c583e9b80 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c583e9b80 20 client.16173 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741333330 0x55e741335750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c583e9b80 4 client.16173 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e741260db0 con 0x55e741333330 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.406+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741333330 0x55e741335750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.407+0000 7f8c2d7fa700 10 client.16173 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.407+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741333330 0x55e741335750 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de3b14b8f8b21ceb server_cookie=9ec6d26ea32a0f7f in_seq=0 out_seq=0 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1150615629 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e741260db0 con 0x55e741333330 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c2d7fa700 10 client.16173 handle_command_reply: tid=0 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 client.16173 shutdown 2023-10-08T09:18:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131e200 msgr2=0x55e7413278b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131e200 0x55e7413278b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8c34010620 tx=0x7f8c34010930 comp rx=0 tx=0).stop 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 shutdown_connections 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741333330 0x55e741335750 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e741327370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1150615629 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74131e200 0x55e7413278b0 secure :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8c34010620 tx=0x7f8c34010930 comp rx=0 tx=0).stop 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 >> 172.21.15.49:0/1150615629 conn(0x55e741289200 msgr2=0x55e74130f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.408+0000 7f8c2d7fa700 0 client.16173 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.409+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 shutdown_connections 2023-10-08T09:18:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.409+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1150615629 wait complete. 2023-10-08T09:18:12.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.409+0000 7f8c583e9b80 20 client.16173 trim_cache size 0 max 16384 2023-10-08T09:18:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.438+0000 7f8c583e9b80 1 Processor -- start 2023-10-08T09:18:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.439+0000 7f8c583e9b80 1 -- start start 2023-10-08T09:18:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.439+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e741321cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.439+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131ecc0 0x55e741322210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.439+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 0x55e74132e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.439+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7410e88c0 con 0x55e74133a070 2023-10-08T09:18:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.439+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7412daa30 con 0x55e74131ecc0 2023-10-08T09:18:12.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.439+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e741325d20 con 0x55e74131d8a0 2023-10-08T09:18:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e741321cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e741321cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38300/0 (socket says 172.21.15.49:38300) 2023-10-08T09:18:12.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 -- 172.21.15.49:0/1062384443 learned_addr learned my addr 172.21.15.49:0/1062384443 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 -- 172.21.15.49:0/1062384443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131ecc0 msgr2=0x55e741322210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1062384443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131ecc0 0x55e741322210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 -- 172.21.15.49:0/1062384443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 msgr2=0x55e74132e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1062384443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 0x55e74132e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 -- 172.21.15.49:0/1062384443 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7410b91f0 con 0x55e74131d8a0 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.446+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1062384443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e741321cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c004cf0 tx=0x7f8c3c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2c9b7a8e7801ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.447+0000 7f8c38ff9700 1 -- 172.21.15.49:0/1062384443 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00e040 con 0x55e74131d8a0 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.447+0000 7f8c38ff9700 1 -- 172.21.15.49:0/1062384443 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c005a70 con 0x55e74131d8a0 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.447+0000 7f8c38ff9700 1 -- 172.21.15.49:0/1062384443 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c038e60 con 0x55e74131d8a0 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.448+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1062384443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 msgr2=0x55e741321cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.448+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1062384443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e741321cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c004cf0 tx=0x7f8c3c0099e0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.448+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1062384443 shutdown_connections 2023-10-08T09:18:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.448+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1062384443 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131d8a0 0x55e741321cd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.448+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1062384443 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 0x55e74132e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.448+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1062384443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131ecc0 0x55e741322210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.448+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1062384443 >> 172.21.15.49:0/1062384443 conn(0x55e741289200 msgr2=0x55e74130fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.449+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1062384443 shutdown_connections 2023-10-08T09:18:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.449+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1062384443 wait complete. 2023-10-08T09:18:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 -- start start 2023-10-08T09:18:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 Processor -- start 2023-10-08T09:18:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e74132c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 0x55e74132d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74132dd40 con 0x55e74133a070 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74132deb0 con 0x55e74131d8a0 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74132e070 con 0x55e74131ecc0 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.450+0000 7f8c583e9b80 10 client.0 _ll_register_callbacks cb 0x7f8c34009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c49f9c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e74132c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c4979b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c49f9c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e74132c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40886/0 (socket says 172.21.15.49:40886) 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c49f9c700 1 -- 172.21.15.49:0/1797004168 learned_addr learned my addr 172.21.15.49:0/1797004168 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c4979b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38308/0 (socket says 172.21.15.49:38308) 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c4979b700 1 -- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 msgr2=0x55e74132c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c4979b700 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e74132c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c4979b700 1 -- 172.21.15.49:0/1797004168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 msgr2=0x55e74132d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c4979b700 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 0x55e74132d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c4979b700 1 -- 172.21.15.49:0/1797004168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e741092d80 con 0x55e74131ecc0 2023-10-08T09:18:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.451+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e74132c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.452+0000 7f8c4979b700 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132c7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55e741330920 tx=0x7f8c3401cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3890f00404b82a7a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.456+0000 7f8c2d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:18:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.456+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1797004168 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c340097b0 con 0x55e74131ecc0 2023-10-08T09:18:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.456+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1797004168 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8c34016070 con 0x55e74131ecc0 2023-10-08T09:18:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.456+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c3c005840 con 0x55e74131ecc0 2023-10-08T09:18:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.459+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1797004168 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c34005040 con 0x55e74131ecc0 2023-10-08T09:18:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 10 client.25837 fetch_fsmap learned FSMap version 35 2023-10-08T09:18:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e741091250 con 0x55e74131ecc0 2023-10-08T09:18:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1797004168 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c005840 con 0x55e74131ecc0 2023-10-08T09:18:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1797004168 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 35) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8c34003c60 con 0x55e74131ecc0 2023-10-08T09:18:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 10 client.25837 fetch_fsmap finished waiting for FSMap version 35 2023-10-08T09:18:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 20 client.25837 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741318960 0x55e741318d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 4 client.25837 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:18:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.460+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e741260ad0 con 0x55e741318960 2023-10-08T09:18:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.463+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741318960 0x55e741318d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.472+0000 7f8c49f9c700 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741318960 0x55e741318d70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=babdb1f78ce9e4c8 server_cookie=c3bfc58837870a97 in_seq=0 out_seq=0 2023-10-08T09:18:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.475+0000 7f8c2d7fa700 10 client.25837 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c2d7fa700 1 -- 172.21.15.49:0/1797004168 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e741260ad0 con 0x55e741318960 2023-10-08T09:18:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c2d7fa700 10 client.25837 handle_command_reply: tid=0 2023-10-08T09:18:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 client.25837 shutdown 2023-10-08T09:18:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 msgr2=0x55e74132c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132c7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55e741330920 tx=0x7f8c3401cf20 comp rx=0 tx=0).stop 2023-10-08T09:18:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 shutdown_connections 2023-10-08T09:18:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e741318960 0x55e741318d70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e74133a070 0x55e74132d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e74131d8a0 0x55e74132c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 --2- 172.21.15.49:0/1797004168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e74131ecc0 0x55e74132c7f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 >> 172.21.15.49:0/1797004168 conn(0x55e741289200 msgr2=0x55e741326530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.494+0000 7f8c2d7fa700 0 client.25837 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.495+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 shutdown_connections 2023-10-08T09:18:12.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.495+0000 7f8c583e9b80 1 -- 172.21.15.49:0/1797004168 wait complete. 2023-10-08T09:18:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.496+0000 7f8c583e9b80 20 client.25837 trim_cache size 0 max 16384 2023-10-08T09:18:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:18:12.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:18:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:18:12.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c300a59e0 msgr2=0x7f8c300a7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c300a59e0 0x7f8c300a7ea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0062a0 tx=0x7f8c3c0061f0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c44109340 msgr2=0x7f8c441a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c44109340 0x7f8c441a64c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000b3f0 tx=0x7f8c4000b700 comp rx=0 tx=0).stop 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 shutdown_connections 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8c300a59e0 0x7f8c300a7ea0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c44109340 0x7f8c441a64c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c44107a60 0x7f8c4419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 --2- 172.21.15.49:0/2171178450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c44108980 0x7f8c4419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.497+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 >> 172.21.15.49:0/2171178450 conn(0x7f8c4406d5f0 msgr2=0x7f8c4410bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.498+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 shutdown_connections 2023-10-08T09:18:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:12.498+0000 7f8c4af9e700 1 -- 172.21.15.49:0/2171178450 wait complete. 2023-10-08T09:18:13.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:13 smithi184 ceph-mon[103675]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.5 MiB/s wr, 663 op/s 2023-10-08T09:18:14.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:13 smithi132 ceph-mon[103389]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.5 MiB/s wr, 663 op/s 2023-10-08T09:18:14.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:13 smithi049 ceph-mon[99485]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.5 MiB/s wr, 663 op/s 2023-10-08T09:18:14.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 170.0 s, 1062.0 tps, lat 91.897 ms stddev 112.291 2023-10-08T09:18:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:15 smithi184 ceph-mon[103675]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.6 MiB/s wr, 636 op/s 2023-10-08T09:18:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:15 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:15 smithi132 ceph-mon[103389]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.6 MiB/s wr, 636 op/s 2023-10-08T09:18:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:15 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:16.131 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:15 smithi049 ceph-mon[99485]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.6 MiB/s wr, 636 op/s 2023-10-08T09:18:16.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:15 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:18.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:17 smithi132 ceph-mon[103389]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 648 op/s 2023-10-08T09:18:18.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:17 smithi049 ceph-mon[99485]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 648 op/s 2023-10-08T09:18:18.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:17 smithi184 ceph-mon[103675]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 648 op/s 2023-10-08T09:18:19.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:18 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:19.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:18 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:19.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:18 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:18:19.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 175.0 s, 973.0 tps, lat 105.494 ms stddev 143.311 2023-10-08T09:18:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:19 smithi132 ceph-mon[103389]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.3 MiB/s wr, 530 op/s 2023-10-08T09:18:20.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:19 smithi049 ceph-mon[99485]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.3 MiB/s wr, 530 op/s 2023-10-08T09:18:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:19 smithi184 ceph-mon[103675]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.3 MiB/s wr, 530 op/s 2023-10-08T09:18:22.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:21 smithi132 ceph-mon[103389]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.1 MiB/s wr, 600 op/s 2023-10-08T09:18:22.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:21 smithi049 ceph-mon[99485]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.1 MiB/s wr, 600 op/s 2023-10-08T09:18:22.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:21 smithi184 ceph-mon[103675]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.1 MiB/s wr, 600 op/s 2023-10-08T09:18:23.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:22 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:18:23.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:22 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:18:23.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:22 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:18:24.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:23 smithi049 ceph-mon[99485]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 MiB/s wr, 597 op/s 2023-10-08T09:18:24.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:23 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:18:24.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:23 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:18:24.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:23 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:18:24.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:23 smithi184 ceph-mon[103675]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 MiB/s wr, 597 op/s 2023-10-08T09:18:24.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:23 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:18:24.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:23 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:18:24.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:23 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:18:24.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:23 smithi132 ceph-mon[103389]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.1 MiB/s wr, 597 op/s 2023-10-08T09:18:24.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:23 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:18:24.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:23 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:18:24.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:23 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:18:24.316 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 180.0 s, 1018.5 tps, lat 99.089 ms stddev 116.675 2023-10-08T09:18:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 -- 172.21.15.49:0/2871552721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c100870 msgr2=0x7f017c100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 --2- 172.21.15.49:0/2871552721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c100870 0x7f017c100cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0170009990 tx=0x7f0170009ca0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 -- 172.21.15.49:0/2871552721 shutdown_connections 2023-10-08T09:18:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 --2- 172.21.15.49:0/2871552721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c101210 0x7f017c107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 --2- 172.21.15.49:0/2871552721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f017c0ff680 0x7f017c0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 --2- 172.21.15.49:0/2871552721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c100870 0x7f017c100cd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 -- 172.21.15.49:0/2871552721 >> 172.21.15.49:0/2871552721 conn(0x7f017c0fabf0 msgr2=0x7f017c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 -- 172.21.15.49:0/2871552721 shutdown_connections 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.685+0000 7f01808cb700 1 -- 172.21.15.49:0/2871552721 wait complete. 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 Processor -- start 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 -- start start 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f017c0ff680 0x7f017c195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c100870 0x7f017c195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c101210 0x7f017c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19d130 con 0x7f017c100870 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19d270 con 0x7f017c0ff680 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.686+0000 7f01808cb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19d400 con 0x7f017c101210 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c100870 0x7f017c195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c100870 0x7f017c195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51284/0 (socket says 172.21.15.49:51284) 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 -- 172.21.15.49:0/1994600138 learned_addr learned my addr 172.21.15.49:0/1994600138 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f017ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c101210 0x7f017c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f017a59c700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f017c0ff680 0x7f017c195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 -- 172.21.15.49:0/1994600138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c101210 msgr2=0x7f017c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c101210 0x7f017c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 -- 172.21.15.49:0/1994600138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f017c0ff680 msgr2=0x7f017c195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f017c0ff680 0x7f017c195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 -- 172.21.15.49:0/1994600138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01700095f0 con 0x7f017c100870 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f017ad9d700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c101210 0x7f017c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f0179d9b700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c100870 0x7f017c195b40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0170006010 tx=0x7f0170004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f016b7fe700 1 -- 172.21.15.49:0/1994600138 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017001d070 con 0x7f017c100870 2023-10-08T09:18:25.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c19d680 con 0x7f017c100870 2023-10-08T09:18:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f016b7fe700 1 -- 172.21.15.49:0/1994600138 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0170005030 con 0x7f017c100870 2023-10-08T09:18:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f016b7fe700 1 -- 172.21.15.49:0/1994600138 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170004b40 con 0x7f017c100870 2023-10-08T09:18:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.687+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c19db00 con 0x7f017c100870 2023-10-08T09:18:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.691+0000 7f016b7fe700 1 -- 172.21.15.49:0/1994600138 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0170004d40 con 0x7f017c100870 2023-10-08T09:18:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.692+0000 7f016b7fe700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01640a1480 0x7f01640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.692+0000 7f016b7fe700 1 -- 172.21.15.49:0/1994600138 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f01700ce0f0 con 0x7f017c100870 2023-10-08T09:18:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.692+0000 7f017a59c700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01640a1480 0x7f01640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.692+0000 7f018cab4b80 1 Processor -- start 2023-10-08T09:18:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.692+0000 7f018cab4b80 1 -- start start 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f017a59c700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01640a1480 0x7f01640a3940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f017c1006d0 tx=0x7f017400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454de20 0x555e5454e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e5455b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 0x555e5455e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f018cab4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e544b6400 con 0x555e5454de20 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f018cab4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e544a7bc0 con 0x555e5455bd90 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f018cab4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e544a54e0 con 0x555e5454e780 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f017a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454de20 0x555e5454e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e5455b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f017a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454de20 0x555e5454e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51292/0 (socket says 172.21.15.49:51292) 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f017ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 0x555e5455e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f017a59c700 1 -- 172.21.15.49:0/2079550484 learned_addr learned my addr 172.21.15.49:0/2079550484 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0179d9b700 1 -- 172.21.15.49:0/2079550484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 msgr2=0x555e5455e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2079550484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 0x555e5455e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0179d9b700 1 -- 172.21.15.49:0/2079550484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454de20 msgr2=0x555e5454e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2079550484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454de20 0x555e5454e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0179d9b700 1 -- 172.21.15.49:0/2079550484 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e542cbd80 con 0x555e5454e780 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2079550484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e5455b850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01700cb8c0 tx=0x7f0170009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237b5bf261b611a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.693+0000 7f0168ff9700 1 -- 172.21.15.49:0/2079550484 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009f2c0 con 0x555e5454e780 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f0168ff9700 1 -- 172.21.15.49:0/2079550484 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01700a3470 con 0x555e5454e780 2023-10-08T09:18:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f0168ff9700 1 -- 172.21.15.49:0/2079550484 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009a660 con 0x555e5454e780 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 -- 172.21.15.49:0/2079550484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 msgr2=0x555e5455b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2079550484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e5455b850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01700cb8c0 tx=0x7f0170009ca0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 -- 172.21.15.49:0/2079550484 shutdown_connections 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2079550484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 0x555e5455e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2079550484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454de20 0x555e5454e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2079550484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e5455b850 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 -- 172.21.15.49:0/2079550484 >> 172.21.15.49:0/2079550484 conn(0x555e544c2200 msgr2=0x555e54547980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 -- 172.21.15.49:0/2079550484 shutdown_connections 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.694+0000 7f018cab4b80 1 -- 172.21.15.49:0/2079550484 wait complete. 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 -- start start 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 Processor -- start 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e5455fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54560330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5455bd90 0x555e54567370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e544a0dd0 con 0x555e5454e780 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5443d6b0 con 0x555e5454de20 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e5443d350 con 0x555e5455bd90 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f018cab4b80 10 client.0 _ll_register_callbacks cb 0x555e5429a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f017a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e5455fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54560330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f017a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e5455fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:40912/0 (socket says 172.21.15.49:40912) 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f017a59c700 1 -- 172.21.15.49:0/2088313283 learned_addr learned my addr 172.21.15.49:0/2088313283 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.695+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54560330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51300/0 (socket says 172.21.15.49:51300) 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017ad9d700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5455bd90 0x555e54567370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017a59c700 1 -- 172.21.15.49:0/2088313283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5455bd90 msgr2=0x555e54567370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017a59c700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5455bd90 0x555e54567370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017a59c700 1 -- 172.21.15.49:0/2088313283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 msgr2=0x555e54560330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017a59c700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54560330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017a59c700 1 -- 172.21.15.49:0/2088313283 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c19db00 con 0x555e5454de20 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017ad9d700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5455bd90 0x555e54567370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54560330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f017a59c700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e5455fdf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0174009fd0 tx=0x7f0174004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329166d805a8f99b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f0161ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f0161ffb700 1 -- 172.21.15.49:0/2088313283 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400e040 con 0x555e5454de20 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e54468e50 con 0x555e5454de20 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.696+0000 7f0161ffb700 1 -- 172.21.15.49:0/2088313283 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f017401c040 con 0x555e5454de20 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.697+0000 7f0161ffb700 1 -- 172.21.15.49:0/2088313283 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0174017e90 con 0x555e5454de20 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.697+0000 7f018cab4b80 10 client.25832 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.697+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555e542b4c00 con 0x555e5454de20 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.697+0000 7f0161ffb700 1 -- 172.21.15.49:0/2088313283 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e54468e50 con 0x555e5454de20 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.698+0000 7f0161ffb700 1 -- 172.21.15.49:0/2088313283 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f017402c530 con 0x555e5454de20 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.698+0000 7f018cab4b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.698+0000 7f018cab4b80 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.698+0000 7f018cab4b80 20 client.25832 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.698+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e5456e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.698+0000 7f018cab4b80 4 client.25832 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:18:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.698+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e54499db0 con 0x555e5456bd00 2023-10-08T09:18:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.699+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e5456e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.699+0000 7f0161ffb700 10 client.25832 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.699+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e5456e120 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c823a65c9872ca05 server_cookie=31fce9305e6ff018 in_seq=0 out_seq=0 2023-10-08T09:18:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f0161ffb700 1 -- 172.21.15.49:0/2088313283 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e54499db0 con 0x555e5456bd00 2023-10-08T09:18:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f0161ffb700 10 client.25832 handle_command_reply: tid=0 2023-10-08T09:18:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 client.25832 shutdown 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 msgr2=0x555e5455fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e5455fdf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0174009fd0 tx=0x7f0174004ad0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 shutdown_connections 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e5456e120 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5455bd90 0x555e54567370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e5455fdf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f0161ffb700 0 client.25832 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2088313283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54560330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 >> 172.21.15.49:0/2088313283 conn(0x555e544c2200 msgr2=0x555e5454ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 shutdown_connections 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 1 -- 172.21.15.49:0/2088313283 wait complete. 2023-10-08T09:18:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.700+0000 7f018cab4b80 20 client.25832 trim_cache size 0 max 16384 2023-10-08T09:18:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.717+0000 7f018cab4b80 1 Processor -- start 2023-10-08T09:18:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.725+0000 7f018cab4b80 1 -- start start 2023-10-08T09:18:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.725+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454de20 0x555e54547950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.725+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54567330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.725+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 0x555e5455b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.725+0000 7f018cab4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e543218c0 con 0x555e5454e780 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.725+0000 7f018cab4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e54547780 con 0x555e5455bd90 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.725+0000 7f018cab4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e54547e90 con 0x555e5454de20 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54567330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54567330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51308/0 (socket says 172.21.15.49:51308) 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 -- 172.21.15.49:0/3150161154 learned_addr learned my addr 172.21.15.49:0/3150161154 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 -- 172.21.15.49:0/3150161154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454de20 msgr2=0x555e54547950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:18:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 --2- 172.21.15.49:0/3150161154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454de20 0x555e54547950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 -- 172.21.15.49:0/3150161154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 msgr2=0x555e5455b610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:18:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 --2- 172.21.15.49:0/3150161154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 0x555e5455b610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 -- 172.21.15.49:0/3150161154 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e542f21f0 con 0x555e5454e780 2023-10-08T09:18:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.726+0000 7f0179d9b700 1 --2- 172.21.15.49:0/3150161154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54567330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f017009ae70 tx=0x7f0170009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a487519ece8a54 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.727+0000 7f01697fa700 1 -- 172.21.15.49:0/3150161154 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009b8f0 con 0x555e5454e780 2023-10-08T09:18:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.727+0000 7f01697fa700 1 -- 172.21.15.49:0/3150161154 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017009f2c0 con 0x555e5454e780 2023-10-08T09:18:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.727+0000 7f01697fa700 1 -- 172.21.15.49:0/3150161154 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01700a3240 con 0x555e5454e780 2023-10-08T09:18:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.728+0000 7f018cab4b80 1 -- 172.21.15.49:0/3150161154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 msgr2=0x555e54567330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.728+0000 7f018cab4b80 1 --2- 172.21.15.49:0/3150161154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54567330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f017009ae70 tx=0x7f0170009310 comp rx=0 tx=0).stop 2023-10-08T09:18:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.738+0000 7f018cab4b80 1 -- 172.21.15.49:0/3150161154 shutdown_connections 2023-10-08T09:18:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.738+0000 7f018cab4b80 1 --2- 172.21.15.49:0/3150161154 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454de20 0x555e54547950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.738+0000 7f018cab4b80 1 --2- 172.21.15.49:0/3150161154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5455bd90 0x555e5455b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.738+0000 7f018cab4b80 1 --2- 172.21.15.49:0/3150161154 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5454e780 0x555e54567330 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.738+0000 7f018cab4b80 1 -- 172.21.15.49:0/3150161154 >> 172.21.15.49:0/3150161154 conn(0x555e544c2200 msgr2=0x555e54572a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.742+0000 7f018cab4b80 1 -- 172.21.15.49:0/3150161154 shutdown_connections 2023-10-08T09:18:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.754+0000 7f018cab4b80 1 -- 172.21.15.49:0/3150161154 wait complete. 2023-10-08T09:18:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.754+0000 7f018cab4b80 1 -- start start 2023-10-08T09:18:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.754+0000 7f018cab4b80 1 Processor -- start 2023-10-08T09:18:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.756+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e54563180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.756+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e545636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.756+0000 7f018cab4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5455bd90 0x555e54566690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.756+0000 7f018cab4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e54566bd0 con 0x555e5455bd90 2023-10-08T09:18:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.756+0000 7f018cab4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e54566d40 con 0x555e5454de20 2023-10-08T09:18:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.756+0000 7f018cab4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e54566f00 con 0x555e5454e780 2023-10-08T09:18:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.756+0000 7f018cab4b80 10 client.0 _ll_register_callbacks cb 0x7f0174017840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.757+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e545636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.757+0000 7f0179d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e545636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38348/0 (socket says 172.21.15.49:38348) 2023-10-08T09:18:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.757+0000 7f0179d9b700 1 -- 172.21.15.49:0/2530582840 learned_addr learned my addr 172.21.15.49:0/2530582840 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0179d9b700 1 -- 172.21.15.49:0/2530582840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 msgr2=0x555e54563180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e54563180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0179d9b700 1 -- 172.21.15.49:0/2530582840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5455bd90 msgr2=0x555e54566690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5455bd90 0x555e54566690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0179d9b700 1 -- 172.21.15.49:0/2530582840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e542cbd80 con 0x555e5454e780 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0179d9b700 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e545636c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f017000b680 tx=0x7f017009f430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3103bb0813c9dde3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0161ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f0161ffb700 1 -- 172.21.15.49:0/2530582840 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017009b8f0 con 0x555e5454e780 2023-10-08T09:18:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.758+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01700a3470 con 0x555e5454e780 2023-10-08T09:18:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.759+0000 7f0161ffb700 1 -- 172.21.15.49:0/2530582840 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f01700a2fe0 con 0x555e5454e780 2023-10-08T09:18:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.759+0000 7f0161ffb700 1 -- 172.21.15.49:0/2530582840 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01700a31a0 con 0x555e5454e780 2023-10-08T09:18:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.759+0000 7f018cab4b80 10 client.25852 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.759+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555e542ca250 con 0x555e5454e780 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f0161ffb700 1 -- 172.21.15.49:0/2530582840 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01700a3470 con 0x555e5454e780 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f0161ffb700 1 -- 172.21.15.49:0/2530582840 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f017000fad0 con 0x555e5454e780 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f018cab4b80 10 client.25852 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f018cab4b80 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f018cab4b80 20 client.25852 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e54555b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f018cab4b80 4 client.25852 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:18:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.760+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555e54499ad0 con 0x555e5456bd00 2023-10-08T09:18:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.766+0000 7f017a59c700 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e54555b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.773+0000 7f0161ffb700 10 client.25852 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.773+0000 7f017a59c700 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e54555b70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ae973943dc8c4e server_cookie=f436d6e9a7d768a2 in_seq=0 out_seq=0 2023-10-08T09:18:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f0161ffb700 1 -- 172.21.15.49:0/2530582840 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555e54499ad0 con 0x555e5456bd00 2023-10-08T09:18:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f0161ffb700 10 client.25852 handle_command_reply: tid=0 2023-10-08T09:18:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 client.25852 shutdown 2023-10-08T09:18:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 msgr2=0x555e545636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e545636c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f017000b680 tx=0x7f017009f430 comp rx=0 tx=0).stop 2023-10-08T09:18:25.801 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 shutdown_connections 2023-10-08T09:18:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555e5456bd00 0x555e54555b70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555e5455bd90 0x555e54566690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f0161ffb700 0 client.25852 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555e5454de20 0x555e54563180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 --2- 172.21.15.49:0/2530582840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555e5454e780 0x555e545636c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 >> 172.21.15.49:0/2530582840 conn(0x555e544c2200 msgr2=0x555e5455b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 shutdown_connections 2023-10-08T09:18:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 1 -- 172.21.15.49:0/2530582840 wait complete. 2023-10-08T09:18:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.798+0000 7f018cab4b80 20 client.25852 trim_cache size 0 max 16384 2023-10-08T09:18:25.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:18:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:18:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:18:25.804 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:18:25.805 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01640a1480 msgr2=0x7f01640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01640a1480 0x7f01640a3940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f017c1006d0 tx=0x7f017400b040 comp rx=0 tx=0).stop 2023-10-08T09:18:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c100870 msgr2=0x7f017c195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c100870 0x7f017c195b40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0170006010 tx=0x7f0170004730 comp rx=0 tx=0).stop 2023-10-08T09:18:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 shutdown_connections 2023-10-08T09:18:25.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f01640a1480 0x7f01640a3940 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f017c101210 0x7f017c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f017c0ff680 0x7f017c195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 --2- 172.21.15.49:0/1994600138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f017c100870 0x7f017c195b40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 >> 172.21.15.49:0/1994600138 conn(0x7f017c0fabf0 msgr2=0x7f017c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 shutdown_connections 2023-10-08T09:18:25.807 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:25.799+0000 7f01808cb700 1 -- 172.21.15.49:0/1994600138 wait complete. 2023-10-08T09:18:26.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:25 smithi049 ceph-mon[99485]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 582 op/s 2023-10-08T09:18:26.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:25 smithi184 ceph-mon[103675]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 582 op/s 2023-10-08T09:18:26.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:25 smithi132 ceph-mon[103389]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 582 op/s 2023-10-08T09:18:28.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:27 smithi049 ceph-mon[99485]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 617 op/s 2023-10-08T09:18:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:27 smithi184 ceph-mon[103675]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 617 op/s 2023-10-08T09:18:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:27 smithi132 ceph-mon[103389]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 617 op/s 2023-10-08T09:18:29.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 185.0 s, 1046.1 tps, lat 92.225 ms stddev 109.616 2023-10-08T09:18:30.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:29 smithi049 ceph-mon[99485]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2023-10-08T09:18:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:29 smithi184 ceph-mon[103675]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2023-10-08T09:18:30.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:29 smithi132 ceph-mon[103389]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2023-10-08T09:18:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:31 smithi049 ceph-mon[99485]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 619 op/s 2023-10-08T09:18:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:31 smithi184 ceph-mon[103675]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 619 op/s 2023-10-08T09:18:32.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:31 smithi132 ceph-mon[103389]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.9 MiB/s wr, 619 op/s 2023-10-08T09:18:33.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:32 smithi184 ceph-mon[103675]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.3 MiB/s wr, 673 op/s 2023-10-08T09:18:33.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:32 smithi132 ceph-mon[103389]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.3 MiB/s wr, 673 op/s 2023-10-08T09:18:33.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:32 smithi049 ceph-mon[99485]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.3 MiB/s wr, 673 op/s 2023-10-08T09:18:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 190.0 s, 715.4 tps, lat 141.890 ms stddev 153.803 2023-10-08T09:18:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:35 smithi049 ceph-mon[99485]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 713 op/s 2023-10-08T09:18:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:35 smithi184 ceph-mon[103675]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 713 op/s 2023-10-08T09:18:35.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:35 smithi132 ceph-mon[103389]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 713 op/s 2023-10-08T09:18:37.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:37 smithi049 ceph-mon[99485]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 808 op/s 2023-10-08T09:18:37.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:37 smithi184 ceph-mon[103675]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 808 op/s 2023-10-08T09:18:38.054 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:37 smithi132 ceph-mon[103389]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 808 op/s 2023-10-08T09:18:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.991+0000 7f73dc21b700 1 -- 172.21.15.49:0/1512980651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73d41077f0 msgr2=0x7f73d4109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.991+0000 7f73dc21b700 1 --2- 172.21.15.49:0/1512980651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73d41077f0 0x7f73d4109c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00b0a0 tx=0x7f73cc00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:18:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.994+0000 7f73dc21b700 1 -- 172.21.15.49:0/1512980651 shutdown_connections 2023-10-08T09:18:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.994+0000 7f73dc21b700 1 --2- 172.21.15.49:0/1512980651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73d41077f0 0x7f73d4109c20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.994+0000 7f73dc21b700 1 --2- 172.21.15.49:0/1512980651 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410cb30 0x7f73d410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.994+0000 7f73dc21b700 1 --2- 172.21.15.49:0/1512980651 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d410a160 0x7f73d410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.994+0000 7f73dc21b700 1 -- 172.21.15.49:0/1512980651 >> 172.21.15.49:0/1512980651 conn(0x7f73d406d770 msgr2=0x7f73d406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.994+0000 7f73dc21b700 1 -- 172.21.15.49:0/1512980651 shutdown_connections 2023-10-08T09:18:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.994+0000 7f73dc21b700 1 -- 172.21.15.49:0/1512980651 wait complete. 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 Processor -- start 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 -- start start 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73d41077f0 0x7f73d41a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410a160 0x7f73d41a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d410cb30 0x7f73d41acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41ad140 con 0x7f73d410a160 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41ad280 con 0x7f73d410cb30 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.995+0000 7f73dc21b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41ad410 con 0x7f73d41077f0 2023-10-08T09:18:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410a160 0x7f73d41a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410a160 0x7f73d41a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51324/0 (socket says 172.21.15.49:51324) 2023-10-08T09:18:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 -- 172.21.15.49:0/2017832093 learned_addr learned my addr 172.21.15.49:0/2017832093 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73da7b8700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d410cb30 0x7f73d41acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 -- 172.21.15.49:0/2017832093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73d41077f0 msgr2=0x7f73d41a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73d41077f0 0x7f73d41a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 -- 172.21.15.49:0/2017832093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d410cb30 msgr2=0x7f73d41acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d410cb30 0x7f73d41acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 -- 172.21.15.49:0/2017832093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc009d00 con 0x7f73d410a160 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73da7b8700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d410cb30 0x7f73d41acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.996+0000 7f73d97b6700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410a160 0x7f73d41a5b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f73c400b550 tx=0x7f73c400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.997+0000 7f73caffd700 1 -- 172.21.15.49:0/2017832093 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4011830 con 0x7f73d410a160 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.997+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d41ad6f0 con 0x7f73d410a160 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.997+0000 7f73caffd700 1 -- 172.21.15.49:0/2017832093 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c4011e50 con 0x7f73d410a160 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.997+0000 7f73caffd700 1 -- 172.21.15.49:0/2017832093 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400f2e0 con 0x7f73d410a160 2023-10-08T09:18:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:38.997+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d41adcd0 con 0x7f73d410a160 2023-10-08T09:18:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.001+0000 7f73caffd700 1 -- 172.21.15.49:0/2017832093 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f73c4011990 con 0x7f73d410a160 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.001+0000 7f73e8404b80 1 Processor -- start 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73e8404b80 1 -- start start 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73caffd700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73c00a1550 0x7f73c00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208ce50 0x56106208d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106208dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 0x56106208e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73e8404b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061fe4400 con 0x56106208d7b0 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73e8404b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061fd5bc0 con 0x56106208e2c0 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.002+0000 7f73e8404b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061fd34e0 con 0x56106208ce50 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106208dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73da7b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 0x56106208e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73caffd700 1 -- 172.21.15.49:0/2017832093 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f73c40cc9d0 con 0x7f73d410a160 2023-10-08T09:18:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73da7b8700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73c00a1550 0x7f73c00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106208dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51334/0 (socket says 172.21.15.49:51334) 2023-10-08T09:18:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d97b6700 1 -- 172.21.15.49:0/1464974397 learned_addr learned my addr 172.21.15.49:0/1464974397 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208ce50 0x56106208d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d9fb7700 1 -- 172.21.15.49:0/1464974397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 msgr2=0x56106208e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 0x56106208e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d9fb7700 1 -- 172.21.15.49:0/1464974397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 msgr2=0x56106208dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106208dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d9fb7700 1 -- 172.21.15.49:0/1464974397 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561061df9d80 con 0x56106208ce50 2023-10-08T09:18:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73d97b6700 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106208dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.003+0000 7f73da7b8700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73c00a1550 0x7f73c00a3a10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f73d0015a80 tx=0x7f73d00097a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.004+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208ce50 0x56106208d270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73cc007d00 tx=0x7f73cc0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89364e1773aaf195 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.004+0000 7f73bffff700 1 -- 172.21.15.49:0/1464974397 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00e040 con 0x56106208ce50 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.004+0000 7f73bffff700 1 -- 172.21.15.49:0/1464974397 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73cc0079a0 con 0x56106208ce50 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.004+0000 7f73bffff700 1 -- 172.21.15.49:0/1464974397 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0059c0 con 0x56106208ce50 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.004+0000 7f73e8404b80 1 -- 172.21.15.49:0/1464974397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208ce50 msgr2=0x56106208d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.004+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208ce50 0x56106208d270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73cc007d00 tx=0x7f73cc0096c0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.005+0000 7f73e8404b80 1 -- 172.21.15.49:0/1464974397 shutdown_connections 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.005+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208ce50 0x56106208d270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.005+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 0x56106208e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.005+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1464974397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106208dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.005+0000 7f73e8404b80 1 -- 172.21.15.49:0/1464974397 >> 172.21.15.49:0/1464974397 conn(0x561061ff0200 msgr2=0x5610620768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.007+0000 7f73e8404b80 1 -- 172.21.15.49:0/1464974397 shutdown_connections 2023-10-08T09:18:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.018+0000 7f73e8404b80 1 -- 172.21.15.49:0/1464974397 wait complete. 2023-10-08T09:18:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.018+0000 7f73e8404b80 1 -- start start 2023-10-08T09:18:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.018+0000 7f73e8404b80 1 Processor -- start 2023-10-08T09:18:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208ce50 0x56106207d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106207d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208e2c0 0x561062084800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73e8404b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061fcedd0 con 0x56106208d7b0 2023-10-08T09:18:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73e8404b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061f6b6b0 con 0x56106208ce50 2023-10-08T09:18:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73e8404b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061f6b350 con 0x56106208e2c0 2023-10-08T09:18:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73e8404b80 10 client.0 _ll_register_callbacks cb 0x561061dc87a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106207d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106207d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51336/0 (socket says 172.21.15.49:51336) 2023-10-08T09:18:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.019+0000 7f73d97b6700 1 -- 172.21.15.49:0/3166750057 learned_addr learned my addr 172.21.15.49:0/3166750057 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.020+0000 7f73da7b8700 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208e2c0 0x561062084800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.020+0000 7f73d97b6700 1 -- 172.21.15.49:0/3166750057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208e2c0 msgr2=0x561062084800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.020+0000 7f73d97b6700 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208e2c0 0x561062084800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.020+0000 7f73d97b6700 1 -- 172.21.15.49:0/3166750057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208ce50 msgr2=0x56106207d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:18:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.020+0000 7f73d97b6700 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208ce50 0x56106207d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.020+0000 7f73d97b6700 1 -- 172.21.15.49:0/3166750057 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d41adcd0 con 0x56106208d7b0 2023-10-08T09:18:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.020+0000 7f73d97b6700 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106207d810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f73c4004820 tx=0x7f73c4099320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3ba91b3ac8b68f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.022+0000 7f73bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:18:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.022+0000 7f73bd7fa700 1 -- 172.21.15.49:0/3166750057 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c409a0c0 con 0x56106208d7b0 2023-10-08T09:18:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.022+0000 7f73bd7fa700 1 -- 172.21.15.49:0/3166750057 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f73c40999c0 con 0x56106208d7b0 2023-10-08T09:18:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.023+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561061f96e50 con 0x56106208d7b0 2023-10-08T09:18:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.024+0000 7f73bd7fa700 1 -- 172.21.15.49:0/3166750057 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73c409db10 con 0x56106208d7b0 2023-10-08T09:18:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.024+0000 7f73e8404b80 10 client.16197 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.024+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f73d410da70 con 0x56106208d7b0 2023-10-08T09:18:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73bd7fa700 1 -- 172.21.15.49:0/3166750057 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c407c080 con 0x56106208d7b0 2023-10-08T09:18:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73bd7fa700 1 -- 172.21.15.49:0/3166750057 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f73c4098eb0 con 0x56106208d7b0 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73e8404b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73e8404b80 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73e8404b80 20 client.16197 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73e8404b80 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561062089240 0x56106208b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73e8404b80 4 client.16197 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561061fc7db0 con 0x561062089240 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.025+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561062089240 0x56106208b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.026+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561062089240 0x56106208b660 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287cf864273ba5bc server_cookie=584555f7ff0a22d4 in_seq=0 out_seq=0 2023-10-08T09:18:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.026+0000 7f73bd7fa700 10 client.16197 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.027+0000 7f73bd7fa700 1 -- 172.21.15.49:0/3166750057 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561061fc7db0 con 0x561062089240 2023-10-08T09:18:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.027+0000 7f73bd7fa700 10 client.16197 handle_command_reply: tid=0 2023-10-08T09:18:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.027+0000 7f73e8404b80 1 client.16197 shutdown 2023-10-08T09:18:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.029+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 msgr2=0x56106207d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.029+0000 7f73e8404b80 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106207d810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f73c4004820 tx=0x7f73c4099320 comp rx=0 tx=0).stop 2023-10-08T09:18:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 shutdown_connections 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561062089240 0x56106208b660 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208e2c0 0x561062084800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208ce50 0x56106207d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73bd7fa700 0 client.16197 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 --2- 172.21.15.49:0/3166750057 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208d7b0 0x56106207d810 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 >> 172.21.15.49:0/3166750057 conn(0x561061ff0200 msgr2=0x561062076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 shutdown_connections 2023-10-08T09:18:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 1 -- 172.21.15.49:0/3166750057 wait complete. 2023-10-08T09:18:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.030+0000 7f73e8404b80 20 client.16197 trim_cache size 0 max 16384 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 Processor -- start 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 -- start start 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561062089240 0x56106208e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208ce50 0x561062084720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208d7b0 0x5610620991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061e4f8c0 con 0x56106208ce50 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561062084c60 con 0x56106208d7b0 2023-10-08T09:18:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.056+0000 7f73e8404b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561061fc7db0 con 0x561062089240 2023-10-08T09:18:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.058+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208ce50 0x561062084720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.058+0000 7f73d97b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208ce50 0x561062084720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51346/0 (socket says 172.21.15.49:51346) 2023-10-08T09:18:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.058+0000 7f73d97b6700 1 -- 172.21.15.49:0/1855015426 learned_addr learned my addr 172.21.15.49:0/1855015426 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.066+0000 7f73da7b8700 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208d7b0 0x5610620991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.068+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561062089240 0x56106208e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.068+0000 7f73d97b6700 1 -- 172.21.15.49:0/1855015426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561062089240 msgr2=0x56106208e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.068+0000 7f73d97b6700 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561062089240 0x56106208e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.068+0000 7f73d97b6700 1 -- 172.21.15.49:0/1855015426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208d7b0 msgr2=0x5610620991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.068+0000 7f73d97b6700 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208d7b0 0x5610620991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.068+0000 7f73d97b6700 1 -- 172.21.15.49:0/1855015426 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561061de2c00 con 0x56106208ce50 2023-10-08T09:18:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.069+0000 7f73d97b6700 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208ce50 0x561062084720 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f73c40ca1f0 tx=0x7f73c4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b1b7b56062086e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.069+0000 7f73c8ff9700 1 -- 172.21.15.49:0/1855015426 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c40a1b70 con 0x56106208ce50 2023-10-08T09:18:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.069+0000 7f73c8ff9700 1 -- 172.21.15.49:0/1855015426 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561061f96e50 con 0x56106208ce50 2023-10-08T09:18:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.069+0000 7f73c8ff9700 1 -- 172.21.15.49:0/1855015426 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4098eb0 con 0x56106208ce50 2023-10-08T09:18:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.070+0000 7f73e8404b80 1 -- 172.21.15.49:0/1855015426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208ce50 msgr2=0x561062084720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.070+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208ce50 0x561062084720 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f73c40ca1f0 tx=0x7f73c4011000 comp rx=0 tx=0).stop 2023-10-08T09:18:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.070+0000 7f73e8404b80 1 -- 172.21.15.49:0/1855015426 shutdown_connections 2023-10-08T09:18:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.070+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561062089240 0x56106208e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.070+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208d7b0 0x5610620991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.070+0000 7f73e8404b80 1 --2- 172.21.15.49:0/1855015426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56106208ce50 0x561062084720 secure :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f73c40ca1f0 tx=0x7f73c4011000 comp rx=0 tx=0).stop 2023-10-08T09:18:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.070+0000 7f73e8404b80 1 -- 172.21.15.49:0/1855015426 >> 172.21.15.49:0/1855015426 conn(0x561061ff0200 msgr2=0x561062094920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.071+0000 7f73e8404b80 1 -- 172.21.15.49:0/1855015426 shutdown_connections 2023-10-08T09:18:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.077+0000 7f73e8404b80 1 -- 172.21.15.49:0/1855015426 wait complete. 2023-10-08T09:18:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.077+0000 7f73e8404b80 1 -- start start 2023-10-08T09:18:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 1 Processor -- start 2023-10-08T09:18:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561062089240 0x561062098210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208d7b0 0x561062098750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 0x5610620e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561062098e30 con 0x561062089240 2023-10-08T09:18:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610620e47f0 con 0x56106208e2c0 2023-10-08T09:18:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610620e4930 con 0x56106208d7b0 2023-10-08T09:18:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.078+0000 7f73e8404b80 10 client.0 _ll_register_callbacks cb 0x7f73c4099880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.079+0000 7f73d9fb7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561062089240 0x561062098210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.079+0000 7f73d9fb7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561062089240 0x561062098210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51350/0 (socket says 172.21.15.49:51350) 2023-10-08T09:18:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.079+0000 7f73d9fb7700 1 -- 172.21.15.49:0/2422954632 learned_addr learned my addr 172.21.15.49:0/2422954632 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.079+0000 7f73d97b6700 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208d7b0 0x561062098750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.080+0000 7f73d9fb7700 1 -- 172.21.15.49:0/2422954632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208d7b0 msgr2=0x561062098750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.080+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208d7b0 0x561062098750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.080+0000 7f73d9fb7700 1 -- 172.21.15.49:0/2422954632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 msgr2=0x5610620e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.080+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 0x5610620e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.080+0000 7f73d9fb7700 1 -- 172.21.15.49:0/2422954632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d410da70 con 0x561062089240 2023-10-08T09:18:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.080+0000 7f73d9fb7700 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561062089240 0x561062098210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009fd0 tx=0x7f73cc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b463924898a628ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.083+0000 7f73bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:18:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.088+0000 7f73bd7fa700 1 -- 172.21.15.49:0/2422954632 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00e040 con 0x561062089240 2023-10-08T09:18:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.088+0000 7f73bd7fa700 1 -- 172.21.15.49:0/2422954632 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f73cc0090d0 con 0x561062089240 2023-10-08T09:18:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.089+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73cc0079a0 con 0x561062089240 2023-10-08T09:18:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.097+0000 7f73bd7fa700 1 -- 172.21.15.49:0/2422954632 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73cc009290 con 0x561062089240 2023-10-08T09:18:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.097+0000 7f73e8404b80 10 client.16203 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.097+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561061e201f0 con 0x561062089240 2023-10-08T09:18:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.099+0000 7f73bd7fa700 1 -- 172.21.15.49:0/2422954632 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73cc0079a0 con 0x561062089240 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.099+0000 7f73bd7fa700 1 -- 172.21.15.49:0/2422954632 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f73cc02d070 con 0x561062089240 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.099+0000 7f73e8404b80 10 client.16203 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.100+0000 7f73e8404b80 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.100+0000 7f73e8404b80 20 client.16203 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.100+0000 7f73e8404b80 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5610620e5f50 0x5610620e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.100+0000 7f73e8404b80 4 client.16203 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.100+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561061fc7ad0 con 0x5610620e5f50 2023-10-08T09:18:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.100+0000 7f73d97b6700 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5610620e5f50 0x5610620e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.100+0000 7f73bd7fa700 10 client.16203 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.101+0000 7f73d97b6700 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5610620e5f50 0x5610620e8370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30e064c6f49bc78 server_cookie=60f283b69f8221f1 in_seq=0 out_seq=0 2023-10-08T09:18:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73bd7fa700 1 -- 172.21.15.49:0/2422954632 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561061fc7ad0 con 0x5610620e5f50 2023-10-08T09:18:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73bd7fa700 10 client.16203 handle_command_reply: tid=0 2023-10-08T09:18:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 client.16203 shutdown 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561062089240 msgr2=0x561062098210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561062089240 0x561062098210 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009fd0 tx=0x7f73cc0099e0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 shutdown_connections 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5610620e5f50 0x5610620e8370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56106208e2c0 0x5610620e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561062089240 0x561062098210 secure :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009fd0 tx=0x7f73cc0099e0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 --2- 172.21.15.49:0/2422954632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56106208d7b0 0x561062098750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 >> 172.21.15.49:0/2422954632 conn(0x561061ff0200 msgr2=0x56106208bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.140+0000 7f73bd7fa700 0 client.16203 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.141+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 shutdown_connections 2023-10-08T09:18:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.141+0000 7f73e8404b80 1 -- 172.21.15.49:0/2422954632 wait complete. 2023-10-08T09:18:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.141+0000 7f73e8404b80 20 client.16203 trim_cache size 0 max 16384 2023-10-08T09:18:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:18:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:18:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:18:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73c00a1550 msgr2=0x7f73c00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73c00a1550 0x7f73c00a3a10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f73d0015a80 tx=0x7f73d00097a0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410a160 msgr2=0x7f73d41a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410a160 0x7f73d41a5b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f73c400b550 tx=0x7f73c400b860 comp rx=0 tx=0).stop 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 shutdown_connections 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73c00a1550 0x7f73c00a3a10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73d41077f0 0x7f73d41a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d410cb30 0x7f73d41acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 --2- 172.21.15.49:0/2017832093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73d410a160 0x7f73d41a5b50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 >> 172.21.15.49:0/2017832093 conn(0x7f73d406d770 msgr2=0x7f73d410a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 shutdown_connections 2023-10-08T09:18:39.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:39.144+0000 7f73dc21b700 1 -- 172.21.15.49:0/2017832093 wait complete. 2023-10-08T09:18:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 195.0 s, 813.0 tps, lat 123.493 ms stddev 146.716 2023-10-08T09:18:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:39 smithi049 ceph-mon[99485]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 721 op/s 2023-10-08T09:18:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:39 smithi184 ceph-mon[103675]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 721 op/s 2023-10-08T09:18:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:39 smithi132 ceph-mon[103389]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 10 MiB/s wr, 721 op/s 2023-10-08T09:18:41.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:41 smithi049 ceph-mon[99485]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 823 op/s 2023-10-08T09:18:41.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:41 smithi184 ceph-mon[103675]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 823 op/s 2023-10-08T09:18:42.033 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:18:42.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:41 smithi132 ceph-mon[103389]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 MiB/s wr, 823 op/s 2023-10-08T09:18:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.258+0000 7f81a90c4700 1 -- 172.21.15.49:0/3707681777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100c30 msgr2=0x7f81a4101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.258+0000 7f81a90c4700 1 --2- 172.21.15.49:0/3707681777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100c30 0x7f81a4101050 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8194009990 tx=0x7f8194009ca0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.259+0000 7f81a90c4700 1 -- 172.21.15.49:0/3707681777 shutdown_connections 2023-10-08T09:18:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.259+0000 7f81a90c4700 1 --2- 172.21.15.49:0/3707681777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a4100300 0x7f81a4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.259+0000 7f81a90c4700 1 --2- 172.21.15.49:0/3707681777 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100c30 0x7f81a4101050 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.259+0000 7f81a90c4700 1 --2- 172.21.15.49:0/3707681777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 0x7f81a40ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.259+0000 7f81a90c4700 1 -- 172.21.15.49:0/3707681777 >> 172.21.15.49:0/3707681777 conn(0x7f81a40fa390 msgr2=0x7f81a40fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.259+0000 7f81a90c4700 1 -- 172.21.15.49:0/3707681777 shutdown_connections 2023-10-08T09:18:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.259+0000 7f81a90c4700 1 -- 172.21.15.49:0/3707681777 wait complete. 2023-10-08T09:18:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.260+0000 7f81a90c4700 1 Processor -- start 2023-10-08T09:18:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a90c4700 1 -- start start 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a90c4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 0x7f81a40fea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a90c4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100300 0x7f81a40fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a90c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a4100c30 0x7f81a40714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a90c4700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4071b30 con 0x7f81a4100300 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a90c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4071ca0 con 0x7f81a4100c30 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a90c4700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a4071e60 con 0x7f81a40ff960 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 0x7f81a40fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100300 0x7f81a40fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a4100c30 0x7f81a40714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 0x7f81a40fea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38396/0 (socket says 172.21.15.49:38396) 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100300 0x7f81a40fefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51362/0 (socket says 172.21.15.49:51362) 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.261+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2422928125 learned_addr learned my addr 172.21.15.49:0/2422928125 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2422928125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a4100c30 msgr2=0x7f81a40714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a4100c30 0x7f81a40714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100300 msgr2=0x7f81a40fefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100300 0x7f81a40fefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2422928125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81940095f0 con 0x7f81a40ff960 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 0x7f81a40fea70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f819400b680 tx=0x7f8194004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:42.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f818bfff700 1 -- 172.21.15.49:0/2422928125 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819401d070 con 0x7f81a40ff960 2023-10-08T09:18:42.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f818bfff700 1 -- 172.21.15.49:0/2422928125 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8194021410 con 0x7f81a40ff960 2023-10-08T09:18:42.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a40720e0 con 0x7f81a40ff960 2023-10-08T09:18:42.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f818bfff700 1 -- 172.21.15.49:0/2422928125 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194017650 con 0x7f81a40ff960 2023-10-08T09:18:42.265 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.262+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a4072590 con 0x7f81a40ff960 2023-10-08T09:18:42.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.265+0000 7f818bfff700 1 -- 172.21.15.49:0/2422928125 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f81940177b0 con 0x7f81a40ff960 2023-10-08T09:18:42.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.265+0000 7f818bfff700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f818c0a1480 0x7f818c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.265+0000 7f818bfff700 1 -- 172.21.15.49:0/2422928125 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f81940ce2f0 con 0x7f81a40ff960 2023-10-08T09:18:42.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.265+0000 7f81a259c700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f818c0a1480 0x7f818c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.266+0000 7f81b52adb80 1 Processor -- start 2023-10-08T09:18:42.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.266+0000 7f81b52adb80 1 -- start start 2023-10-08T09:18:42.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.266+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.266+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a09e0 0x55a9d62a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.266+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a17d0 0x55a9d62a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.267+0000 7f81b52adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d620b460 con 0x55a9d62a0080 2023-10-08T09:18:42.267 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.267+0000 7f81b52adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d61fcc20 con 0x55a9d62a17d0 2023-10-08T09:18:42.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.267+0000 7f81b52adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d61fa540 con 0x55a9d62a09e0 2023-10-08T09:18:42.268 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.267+0000 7f81a259c700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f818c0a1480 0x7f818c0a3940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f81a406f0d0 tx=0x7f8198009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.267+0000 7f81a259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a09e0 0x55a9d62a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.267+0000 7f81a259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a09e0 0x55a9d62a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38406/0 (socket says 172.21.15.49:38406) 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.267+0000 7f81a259c700 1 -- 172.21.15.49:0/3243204760 learned_addr learned my addr 172.21.15.49:0/3243204760 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.268+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.268+0000 7f81a2d9d700 1 -- 172.21.15.49:0/3243204760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a09e0 msgr2=0x55a9d62a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.268+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a09e0 0x55a9d62a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.268+0000 7f81a2d9d700 1 -- 172.21.15.49:0/3243204760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a17d0 msgr2=0x55a9d62a1c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:18:42.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.268+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a17d0 0x55a9d62a1c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.268+0000 7f81a2d9d700 1 -- 172.21.15.49:0/3243204760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9d6020de0 con 0x55a9d62a0080 2023-10-08T09:18:42.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.268+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62a04a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f81940cbbb0 tx=0x7f8194009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f10aedbc486d19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.278+0000 7f81897fa700 1 -- 172.21.15.49:0/3243204760 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819409b410 con 0x55a9d62a0080 2023-10-08T09:18:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.278+0000 7f81897fa700 1 -- 172.21.15.49:0/3243204760 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f819409f560 con 0x55a9d62a0080 2023-10-08T09:18:42.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.278+0000 7f81897fa700 1 -- 172.21.15.49:0/3243204760 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194021680 con 0x55a9d62a0080 2023-10-08T09:18:42.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.279+0000 7f81b52adb80 1 -- 172.21.15.49:0/3243204760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 msgr2=0x55a9d62a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.279+0000 7f81b52adb80 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62a04a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f81940cbbb0 tx=0x7f8194009310 comp rx=0 tx=0).stop 2023-10-08T09:18:42.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.279+0000 7f81b52adb80 1 -- 172.21.15.49:0/3243204760 shutdown_connections 2023-10-08T09:18:42.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.279+0000 7f81b52adb80 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a17d0 0x55a9d62a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.279+0000 7f81b52adb80 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62a04a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.279+0000 7f81b52adb80 1 --2- 172.21.15.49:0/3243204760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a09e0 0x55a9d62a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.279+0000 7f81b52adb80 1 -- 172.21.15.49:0/3243204760 >> 172.21.15.49:0/3243204760 conn(0x55a9d6217260 msgr2=0x55a9d629de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:42.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.280+0000 7f81b52adb80 1 -- 172.21.15.49:0/3243204760 shutdown_connections 2023-10-08T09:18:42.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 -- 172.21.15.49:0/3243204760 wait complete. 2023-10-08T09:18:42.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 -- start start 2023-10-08T09:18:42.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 Processor -- start 2023-10-08T09:18:42.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62bdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62be0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 0x55a9d62c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d61f5e30 con 0x55a9d62a0080 2023-10-08T09:18:42.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d6192710 con 0x55a9d62a09e0 2023-10-08T09:18:42.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.283+0000 7f81b52adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d61923b0 con 0x55a9d62a17d0 2023-10-08T09:18:42.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.284+0000 7f81b52adb80 10 client.0 _ll_register_callbacks cb 0x55a9d5fef800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:42.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.286+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.286+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62bdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51378/0 (socket says 172.21.15.49:51378) 2023-10-08T09:18:42.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.286+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2390266920 learned_addr learned my addr 172.21.15.49:0/2390266920 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:42.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.287+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2390266920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 msgr2=0x55a9d62c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:42.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.287+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 0x55a9d62c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.287+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2390266920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 msgr2=0x55a9d62be0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:42.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.287+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62be0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.287+0000 7f81a2d9d700 1 -- 172.21.15.49:0/2390266920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a4072590 con 0x55a9d62a0080 2023-10-08T09:18:42.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.287+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62bdba0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8194097910 tx=0x7f819409b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3e6b1df8dec627 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:42.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.291+0000 7f8186ff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:18:42.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.291+0000 7f8186ff5700 1 -- 172.21.15.49:0/2390266920 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819409f340 con 0x55a9d62a0080 2023-10-08T09:18:42.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.291+0000 7f8186ff5700 1 -- 172.21.15.49:0/2390266920 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f81940b3e40 con 0x55a9d62a0080 2023-10-08T09:18:42.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.291+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9d61bdeb0 con 0x55a9d62a0080 2023-10-08T09:18:42.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f8186ff5700 1 -- 172.21.15.49:0/2390266920 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81940a3710 con 0x55a9d62a0080 2023-10-08T09:18:42.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 10 client.16215 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:42.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a9d6009c60 con 0x55a9d62a0080 2023-10-08T09:18:42.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f8186ff5700 1 -- 172.21.15.49:0/2390266920 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9d61bdeb0 con 0x55a9d62a0080 2023-10-08T09:18:42.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f8186ff5700 1 -- 172.21.15.49:0/2390266920 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f81940a3280 con 0x55a9d62a0080 2023-10-08T09:18:42.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 10 client.16215 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:42.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:42.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 20 client.16215 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:42.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d62ae360 0x55a9d62b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 4 client.16215 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:18:42.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.292+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9d61eee10 con 0x55a9d62ae360 2023-10-08T09:18:42.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.295+0000 7f81a259c700 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d62ae360 0x55a9d62b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.299+0000 7f81a259c700 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d62ae360 0x55a9d62b0780 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=658b2d8212865dfb server_cookie=f091f437ba2cc771 in_seq=0 out_seq=0 2023-10-08T09:18:42.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.306+0000 7f8186ff5700 10 client.16215 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:42.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.307+0000 7f8186ff5700 1 -- 172.21.15.49:0/2390266920 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a9d61eee10 con 0x55a9d62ae360 2023-10-08T09:18:42.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.307+0000 7f8186ff5700 10 client.16215 handle_command_reply: tid=0 2023-10-08T09:18:42.310 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.307+0000 7f81b52adb80 1 client.16215 shutdown 2023-10-08T09:18:42.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.336+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 msgr2=0x55a9d62bdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.336+0000 7f81b52adb80 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62bdba0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8194097910 tx=0x7f819409b050 comp rx=0 tx=0).stop 2023-10-08T09:18:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.339+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 shutdown_connections 2023-10-08T09:18:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.339+0000 7f81b52adb80 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d62ae360 0x55a9d62b0780 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.339+0000 7f81b52adb80 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 0x55a9d62c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.339+0000 7f81b52adb80 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62bdba0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.339+0000 7f81b52adb80 1 --2- 172.21.15.49:0/2390266920 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62be0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.339+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 >> 172.21.15.49:0/2390266920 conn(0x55a9d6217260 msgr2=0x55a9d629d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.339+0000 7f8186ff5700 0 client.16215 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.341+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 shutdown_connections 2023-10-08T09:18:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.341+0000 7f81b52adb80 1 -- 172.21.15.49:0/2390266920 wait complete. 2023-10-08T09:18:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.341+0000 7f81b52adb80 20 client.16215 trim_cache size 0 max 16384 2023-10-08T09:18:42.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.370+0000 7f81b52adb80 1 Processor -- start 2023-10-08T09:18:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81b52adb80 1 -- start start 2023-10-08T09:18:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a0080 0x55a9d62bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a17d0 0x55a9d62bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81b52adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d6076920 con 0x55a9d62a17d0 2023-10-08T09:18:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81b52adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d61eee10 con 0x55a9d62a09e0 2023-10-08T09:18:42.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81b52adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d61bdeb0 con 0x55a9d62a0080 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a0080 0x55a9d62bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a0080 0x55a9d62bd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38418/0 (socket says 172.21.15.49:38418) 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81a2d9d700 1 -- 172.21.15.49:0/1947920505 learned_addr learned my addr 172.21.15.49:0/1947920505 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81a259c700 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.371+0000 7f81a359e700 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a17d0 0x55a9d62bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f81a259c700 1 -- 172.21.15.49:0/1947920505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a0080 msgr2=0x55a9d62bd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f81a259c700 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a0080 0x55a9d62bd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f81a259c700 1 -- 172.21.15.49:0/1947920505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a17d0 msgr2=0x55a9d62bd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f81a259c700 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a17d0 0x55a9d62bd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f81a259c700 1 -- 172.21.15.49:0/1947920505 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9d6047250 con 0x55a9d62a09e0 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f81a259c700 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c50b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55a9d62b1030 tx=0x7f8198013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed546874cc19133 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:42.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f8189ffb700 1 -- 172.21.15.49:0/1947920505 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198022040 con 0x55a9d62a09e0 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f8189ffb700 1 -- 172.21.15.49:0/1947920505 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81980275b0 con 0x55a9d62a09e0 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.372+0000 7f8189ffb700 1 -- 172.21.15.49:0/1947920505 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198003dd0 con 0x55a9d62a09e0 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 -- 172.21.15.49:0/1947920505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 msgr2=0x55a9d62c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c50b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55a9d62b1030 tx=0x7f8198013040 comp rx=0 tx=0).stop 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 -- 172.21.15.49:0/1947920505 shutdown_connections 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a0080 0x55a9d62bd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a17d0 0x55a9d62bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1947920505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c50b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 -- 172.21.15.49:0/1947920505 >> 172.21.15.49:0/1947920505 conn(0x55a9d6217260 msgr2=0x55a9d62bbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 -- 172.21.15.49:0/1947920505 shutdown_connections 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.373+0000 7f81b52adb80 1 -- 172.21.15.49:0/1947920505 wait complete. 2023-10-08T09:18:42.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 -- start start 2023-10-08T09:18:42.377 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 Processor -- start 2023-10-08T09:18:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 0x55a9d62c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d62c3770 con 0x55a9d62a0080 2023-10-08T09:18:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d62c38e0 con 0x55a9d62a09e0 2023-10-08T09:18:42.379 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9d62c3aa0 con 0x55a9d62a17d0 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81b52adb80 10 client.0 _ll_register_callbacks cb 0x7f8198011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81a2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51390/0 (socket says 172.21.15.49:51390) 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.374+0000 7f81a2d9d700 1 -- 172.21.15.49:0/1077176902 learned_addr learned my addr 172.21.15.49:0/1077176902 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a359e700 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 0x55a9d62c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a259c700 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a2d9d700 1 -- 172.21.15.49:0/1077176902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 msgr2=0x55a9d62c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 0x55a9d62c30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a2d9d700 1 -- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 msgr2=0x55a9d62c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a2d9d700 1 -- 172.21.15.49:0/1077176902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8198017070 con 0x55a9d62a0080 2023-10-08T09:18:42.380 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81a2d9d700 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62c49e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8194097e80 tx=0x7f81940a14e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee08a0d738f90f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f8186ff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f8186ff5700 1 -- 172.21.15.49:0/1077176902 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194097030 con 0x55a9d62a0080 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f8186ff5700 1 -- 172.21.15.49:0/1077176902 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f81940b3e40 con 0x55a9d62a0080 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9d61eeb30 con 0x55a9d62a0080 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.375+0000 7f8186ff5700 1 -- 172.21.15.49:0/1077176902 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81940a3710 con 0x55a9d62a0080 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 10 client.16221 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a9d601f2b0 con 0x55a9d62a0080 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f8186ff5700 1 -- 172.21.15.49:0/1077176902 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9d61eeb30 con 0x55a9d62a0080 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f8186ff5700 1 -- 172.21.15.49:0/1077176902 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f819409a900 con 0x55a9d62a0080 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 10 client.16221 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 20 client.16221 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:42.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d632f990 0x55a9d6331d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 4 client.16221 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:18:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9d6041f70 con 0x55a9d632f990 2023-10-08T09:18:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.376+0000 7f81a259c700 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d632f990 0x55a9d6331d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.377+0000 7f8186ff5700 10 client.16221 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.377+0000 7f81a259c700 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d632f990 0x55a9d6331d80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d882b74b79bdbf4 server_cookie=a932b06658edea3a in_seq=0 out_seq=0 2023-10-08T09:18:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.377+0000 7f8186ff5700 1 -- 172.21.15.49:0/1077176902 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a9d6041f70 con 0x55a9d632f990 2023-10-08T09:18:42.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.377+0000 7f8186ff5700 10 client.16221 handle_command_reply: tid=0 2023-10-08T09:18:42.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.380+0000 7f81b52adb80 1 client.16221 shutdown 2023-10-08T09:18:42.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.398+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 msgr2=0x55a9d62c49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.398+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62c49e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8194097e80 tx=0x7f81940a14e0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 shutdown_connections 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a9d632f990 0x55a9d6331d80 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a9d62a17d0 0x55a9d62c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a9d62a0080 0x55a9d62c49e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 --2- 172.21.15.49:0/1077176902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a9d62a09e0 0x55a9d62c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 >> 172.21.15.49:0/1077176902 conn(0x55a9d6217260 msgr2=0x55a9d62bd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f8186ff5700 0 client.16221 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:42.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 shutdown_connections 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.403+0000 7f81b52adb80 1 -- 172.21.15.49:0/1077176902 wait complete. 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.404+0000 7f81b52adb80 20 client.16221 trim_cache size 0 max 16384 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:18:42.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:18:42.406 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:18:42.406 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:18:42.406 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:18:42.406 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:18:42.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.404+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f818c0a1480 msgr2=0x7f818c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.404+0000 7f81a90c4700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f818c0a1480 0x7f818c0a3940 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f81a406f0d0 tx=0x7f8198009260 comp rx=0 tx=0).stop 2023-10-08T09:18:42.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.404+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 msgr2=0x7f81a40fea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.404+0000 7f81a90c4700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 0x7f81a40fea70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f819400b680 tx=0x7f8194004730 comp rx=0 tx=0).stop 2023-10-08T09:18:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.405+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 shutdown_connections 2023-10-08T09:18:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.405+0000 7f81a90c4700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f818c0a1480 0x7f818c0a3940 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.405+0000 7f81a90c4700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81a40ff960 0x7f81a40fea70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.405+0000 7f81a90c4700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81a4100c30 0x7f81a40714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.405+0000 7f81a90c4700 1 --2- 172.21.15.49:0/2422928125 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f81a4100300 0x7f81a40fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:42.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.405+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 >> 172.21.15.49:0/2422928125 conn(0x7f81a40fa390 msgr2=0x7f81a4103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:42.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.408+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 shutdown_connections 2023-10-08T09:18:42.409 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:18:42.409+0000 7f81a90c4700 1 -- 172.21.15.49:0/2422928125 wait complete. 2023-10-08T09:18:42.430 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:18:43.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:43 smithi049 ceph-mon[99485]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 13 MiB/s wr, 798 op/s 2023-10-08T09:18:43.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:43 smithi184 ceph-mon[103675]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 13 MiB/s wr, 798 op/s 2023-10-08T09:18:44.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:43 smithi132 ceph-mon[103389]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 13 MiB/s wr, 798 op/s 2023-10-08T09:18:44.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 200.0 s, 950.9 tps, lat 105.199 ms stddev 123.725 2023-10-08T09:18:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:45 smithi049 ceph-mon[99485]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 738 op/s 2023-10-08T09:18:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:45 smithi184 ceph-mon[103675]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 738 op/s 2023-10-08T09:18:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:45 smithi132 ceph-mon[103389]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 12 MiB/s wr, 738 op/s 2023-10-08T09:18:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:47 smithi184 ceph-mon[103675]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 741 op/s 2023-10-08T09:18:48.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:47 smithi132 ceph-mon[103389]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 741 op/s 2023-10-08T09:18:48.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:47 smithi049 ceph-mon[99485]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 741 op/s 2023-10-08T09:18:49.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 205.0 s, 1004.6 tps, lat 101.615 ms stddev 133.516 2023-10-08T09:18:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:49 smithi184 ceph-mon[103675]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 MiB/s wr, 600 op/s 2023-10-08T09:18:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:49 smithi132 ceph-mon[103389]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 MiB/s wr, 600 op/s 2023-10-08T09:18:50.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:49 smithi049 ceph-mon[99485]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 11 MiB/s wr, 600 op/s 2023-10-08T09:18:51.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:51 smithi184 ceph-mon[103675]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 648 op/s 2023-10-08T09:18:52.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:51 smithi132 ceph-mon[103389]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 648 op/s 2023-10-08T09:18:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:51 smithi049 ceph-mon[99485]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 648 op/s 2023-10-08T09:18:52.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.328+0000 7fdd64629700 1 -- 172.21.15.49:0/119202738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0725e0 msgr2=0x7fdd5c072a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.328+0000 7fdd64629700 1 --2- 172.21.15.49:0/119202738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0725e0 0x7fdd5c072a00 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400b4f0 tx=0x7fdd5400b800 comp rx=0 tx=0).stop 2023-10-08T09:18:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.328+0000 7fdd64629700 1 -- 172.21.15.49:0/119202738 shutdown_connections 2023-10-08T09:18:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.328+0000 7fdd64629700 1 --2- 172.21.15.49:0/119202738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5c077ee0 0x7fdd5c07a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.328+0000 7fdd64629700 1 --2- 172.21.15.49:0/119202738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c0725e0 0x7fdd5c072a00 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.328+0000 7fdd64629700 1 --2- 172.21.15.49:0/119202738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd5c075510 0x7fdd5c0779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.328+0000 7fdd64629700 1 -- 172.21.15.49:0/119202738 >> 172.21.15.49:0/119202738 conn(0x7fdd5c06d770 msgr2=0x7fdd5c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.329+0000 7fdd64629700 1 -- 172.21.15.49:0/119202738 shutdown_connections 2023-10-08T09:18:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.329+0000 7fdd64629700 1 -- 172.21.15.49:0/119202738 wait complete. 2023-10-08T09:18:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.329+0000 7fdd64629700 1 Processor -- start 2023-10-08T09:18:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.329+0000 7fdd64629700 1 -- start start 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd64629700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd5c0725e0 0x7fdd5c1b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd64629700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5c075510 0x7fdd5c1b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd64629700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c077ee0 0x7fdd5c1b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd64629700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1b7bb0 con 0x7fdd5c077ee0 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd64629700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1b7cf0 con 0x7fdd5c0725e0 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd64629700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1b7e80 con 0x7fdd5c075510 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c077ee0 0x7fdd5c1b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd623c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd5c0725e0 0x7fdd5c1b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd61bc4700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5c075510 0x7fdd5c1b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c077ee0 0x7fdd5c1b75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51408/0 (socket says 172.21.15.49:51408) 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 -- 172.21.15.49:0/3071225763 learned_addr learned my addr 172.21.15.49:0/3071225763 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd623c5700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd5c0725e0 0x7fdd5c1b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41018/0 (socket says 172.21.15.49:41018) 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 -- 172.21.15.49:0/3071225763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5c075510 msgr2=0x7fdd5c1b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5c075510 0x7fdd5c1b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 -- 172.21.15.49:0/3071225763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd5c0725e0 msgr2=0x7fdd5c1b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd5c0725e0 0x7fdd5c1b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 -- 172.21.15.49:0/3071225763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd54009d00 con 0x7fdd5c077ee0 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5c075510 0x7fdd5c1b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd62bc6700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c077ee0 0x7fdd5c1b75e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800e940 tx=0x7fdd5800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.330+0000 7fdd537fe700 1 -- 172.21.15.49:0/3071225763 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580097b0 con 0x7fdd5c077ee0 2023-10-08T09:18:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.331+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c1b8160 con 0x7fdd5c077ee0 2023-10-08T09:18:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.331+0000 7fdd537fe700 1 -- 172.21.15.49:0/3071225763 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd58005020 con 0x7fdd5c077ee0 2023-10-08T09:18:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.331+0000 7fdd537fe700 1 -- 172.21.15.49:0/3071225763 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800cc40 con 0x7fdd5c077ee0 2023-10-08T09:18:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.331+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c1b8670 con 0x7fdd5c077ee0 2023-10-08T09:18:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.334+0000 7fdd537fe700 1 -- 172.21.15.49:0/3071225763 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdd58021440 con 0x7fdd5c077ee0 2023-10-08T09:18:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.335+0000 7fdd537fe700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd480a14d0 0x7fdd480a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.335+0000 7fdd537fe700 1 -- 172.21.15.49:0/3071225763 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdd58014070 con 0x7fdd5c077ee0 2023-10-08T09:18:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.335+0000 7fdd623c5700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd480a14d0 0x7fdd480a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.335+0000 7fdd70812b80 1 Processor -- start 2023-10-08T09:18:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd70812b80 1 -- start start 2023-10-08T09:18:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1ea9d60 0x55f0d1eaa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1eb7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd70812b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1e12400 con 0x55f0d1eb7d60 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd70812b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1e03bc0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd70812b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1e014e0 con 0x55f0d1ea9d60 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd61bc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1eb7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd61bc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1eb7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41026/0 (socket says 172.21.15.49:41026) 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd61bc4700 1 -- 172.21.15.49:0/2357127593 learned_addr learned my addr 172.21.15.49:0/2357127593 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd623c5700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd480a14d0 0x7fdd480a3990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007e10 tx=0x7fdd5400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd623c5700 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1ea9d60 0x55f0d1eaa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.336+0000 7fdd62bc6700 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd61bc4700 1 -- 172.21.15.49:0/2357127593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1ea9d60 msgr2=0x55f0d1eaa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1ea9d60 0x55f0d1eaa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd61bc4700 1 -- 172.21.15.49:0/2357127593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eb7d60 msgr2=0x55f0d1eba190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd61bc4700 1 -- 172.21.15.49:0/2357127593 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d1c27d80 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1eb7820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0115f0 tx=0x7fdd4c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef976f45bfd4aaa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd50ff9700 1 -- 172.21.15.49:0/2357127593 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c019630 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd50ff9700 1 -- 172.21.15.49:0/2357127593 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c019c50 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.337+0000 7fdd50ff9700 1 -- 172.21.15.49:0/2357127593 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00ece0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.338+0000 7fdd70812b80 1 -- 172.21.15.49:0/2357127593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 msgr2=0x55f0d1eb7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.338+0000 7fdd70812b80 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1eb7820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0115f0 tx=0x7fdd4c011900 comp rx=0 tx=0).stop 2023-10-08T09:18:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.338+0000 7fdd70812b80 1 -- 172.21.15.49:0/2357127593 shutdown_connections 2023-10-08T09:18:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.338+0000 7fdd70812b80 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1ea9d60 0x55f0d1eaa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.338+0000 7fdd70812b80 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.338+0000 7fdd70812b80 1 --2- 172.21.15.49:0/2357127593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1eb7820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.338+0000 7fdd70812b80 1 -- 172.21.15.49:0/2357127593 >> 172.21.15.49:0/2357127593 conn(0x55f0d1e1e200 msgr2=0x55f0d1ea48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.339+0000 7fdd70812b80 1 -- 172.21.15.49:0/2357127593 shutdown_connections 2023-10-08T09:18:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.339+0000 7fdd70812b80 1 -- 172.21.15.49:0/2357127593 wait complete. 2023-10-08T09:18:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.339+0000 7fdd70812b80 1 -- start start 2023-10-08T09:18:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.339+0000 7fdd70812b80 1 Processor -- start 2023-10-08T09:18:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd70812b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1dfcdd0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd70812b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d996b0 con 0x55f0d1ea9d60 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd70812b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d99350 con 0x55f0d1eb7d60 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd70812b80 10 client.0 _ll_register_callbacks cb 0x55f0d1bf67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51420/0 (socket says 172.21.15.49:51420) 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 -- 172.21.15.49:0/1685975760 learned_addr learned my addr 172.21.15.49:0/1685975760 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd62bc6700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd623c5700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 -- 172.21.15.49:0/1685975760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 msgr2=0x55f0d1ec35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 -- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1ea9d60 msgr2=0x55f0d1ebc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd61bc4700 1 -- 172.21.15.49:0/1685975760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c009dd0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd62bc6700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.340+0000 7fdd623c5700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebc570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c011d00 tx=0x7fdd4c00e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd888331a4c8743 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd45ffb700 1 -- 172.21.15.49:0/1685975760 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004460 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd45ffb700 1 -- 172.21.15.49:0/1685975760 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdd4c005ce0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0d1dc4e50 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd45ffb700 1 -- 172.21.15.49:0/1685975760 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4c019430 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd70812b80 10 client.16233 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f0d1c10c00 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.341+0000 7fdd45ffb700 1 -- 172.21.15.49:0/1685975760 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0d1dc4e50 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd45ffb700 1 -- 172.21.15.49:0/1685975760 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdd4c028020 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd70812b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd70812b80 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd70812b80 20 client.16233 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd70812b80 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1ec7e80 0x55f0d1eca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd70812b80 4 client.16233 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0d1df5db0 con 0x55f0d1ec7e80 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.342+0000 7fdd623c5700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1ec7e80 0x55f0d1eca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.343+0000 7fdd45ffb700 10 client.16233 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.343+0000 7fdd623c5700 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1ec7e80 0x55f0d1eca2a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e24486c478ddc07 server_cookie=ee48017fb99873e8 in_seq=0 out_seq=0 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd45ffb700 1 -- 172.21.15.49:0/1685975760 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f0d1df5db0 con 0x55f0d1ec7e80 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd45ffb700 10 client.16233 handle_command_reply: tid=0 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 client.16233 shutdown 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eaa6c0 msgr2=0x55f0d1ebc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebc570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c011d00 tx=0x7fdd4c00e5e0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 shutdown_connections 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1ec7e80 0x55f0d1eca2a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 --2- 172.21.15.49:0/1685975760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebc570 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 >> 172.21.15.49:0/1685975760 conn(0x55f0d1e1e200 msgr2=0x55f0d1ea4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd45ffb700 0 client.16233 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 shutdown_connections 2023-10-08T09:18:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 1 -- 172.21.15.49:0/1685975760 wait complete. 2023-10-08T09:18:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.344+0000 7fdd70812b80 20 client.16233 trim_cache size 0 max 16384 2023-10-08T09:18:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 Processor -- start 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 -- start start 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ec3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ec3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1c7d8c0 con 0x55f0d1ea9d60 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1ebbba0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.381+0000 7fdd70812b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1ebccb0 con 0x55f0d1eb7d60 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ec3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ec3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41042/0 (socket says 172.21.15.49:41042) 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 -- 172.21.15.49:0/3799631947 learned_addr learned my addr 172.21.15.49:0/3799631947 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd62bc6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 -- 172.21.15.49:0/3799631947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 msgr2=0x55f0d1eba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 -- 172.21.15.49:0/3799631947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 msgr2=0x55f0d1ec3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ec3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 -- 172.21.15.49:0/3799631947 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d1c4e1f0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd62bc6700 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ec3570 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005a80 tx=0x7fdd4c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91981b5c6bfa9f20 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.382+0000 7fdd517fa700 1 -- 172.21.15.49:0/3799631947 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c011db0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd517fa700 1 -- 172.21.15.49:0/3799631947 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c005ce0 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd517fa700 1 -- 172.21.15.49:0/3799631947 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c019650 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd70812b80 1 -- 172.21.15.49:0/3799631947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 msgr2=0x55f0d1ec3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ec3570 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005a80 tx=0x7fdd4c011900 comp rx=0 tx=0).stop 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd70812b80 1 -- 172.21.15.49:0/3799631947 shutdown_connections 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eb7d60 0x55f0d1eba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ec3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3799631947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ec3570 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.383+0000 7fdd70812b80 1 -- 172.21.15.49:0/3799631947 >> 172.21.15.49:0/3799631947 conn(0x55f0d1e1e200 msgr2=0x55f0d1ecebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 -- 172.21.15.49:0/3799631947 shutdown_connections 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 -- 172.21.15.49:0/3799631947 wait complete. 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 -- start start 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 Processor -- start 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1ec2130 con 0x55f0d1ea9d60 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1ec22a0 con 0x55f0d1eb7d60 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1ec2460 con 0x55f0d1eaa6c0 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.384+0000 7fdd70812b80 10 client.0 _ll_register_callbacks cb 0x7fdd4c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebe7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51432/0 (socket says 172.21.15.49:51432) 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 -- 172.21.15.49:0/3124895428 learned_addr learned my addr 172.21.15.49:0/3124895428 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 -- 172.21.15.49:0/3124895428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eaa6c0 msgr2=0x55f0d1ebed10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 -- 172.21.15.49:0/3124895428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eb7d60 msgr2=0x55f0d1ec1bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:18:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec1bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 -- 172.21.15.49:0/3124895428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c1b8670 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd623c5700 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebe7d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000f80 tx=0x7fdd540099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3aa9c1ebd6bd22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.385+0000 7fdd45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd45ffb700 1 -- 172.21.15.49:0/3124895428 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400e040 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd45ffb700 1 -- 172.21.15.49:0/3124895428 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdd54005380 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd4c019c50 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd45ffb700 1 -- 172.21.15.49:0/3124895428 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd54005560 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 10 client.16236 fetch_fsmap learned FSMap version 37 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f0d1c10c00 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd45ffb700 1 -- 172.21.15.49:0/3124895428 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c019c50 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd45ffb700 1 -- 172.21.15.49:0/3124895428 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdd54012be0 con 0x55f0d1ea9d60 2023-10-08T09:18:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:18:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:18:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 20 client.16236 populate_metadata read hostname 'smithi049' 2023-10-08T09:18:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1eaf2c0 0x55f0d1eb1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:18:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.386+0000 7fdd70812b80 4 client.16236 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:18:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.387+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0d1df5ad0 con 0x55f0d1eaf2c0 2023-10-08T09:18:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.391+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1eaf2c0 0x55f0d1eb1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:18:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.391+0000 7fdd61bc4700 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1eaf2c0 0x55f0d1eb1780 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c0e23bbd88e0f8 server_cookie=462d93969bb3fb96 in_seq=0 out_seq=0 2023-10-08T09:18:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.391+0000 7fdd45ffb700 10 client.16236 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd45ffb700 1 -- 172.21.15.49:0/3124895428 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0d1df5ad0 con 0x55f0d1eaf2c0 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd45ffb700 10 client.16236 handle_command_reply: tid=0 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 client.16236 shutdown 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 msgr2=0x55f0d1ebe7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebe7d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000f80 tx=0x7fdd540099e0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 shutdown_connections 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f0d1eaf2c0 0x55f0d1eb1780 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f0d1eb7d60 0x55f0d1ec1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f0d1ea9d60 0x55f0d1ebe7d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 --2- 172.21.15.49:0/3124895428 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f0d1eaa6c0 0x55f0d1ebed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.444+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 >> 172.21.15.49:0/3124895428 conn(0x55f0d1e1e200 msgr2=0x55f0d1eba150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.447+0000 7fdd45ffb700 0 client.16236 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:18:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.447+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 shutdown_connections 2023-10-08T09:18:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd70812b80 1 -- 172.21.15.49:0/3124895428 wait complete. 2023-10-08T09:18:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd70812b80 20 client.16236 trim_cache size 0 max 16384 2023-10-08T09:18:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:18:52.448 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:18:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:18:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:18:52.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd480a14d0 msgr2=0x7fdd480a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd480a14d0 0x7fdd480a3990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdd54007e10 tx=0x7fdd5400b040 comp rx=0 tx=0).stop 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c077ee0 msgr2=0x7fdd5c1b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c077ee0 0x7fdd5c1b75e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800e940 tx=0x7fdd5800ec50 comp rx=0 tx=0).stop 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 shutdown_connections 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdd480a14d0 0x7fdd480a3990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdd5c077ee0 0x7fdd5c1b75e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd5c0725e0 0x7fdd5c1b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 --2- 172.21.15.49:0/3071225763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5c075510 0x7fdd5c1b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:18:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 >> 172.21.15.49:0/3071225763 conn(0x7fdd5c06d770 msgr2=0x7fdd5c079500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:18:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.448+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 shutdown_connections 2023-10-08T09:18:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:18:52.449+0000 7fdd64629700 1 -- 172.21.15.49:0/3071225763 wait complete. 2023-10-08T09:18:53.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:53 smithi184 ceph-mon[103675]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 631 op/s 2023-10-08T09:18:54.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:53 smithi132 ceph-mon[103389]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 631 op/s 2023-10-08T09:18:54.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:53 smithi049 ceph-mon[99485]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 MiB/s wr, 631 op/s 2023-10-08T09:18:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 210.0 s, 922.2 tps, lat 104.898 ms stddev 133.970 2023-10-08T09:18:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:55 smithi184 ceph-mon[103675]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 609 op/s 2023-10-08T09:18:56.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:55 smithi132 ceph-mon[103389]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 609 op/s 2023-10-08T09:18:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:55 smithi049 ceph-mon[99485]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 609 op/s 2023-10-08T09:18:57.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:57 smithi184 ceph-mon[103675]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 667 op/s 2023-10-08T09:18:58.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:57 smithi132 ceph-mon[103389]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 667 op/s 2023-10-08T09:18:58.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:57 smithi049 ceph-mon[99485]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 667 op/s 2023-10-08T09:18:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 215.0 s, 484.6 tps, lat 132.588 ms stddev 152.401 2023-10-08T09:19:00.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:18:59 smithi132 ceph-mon[103389]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 600 op/s 2023-10-08T09:19:00.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:18:59 smithi049 ceph-mon[99485]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 600 op/s 2023-10-08T09:19:00.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:18:59 smithi184 ceph-mon[103675]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 600 op/s 2023-10-08T09:19:02.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:01 smithi132 ceph-mon[103389]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 877 op/s 2023-10-08T09:19:02.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:01 smithi049 ceph-mon[99485]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 877 op/s 2023-10-08T09:19:02.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:01 smithi184 ceph-mon[103675]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 877 op/s 2023-10-08T09:19:04.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:03 smithi049 ceph-mon[99485]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.42k op/s 2023-10-08T09:19:04.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:03 smithi184 ceph-mon[103675]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.42k op/s 2023-10-08T09:19:04.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:03 smithi132 ceph-mon[103389]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.42k op/s 2023-10-08T09:19:04.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 220.0 s, 922.4 tps, lat 148.895 ms stddev 326.050 2023-10-08T09:19:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.640+0000 7fea010f0700 1 -- 172.21.15.49:0/852841874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10cb30 msgr2=0x7fe9fc10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.640+0000 7fea010f0700 1 --2- 172.21.15.49:0/852841874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc10cfb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400b0a0 tx=0x7fe9f400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.640+0000 7fea010f0700 1 -- 172.21.15.49:0/852841874 shutdown_connections 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.640+0000 7fea010f0700 1 --2- 172.21.15.49:0/852841874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc10cfb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.640+0000 7fea010f0700 1 --2- 172.21.15.49:0/852841874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9fc1077f0 0x7fe9fc109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.640+0000 7fea010f0700 1 --2- 172.21.15.49:0/852841874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10a160 0x7fe9fc10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.640+0000 7fea010f0700 1 -- 172.21.15.49:0/852841874 >> 172.21.15.49:0/852841874 conn(0x7fe9fc06d770 msgr2=0x7fe9fc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.641+0000 7fea010f0700 1 -- 172.21.15.49:0/852841874 shutdown_connections 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.641+0000 7fea010f0700 1 -- 172.21.15.49:0/852841874 wait complete. 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 Processor -- start 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 -- start start 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9fc1077f0 0x7fe9fc077a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10a160 0x7fe9fc077fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc07ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc07f550 con 0x7fe9fc10a160 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc07f6c0 con 0x7fe9fc1077f0 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fea010f0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc07f880 con 0x7fe9fc10cb30 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10a160 0x7fe9fc077fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fe9fb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc07ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10a160 0x7fe9fc077fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51446/0 (socket says 172.21.15.49:51446) 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fe9fb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc07ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38482/0 (socket says 172.21.15.49:38482) 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.642+0000 7fe9fa59c700 1 -- 172.21.15.49:0/444871254 learned_addr learned my addr 172.21.15.49:0/444871254 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9fc1077f0 0x7fe9fc077a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9fb59e700 1 -- 172.21.15.49:0/444871254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9fc1077f0 msgr2=0x7fe9fc077a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9fb59e700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9fc1077f0 0x7fe9fc077a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9fb59e700 1 -- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10a160 msgr2=0x7fe9fc077fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9fb59e700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10a160 0x7fe9fc077fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9fb59e700 1 -- 172.21.15.49:0/444871254 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0009520 con 0x7fe9fc10cb30 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9fb59e700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc07ef80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007b90 tx=0x7fe9f4007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.643+0000 7fe9e3fff700 1 -- 172.21.15.49:0/444871254 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400e040 con 0x7fe9fc10cb30 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.644+0000 7fe9e3fff700 1 -- 172.21.15.49:0/444871254 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f4003c00 con 0x7fe9fc10cb30 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.644+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f4009d00 con 0x7fe9fc10cb30 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.644+0000 7fe9e3fff700 1 -- 172.21.15.49:0/444871254 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f401b6c0 con 0x7fe9fc10cb30 2023-10-08T09:19:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.644+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc07fd30 con 0x7fe9fc10cb30 2023-10-08T09:19:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.649+0000 7fea0d2d9b80 1 Processor -- start 2023-10-08T09:19:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fe9e3fff700 1 -- 172.21.15.49:0/444871254 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe9f401d470 con 0x7fe9fc10cb30 2023-10-08T09:19:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fea0d2d9b80 1 -- start start 2023-10-08T09:19:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9ca7a0 0x55f48a9cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a932400 con 0x55f48a9c9e40 2023-10-08T09:19:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a923bc0 con 0x55f48a9cb2b0 2023-10-08T09:19:05.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.650+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a9214e0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.651+0000 7fe9e3fff700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe9e40a1500 0x7fe9e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.651+0000 7fe9e3fff700 1 -- 172.21.15.49:0/444871254 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9f40cdf50 con 0x7fe9fc10cb30 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fb59e700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe9e40a1500 0x7fe9e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9ca7a0 0x55f48a9cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9ca7a0 0x55f48a9cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38486/0 (socket says 172.21.15.49:38486) 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1556392773 learned_addr learned my addr 172.21.15.49:0/1556392773 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1556392773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 msgr2=0x55f48a9cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1556392773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 msgr2=0x55f48a9ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1556392773 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f48a747d80 con 0x55f48a9ca7a0 2023-10-08T09:19:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:19:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.652+0000 7fe9fb59e700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe9e40a1500 0x7fe9e40a39c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007a60 tx=0x7fe9f401dbf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.653+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9ca7a0 0x55f48a9cacd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009d10 tx=0x7fe9ec0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b5f95bf7343fe3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.653+0000 7fe9e17fa700 1 -- 172.21.15.49:0/1556392773 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0097e0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.653+0000 7fe9e17fa700 1 -- 172.21.15.49:0/1556392773 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9ec011ed0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.653+0000 7fe9e17fa700 1 -- 172.21.15.49:0/1556392773 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00f8e0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1556392773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9ca7a0 msgr2=0x55f48a9cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9ca7a0 0x55f48a9cacd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009d10 tx=0x7fe9ec0116c0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1556392773 shutdown_connections 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1556392773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9ca7a0 0x55f48a9cacd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1556392773 >> 172.21.15.49:0/1556392773 conn(0x55f48a93e200 msgr2=0x55f48a9c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1556392773 shutdown_connections 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1556392773 wait complete. 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.654+0000 7fea0d2d9b80 1 -- start start 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 1 Processor -- start 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a91cdd0 con 0x55f48a9c9e40 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a8b96b0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a8b9350 con 0x55f48a9cb2b0 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fea0d2d9b80 10 client.0 _ll_register_callbacks cb 0x55f48a7167a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51460/0 (socket says 172.21.15.49:51460) 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 -- 172.21.15.49:0/3087802845 learned_addr learned my addr 172.21.15.49:0/3087802845 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 -- 172.21.15.49:0/3087802845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 msgr2=0x55f48a9dadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9dadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 -- 172.21.15.49:0/3087802845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9ca7a0 msgr2=0x55f48a9d3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.655+0000 7fe9fa59c700 1 -- 172.21.15.49:0/3087802845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec009940 con 0x55f48a9c9e40 2023-10-08T09:19:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9d38e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0054df0 tx=0x7fe9f0056040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8635b5c24816e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fe9deff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fe9deff5700 1 -- 172.21.15.49:0/3087802845 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0053dd0 con 0x55f48a9c9e40 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f48a8e4e50 con 0x55f48a9c9e40 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fe9deff5700 1 -- 172.21.15.49:0/3087802845 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9f0054e20 con 0x55f48a9c9e40 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fe9deff5700 1 -- 172.21.15.49:0/3087802845 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f0004030 con 0x55f48a9c9e40 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fea0d2d9b80 10 client.16248 fetch_fsmap learned FSMap version 37 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f48a730c00 con 0x55f48a9c9e40 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fe9deff5700 1 -- 172.21.15.49:0/3087802845 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f48a8e4e50 con 0x55f48a9c9e40 2023-10-08T09:19:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.656+0000 7fe9deff5700 1 -- 172.21.15.49:0/3087802845 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe9f0003bb0 con 0x55f48a9c9e40 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.657+0000 7fea0d2d9b80 10 client.16248 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.657+0000 7fea0d2d9b80 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.657+0000 7fea0d2d9b80 20 client.16248 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.657+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48a9df790 0x55f48a9e1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.657+0000 7fea0d2d9b80 4 client.16248 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.657+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f48a915db0 con 0x55f48a9df790 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.657+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48a9df790 0x55f48a9e1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.658+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48a9df790 0x55f48a9e1bb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a42dc534e12c73a server_cookie=42d6f4e25173ba7c in_seq=0 out_seq=0 2023-10-08T09:19:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.658+0000 7fe9deff5700 10 client.16248 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fe9deff5700 1 -- 172.21.15.49:0/3087802845 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f48a915db0 con 0x55f48a9df790 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fe9deff5700 10 client.16248 handle_command_reply: tid=0 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 client.16248 shutdown 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 msgr2=0x55f48a9d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9d38e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0054df0 tx=0x7fe9f0056040 comp rx=0 tx=0).stop 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 shutdown_connections 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48a9df790 0x55f48a9e1bb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9dadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9c9e40 0x55f48a9d38e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fe9deff5700 0 client.16248 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/3087802845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 >> 172.21.15.49:0/3087802845 conn(0x55f48a93e200 msgr2=0x55f48a9c4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 shutdown_connections 2023-10-08T09:19:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/3087802845 wait complete. 2023-10-08T09:19:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.659+0000 7fea0d2d9b80 20 client.16248 trim_cache size 0 max 16384 2023-10-08T09:19:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.682+0000 7fea0d2d9b80 1 Processor -- start 2023-10-08T09:19:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fea0d2d9b80 1 -- start start 2023-10-08T09:19:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9c9e40 0x55f48a9db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9dad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a79d8c0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a8e4e50 con 0x55f48a9cb2b0 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a9d4890 con 0x55f48a9c9e40 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9dad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9dad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51470/0 (socket says 172.21.15.49:51470) 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1129839824 learned_addr learned my addr 172.21.15.49:0/1129839824 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9c9e40 0x55f48a9db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1129839824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9c9e40 msgr2=0x55f48a9db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9c9e40 0x55f48a9db470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1129839824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 msgr2=0x55f48a9d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.683+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1129839824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f48a76e1f0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.684+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9dad80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009ce0 tx=0x7fe9ec005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d393b1d0bcb2c2ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.684+0000 7fe9e1ffb700 1 -- 172.21.15.49:0/1129839824 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0097e0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.684+0000 7fe9e1ffb700 1 -- 172.21.15.49:0/1129839824 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9ec009e60 con 0x55f48a9ca7a0 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.684+0000 7fe9e1ffb700 1 -- 172.21.15.49:0/1129839824 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec0038e0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.685+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1129839824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 msgr2=0x55f48a9dad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.685+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9dad80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009ce0 tx=0x7fe9ec005040 comp rx=0 tx=0).stop 2023-10-08T09:19:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.687+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1129839824 shutdown_connections 2023-10-08T09:19:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.687+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9c9e40 0x55f48a9db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.687+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.687+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1129839824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9dad80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.687+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1129839824 >> 172.21.15.49:0/1129839824 conn(0x55f48a93e200 msgr2=0x55f48a9e64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.689+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1129839824 shutdown_connections 2023-10-08T09:19:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.689+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1129839824 wait complete. 2023-10-08T09:19:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.689+0000 7fea0d2d9b80 1 -- start start 2023-10-08T09:19:05.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.689+0000 7fea0d2d9b80 1 Processor -- start 2023-10-08T09:19:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9c9e40 0x55f48a9d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fea0d2d9b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a9d9fe0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a9da150 con 0x55f48a9c9e40 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fea0d2d9b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f48a9da310 con 0x55f48a9cb2b0 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fea0d2d9b80 10 client.0 _ll_register_callbacks cb 0x7fe9f0054b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51474/0 (socket says 172.21.15.49:51474) 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1471449390 learned_addr learned my addr 172.21.15.49:0/1471449390 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9c9e40 0x55f48a9d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1471449390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 msgr2=0x55f48a9d9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9c9e40 msgr2=0x55f48a9d8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9c9e40 0x55f48a9d8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 -- 172.21.15.49:0/1471449390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f48a747d80 con 0x55f48a9ca7a0 2023-10-08T09:19:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9fad9d700 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d8a90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec012f80 tx=0x7fe9ec012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522acea3281e7eff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.690+0000 7fe9deff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:19:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.691+0000 7fe9deff5700 1 -- 172.21.15.49:0/1471449390 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec039070 con 0x55f48a9ca7a0 2023-10-08T09:19:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.691+0000 7fe9deff5700 1 -- 172.21.15.49:0/1471449390 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9ec012660 con 0x55f48a9ca7a0 2023-10-08T09:19:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9ec011ed0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fe9deff5700 1 -- 172.21.15.49:0/1471449390 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9ec012820 con 0x55f48a9ca7a0 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 10 client.16254 fetch_fsmap learned FSMap version 37 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f48a746250 con 0x55f48a9ca7a0 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fe9deff5700 1 -- 172.21.15.49:0/1471449390 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9ec011ed0 con 0x55f48a9ca7a0 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fe9deff5700 1 -- 172.21.15.49:0/1471449390 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 37) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe9ec03c020 con 0x55f48a9ca7a0 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 10 client.16254 fetch_fsmap finished waiting for FSMap version 37 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 20 client.16254 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48aa33cc0 0x55f48aa360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 4 client.16254 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.692+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f48a915ad0 con 0x55f48aa33cc0 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.693+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48aa33cc0 0x55f48aa360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.693+0000 7fe9deff5700 10 client.16254 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.693+0000 7fe9fa59c700 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48aa33cc0 0x55f48aa360b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ce5edf28354a8b server_cookie=963be67f1446035b in_seq=0 out_seq=0 2023-10-08T09:19:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fe9deff5700 1 -- 172.21.15.49:0/1471449390 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f48a915ad0 con 0x55f48aa33cc0 2023-10-08T09:19:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fe9deff5700 10 client.16254 handle_command_reply: tid=0 2023-10-08T09:19:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 client.16254 shutdown 2023-10-08T09:19:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 msgr2=0x55f48a9d8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d8a90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec012f80 tx=0x7fe9ec012c90 comp rx=0 tx=0).stop 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 shutdown_connections 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f48aa33cc0 0x55f48aa360b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f48a9cb2b0 0x55f48a9d9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f48a9c9e40 0x55f48a9d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 --2- 172.21.15.49:0/1471449390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f48a9ca7a0 0x55f48a9d8a90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 >> 172.21.15.49:0/1471449390 conn(0x55f48a93e200 msgr2=0x55f48a9d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fe9deff5700 0 client.16254 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 shutdown_connections 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.703+0000 7fea0d2d9b80 1 -- 172.21.15.49:0/1471449390 wait complete. 2023-10-08T09:19:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea0d2d9b80 20 client.16254 trim_cache size 0 max 16384 2023-10-08T09:19:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:19:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:19:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:19:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:19:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe9e40a1500 msgr2=0x7fe9e40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe9e40a1500 0x7fe9e40a39c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007a60 tx=0x7fe9f401dbf0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10cb30 msgr2=0x7fe9fc07ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc07ef80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4007b90 tx=0x7fe9f4007930 comp rx=0 tx=0).stop 2023-10-08T09:19:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 shutdown_connections 2023-10-08T09:19:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe9e40a1500 0x7fe9e40a39c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9fc10cb30 0x7fe9fc07ef80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9fc1077f0 0x7fe9fc077a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 --2- 172.21.15.49:0/444871254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe9fc10a160 0x7fe9fc077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 >> 172.21.15.49:0/444871254 conn(0x7fe9fc06d770 msgr2=0x7fe9fc108da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 shutdown_connections 2023-10-08T09:19:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:05.704+0000 7fea010f0700 1 -- 172.21.15.49:0/444871254 wait complete. 2023-10-08T09:19:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:05 smithi049 ceph-mon[99485]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-10-08T09:19:06.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:05 smithi184 ceph-mon[103675]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-10-08T09:19:06.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:05 smithi132 ceph-mon[103389]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 1.58k op/s 2023-10-08T09:19:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:07 smithi049 ceph-mon[99485]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2023-10-08T09:19:08.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:07 smithi184 ceph-mon[103675]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2023-10-08T09:19:08.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:07 smithi132 ceph-mon[103389]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2023-10-08T09:19:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 225.0 s, 743.0 tps, lat 123.873 ms stddev 144.617 2023-10-08T09:19:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:09 smithi049 ceph-mon[99485]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2023-10-08T09:19:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:09 smithi184 ceph-mon[103675]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2023-10-08T09:19:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:09 smithi132 ceph-mon[103389]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2023-10-08T09:19:12.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:11 smithi184 ceph-mon[103675]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.54k op/s 2023-10-08T09:19:12.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:11 smithi132 ceph-mon[103389]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.54k op/s 2023-10-08T09:19:12.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:11 smithi049 ceph-mon[99485]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 1.54k op/s 2023-10-08T09:19:12.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:19:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.707+0000 7f661ed32700 1 -- 172.21.15.49:0/3423086357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66181008a0 msgr2=0x7f6618100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.708 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.707+0000 7f661ed32700 1 --2- 172.21.15.49:0/3423086357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66181008a0 0x7f6618100d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f661000b0a0 tx=0x7f661000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 -- 172.21.15.49:0/3423086357 shutdown_connections 2023-10-08T09:19:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 --2- 172.21.15.49:0/3423086357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6618101240 0x7f6618107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 --2- 172.21.15.49:0/3423086357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66180ff6b0 0x7f66180ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 --2- 172.21.15.49:0/3423086357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66181008a0 0x7f6618100d00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 -- 172.21.15.49:0/3423086357 >> 172.21.15.49:0/3423086357 conn(0x7f66180fac40 msgr2=0x7f66180fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:12.709 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 -- 172.21.15.49:0/3423086357 shutdown_connections 2023-10-08T09:19:12.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 -- 172.21.15.49:0/3423086357 wait complete. 2023-10-08T09:19:12.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.708+0000 7f661ed32700 1 Processor -- start 2023-10-08T09:19:12.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.709+0000 7f661ed32700 1 -- start start 2023-10-08T09:19:12.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.709+0000 7f661ed32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66180ff6b0 0x7f66181968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.709+0000 7f661ed32700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66181008a0 0x7f6618196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.709+0000 7f661ed32700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6618101240 0x7f661819de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.709+0000 7f661ed32700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661819e3e0 con 0x7f6618101240 2023-10-08T09:19:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.709+0000 7f661ed32700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661819e520 con 0x7f66180ff6b0 2023-10-08T09:19:12.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.709+0000 7f661ed32700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661819e6b0 con 0x7f66181008a0 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66180ff6b0 0x7f66181968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66180ff6b0 0x7f66181968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41100/0 (socket says 172.21.15.49:41100) 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 -- 172.21.15.49:0/3674575370 learned_addr learned my addr 172.21.15.49:0/3674575370 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661d2cf700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6618101240 0x7f661819de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 -- 172.21.15.49:0/3674575370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66181008a0 msgr2=0x7f6618196df0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66181008a0 0x7f6618196df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 -- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6618101240 msgr2=0x7f661819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6618101240 0x7f661819de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.712 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.710+0000 7f661cace700 1 -- 172.21.15.49:0/3674575370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610009d00 con 0x7f66180ff6b0 2023-10-08T09:19:12.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.711+0000 7f661cace700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66180ff6b0 0x7f66181968b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f660800b550 tx=0x7f660800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:12.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.711+0000 7f6615ffb700 1 -- 172.21.15.49:0/3674575370 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608011830 con 0x7f66180ff6b0 2023-10-08T09:19:12.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.711+0000 7f6615ffb700 1 -- 172.21.15.49:0/3674575370 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6608011e50 con 0x7f66180ff6b0 2023-10-08T09:19:12.713 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.711+0000 7f6615ffb700 1 -- 172.21.15.49:0/3674575370 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800f2e0 con 0x7f66180ff6b0 2023-10-08T09:19:12.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.711+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661819e990 con 0x7f66180ff6b0 2023-10-08T09:19:12.714 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.711+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661819ee50 con 0x7f66180ff6b0 2023-10-08T09:19:12.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 Processor -- start 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f6615ffb700 1 -- 172.21.15.49:0/3674575370 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6608010580 con 0x7f66180ff6b0 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 -- start start 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd781480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd782270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd782c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd6e9460 con 0x5555dd781060 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd6dac20 con 0x5555dd7819c0 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f662af1bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd6d8540 con 0x5555dd782800 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f661d2cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd781480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f661cace700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd782270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f661d2cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd781480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51488/0 (socket says 172.21.15.49:51488) 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f661cace700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd782270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41104/0 (socket says 172.21.15.49:41104) 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.716+0000 7f661d2cf700 1 -- 172.21.15.49:0/830958708 learned_addr learned my addr 172.21.15.49:0/830958708 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f6617fff700 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd782c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f661d2cf700 1 -- 172.21.15.49:0/830958708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 msgr2=0x5555dd782c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f661d2cf700 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd782c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f661d2cf700 1 -- 172.21.15.49:0/830958708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 msgr2=0x5555dd782270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f6615ffb700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66000a1500 0x7f66000a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f661d2cf700 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd782270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f661d2cf700 1 -- 172.21.15.49:0/830958708 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555dd4fede0 con 0x5555dd781060 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f6615ffb700 1 -- 172.21.15.49:0/3674575370 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f66080cca60 con 0x7f66180ff6b0 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f661d2cf700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66000a1500 0x7f66000a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f661d2cf700 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd781480 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f660c009a50 tx=0x7f660c009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae4021372f8f15f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:12.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.717+0000 7f65feffd700 1 -- 172.21.15.49:0/830958708 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00ecf0 con 0x5555dd781060 2023-10-08T09:19:12.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f65feffd700 1 -- 172.21.15.49:0/830958708 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660c01fd90 con 0x5555dd781060 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f65feffd700 1 -- 172.21.15.49:0/830958708 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c013cb0 con 0x5555dd781060 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f662af1bb80 1 -- 172.21.15.49:0/830958708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 msgr2=0x5555dd781480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f661d2cf700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66000a1500 0x7f66000a39c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f660c01e000 tx=0x7f660c01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f662af1bb80 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd781480 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f660c009a50 tx=0x7f660c009d60 comp rx=0 tx=0).stop 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f662af1bb80 1 -- 172.21.15.49:0/830958708 shutdown_connections 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f662af1bb80 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd782c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f662af1bb80 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd781480 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f662af1bb80 1 --2- 172.21.15.49:0/830958708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd782270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.718+0000 7f662af1bb80 1 -- 172.21.15.49:0/830958708 >> 172.21.15.49:0/830958708 conn(0x5555dd6f5260 msgr2=0x5555dd77be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:12.722 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 -- 172.21.15.49:0/830958708 shutdown_connections 2023-10-08T09:19:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 -- 172.21.15.49:0/830958708 wait complete. 2023-10-08T09:19:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 -- start start 2023-10-08T09:19:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 Processor -- start 2023-10-08T09:19:12.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd78ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd796790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd6d3e30 con 0x5555dd781060 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd670710 con 0x5555dd7819c0 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.719+0000 7f662af1bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd6703b0 con 0x5555dd782800 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.720+0000 7f662af1bb80 10 client.0 _ll_register_callbacks cb 0x5555dd4cd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd796790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd796790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38530/0 (socket says 172.21.15.49:38530) 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 -- 172.21.15.49:0/1622373657 learned_addr learned my addr 172.21.15.49:0/1622373657 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:12.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661cace700 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd78ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 -- 172.21.15.49:0/1622373657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 msgr2=0x5555dd78b340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:19:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78b340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 -- 172.21.15.49:0/1622373657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 msgr2=0x5555dd78ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd78ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.722+0000 7f661d2cf700 1 -- 172.21.15.49:0/1622373657 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c009680 con 0x5555dd782800 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f661d2cf700 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd796790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f660c00e4b0 tx=0x7f660c0135f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d872133bf0d4b1da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f65f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f65f7fff700 1 -- 172.21.15.49:0/1622373657 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c006da0 con 0x5555dd782800 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f65f7fff700 1 -- 172.21.15.49:0/1622373657 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f660c018040 con 0x5555dd782800 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555dd69beb0 con 0x5555dd782800 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f65f7fff700 1 -- 172.21.15.49:0/1622373657 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660c013c70 con 0x5555dd782800 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f662af1bb80 10 client.25921 fetch_fsmap learned FSMap version 38 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5555dd4e7c60 con 0x5555dd782800 2023-10-08T09:19:12.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.723+0000 7f65f7fff700 1 -- 172.21.15.49:0/1622373657 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555dd69beb0 con 0x5555dd782800 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f65f7fff700 1 -- 172.21.15.49:0/1622373657 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f660c030020 con 0x5555dd782800 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f662af1bb80 10 client.25921 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f662af1bb80 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f662af1bb80 20 client.25921 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd79f530 0x5555dd7a1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f662af1bb80 4 client.25921 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555dd6cce10 con 0x5555dd79f530 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.724+0000 7f661cace700 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd79f530 0x5555dd7a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.725+0000 7f661cace700 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd79f530 0x5555dd7a1920 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d103b26e4f638654 server_cookie=c07413fd87cbd5ad in_seq=0 out_seq=0 2023-10-08T09:19:12.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.725+0000 7f65f7fff700 10 client.25921 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.725+0000 7f65f7fff700 1 -- 172.21.15.49:0/1622373657 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5555dd6cce10 con 0x5555dd79f530 2023-10-08T09:19:12.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.725+0000 7f65f7fff700 10 client.25921 handle_command_reply: tid=0 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 client.25921 shutdown 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 msgr2=0x5555dd796790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd796790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f660c00e4b0 tx=0x7f660c0135f0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 shutdown_connections 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd79f530 0x5555dd7a1920 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd796790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd78ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f65f7fff700 0 client.25921 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1622373657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 >> 172.21.15.49:0/1622373657 conn(0x5555dd6f5260 msgr2=0x5555dd77b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:12.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 shutdown_connections 2023-10-08T09:19:12.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 1 -- 172.21.15.49:0/1622373657 wait complete. 2023-10-08T09:19:12.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.726+0000 7f662af1bb80 20 client.25921 trim_cache size 0 max 16384 2023-10-08T09:19:12.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.752+0000 7f662af1bb80 1 Processor -- start 2023-10-08T09:19:12.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.752+0000 7f662af1bb80 1 -- start start 2023-10-08T09:19:12.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.752+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd7aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.752+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd78b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f662af1bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd554920 con 0x5555dd781060 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f662af1bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd782100 con 0x5555dd7819c0 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f662af1bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd789930 con 0x5555dd782800 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f661d2cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd78b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f6617fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f661d2cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd78b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38538/0 (socket says 172.21.15.49:38538) 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f661d2cf700 1 -- 172.21.15.49:0/2710481066 learned_addr learned my addr 172.21.15.49:0/2710481066 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:12.755 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f661cace700 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd7aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f6617fff700 1 -- 172.21.15.49:0/2710481066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 msgr2=0x5555dd78b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f6617fff700 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd78b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f6617fff700 1 -- 172.21.15.49:0/2710481066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 msgr2=0x5555dd7aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f6617fff700 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd7aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f6617fff700 1 -- 172.21.15.49:0/2710481066 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555dd525250 con 0x5555dd7819c0 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.753+0000 7f661d2cf700 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd78b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.754+0000 7f6617fff700 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78adc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6610006130 tx=0x7f66100091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d8103674c40125 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.754+0000 7f65ff7fe700 1 -- 172.21.15.49:0/2710481066 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661000e040 con 0x5555dd7819c0 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.754+0000 7f65ff7fe700 1 -- 172.21.15.49:0/2710481066 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66100074a0 con 0x5555dd7819c0 2023-10-08T09:19:12.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.754+0000 7f65ff7fe700 1 -- 172.21.15.49:0/2710481066 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66100053c0 con 0x5555dd7819c0 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 -- 172.21.15.49:0/2710481066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 msgr2=0x5555dd78adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78adc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6610006130 tx=0x7f66100091e0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 -- 172.21.15.49:0/2710481066 shutdown_connections 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd78b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd7aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 --2- 172.21.15.49:0/2710481066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd78adc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 -- 172.21.15.49:0/2710481066 >> 172.21.15.49:0/2710481066 conn(0x5555dd6f5260 msgr2=0x5555dd796560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 -- 172.21.15.49:0/2710481066 shutdown_connections 2023-10-08T09:19:12.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 -- 172.21.15.49:0/2710481066 wait complete. 2023-10-08T09:19:12.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 -- start start 2023-10-08T09:19:12.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.755+0000 7f662af1bb80 1 Processor -- start 2023-10-08T09:19:12.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd7eb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd7ebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f662af1bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd7f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f662af1bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd7f2e90 con 0x5555dd781060 2023-10-08T09:19:12.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f662af1bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd7f3000 con 0x5555dd7819c0 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f662af1bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555dd7f31c0 con 0x5555dd782800 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f662af1bb80 10 client.0 _ll_register_callbacks cb 0x7f66080d03d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f6617fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd7ebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f6617fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd7ebea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41124/0 (socket says 172.21.15.49:41124) 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.756+0000 7f6617fff700 1 -- 172.21.15.49:0/1432922375 learned_addr learned my addr 172.21.15.49:0/1432922375 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.757+0000 7f661d2cf700 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd7f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.757+0000 7f661d2cf700 1 -- 172.21.15.49:0/1432922375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 msgr2=0x5555dd7ebea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.757+0000 7f661d2cf700 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd7ebea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.757+0000 7f661d2cf700 1 -- 172.21.15.49:0/1432922375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 msgr2=0x5555dd7eb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.757+0000 7f661d2cf700 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd7eb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.757+0000 7f661d2cf700 1 -- 172.21.15.49:0/1432922375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555dd4e7c60 con 0x5555dd782800 2023-10-08T09:19:12.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.757+0000 7f661d2cf700 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd7f2950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f660c0137e0 tx=0x7f660c009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871ed66b56a51212 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:12.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.758+0000 7f65f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:19:12.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.758+0000 7f65f7fff700 1 -- 172.21.15.49:0/1432922375 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c006da0 con 0x5555dd782800 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.758+0000 7f65f7fff700 1 -- 172.21.15.49:0/1432922375 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f660c013c40 con 0x5555dd782800 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.758+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f660c01fd90 con 0x5555dd782800 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.758+0000 7f65f7fff700 1 -- 172.21.15.49:0/1432922375 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f660c013880 con 0x5555dd782800 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.758+0000 7f662af1bb80 10 client.25927 fetch_fsmap learned FSMap version 38 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.758+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f6618068eb0 con 0x5555dd782800 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.759+0000 7f65f7fff700 1 -- 172.21.15.49:0/1432922375 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660c01fd90 con 0x5555dd782800 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.759+0000 7f65f7fff700 1 -- 172.21.15.49:0/1432922375 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 38) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f660c03f830 con 0x5555dd782800 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.760+0000 7f662af1bb80 10 client.25927 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.760+0000 7f662af1bb80 10 client.25927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.760+0000 7f662af1bb80 20 client.25927 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:12.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.760+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd7f48c0 0x5555dd7f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.760+0000 7f662af1bb80 4 client.25927 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:19:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.760+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5555dd6ccb30 con 0x5555dd7f48c0 2023-10-08T09:19:12.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.762+0000 7f661cace700 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd7f48c0 0x5555dd7f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:12.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.762+0000 7f65f7fff700 10 client.25927 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:12.764 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.762+0000 7f661cace700 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd7f48c0 0x5555dd7f6ce0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f52c5af8917adaa server_cookie=5a42ddbd4ecd7476 in_seq=0 out_seq=0 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f65f7fff700 1 -- 172.21.15.49:0/1432922375 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5555dd6ccb30 con 0x5555dd7f48c0 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f65f7fff700 10 client.25927 handle_command_reply: tid=0 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 client.25927 shutdown 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 msgr2=0x5555dd7f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd7f2950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f660c0137e0 tx=0x7f660c009590 comp rx=0 tx=0).stop 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 shutdown_connections 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5555dd7f48c0 0x5555dd7f6ce0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5555dd782800 0x5555dd7f2950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5555dd781060 0x5555dd7eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 --2- 172.21.15.49:0/1432922375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5555dd7819c0 0x5555dd7ebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 >> 172.21.15.49:0/1432922375 conn(0x5555dd6f5260 msgr2=0x5555dd78ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f65f7fff700 0 client.25927 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 shutdown_connections 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.763+0000 7f662af1bb80 1 -- 172.21.15.49:0/1432922375 wait complete. 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f662af1bb80 20 client.25927 trim_cache size 0 max 16384 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:19:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:19:12.767 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:19:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66000a1500 msgr2=0x7f66000a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66000a1500 0x7f66000a39c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f660c01e000 tx=0x7f660c01f000 comp rx=0 tx=0).stop 2023-10-08T09:19:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66180ff6b0 msgr2=0x7f66181968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66180ff6b0 0x7f66181968b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f660800b550 tx=0x7f660800b860 comp rx=0 tx=0).stop 2023-10-08T09:19:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 shutdown_connections 2023-10-08T09:19:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66000a1500 0x7f66000a39c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6618101240 0x7f661819de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66180ff6b0 0x7f66181968b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 --2- 172.21.15.49:0/3674575370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f66181008a0 0x7f6618196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:12.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 >> 172.21.15.49:0/3674575370 conn(0x7f66180fac40 msgr2=0x7f6618103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:12.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.764+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 shutdown_connections 2023-10-08T09:19:12.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:12.765+0000 7f661ed32700 1 -- 172.21.15.49:0/3674575370 wait complete. 2023-10-08T09:19:12.785 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:19:13.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:12 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:13.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:12 smithi184 ceph-mon[103675]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.33k op/s 2023-10-08T09:19:13.296 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:12 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:13.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:12 smithi132 ceph-mon[103389]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.33k op/s 2023-10-08T09:19:13.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:12 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:13.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:12 smithi049 ceph-mon[99485]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 21 MiB/s wr, 1.33k op/s 2023-10-08T09:19:14.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 230.0 s, 772.0 tps, lat 136.857 ms stddev 212.756 2023-10-08T09:19:15.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:15 smithi049 ceph-mon[99485]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 783 op/s 2023-10-08T09:19:15.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:15 smithi184 ceph-mon[103675]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 783 op/s 2023-10-08T09:19:15.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:15 smithi132 ceph-mon[103389]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 783 op/s 2023-10-08T09:19:17.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:17 smithi049 ceph-mon[99485]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 639 op/s 2023-10-08T09:19:17.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:17 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:17 smithi184 ceph-mon[103675]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 639 op/s 2023-10-08T09:19:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:17 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:18.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:17 smithi132 ceph-mon[103389]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 639 op/s 2023-10-08T09:19:18.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:17 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.901+0000 7f63a3da6700 1 -- 172.21.15.49:0/3546381836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f639c1000e0 msgr2=0x7f639c104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.901+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3546381836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f639c1000e0 0x7f639c104980 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f639400b0a0 tx=0x7f639400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.902+0000 7f63a3da6700 1 -- 172.21.15.49:0/3546381836 shutdown_connections 2023-10-08T09:19:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.902+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3546381836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c0fe5f0 0x7f639c0fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.902+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3546381836 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f639c1000e0 0x7f639c104980 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.902+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3546381836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f639c0ff720 0x7f639c0ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.902+0000 7f63a3da6700 1 -- 172.21.15.49:0/3546381836 >> 172.21.15.49:0/3546381836 conn(0x7f639c0f9ef0 msgr2=0x7f639c0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.910+0000 7f63a3da6700 1 -- 172.21.15.49:0/3546381836 shutdown_connections 2023-10-08T09:19:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.911+0000 7f63a3da6700 1 -- 172.21.15.49:0/3546381836 wait complete. 2023-10-08T09:19:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.912+0000 7f63a3da6700 1 Processor -- start 2023-10-08T09:19:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.916+0000 7f63a3da6700 1 -- start start 2023-10-08T09:19:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.916+0000 7f63a3da6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f639c0fe5f0 0x7f639c10b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.916+0000 7f63a3da6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f639c0ff720 0x7f639c10b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.916+0000 7f63a3da6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c1000e0 0x7f639c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.916+0000 7f63a3da6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c10bee0 con 0x7f639c0ff720 2023-10-08T09:19:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.916+0000 7f63a3da6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c06e590 con 0x7f639c0fe5f0 2023-10-08T09:19:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.916+0000 7f63a3da6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c06e750 con 0x7f639c1000e0 2023-10-08T09:19:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.919+0000 7f63a2343700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c1000e0 0x7f639c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c1000e0 0x7f639c06e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38560/0 (socket says 172.21.15.49:38560) 2023-10-08T09:19:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 -- 172.21.15.49:0/3185391817 learned_addr learned my addr 172.21.15.49:0/3185391817 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 -- 172.21.15.49:0/3185391817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f639c0fe5f0 msgr2=0x7f639c10b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f639c0fe5f0 0x7f639c10b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 -- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f639c0ff720 msgr2=0x7f639c10b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f639c0ff720 0x7f639c10b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 -- 172.21.15.49:0/3185391817 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6394009d00 con 0x7f639c1000e0 2023-10-08T09:19:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f63a2343700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c1000e0 0x7f639c06e050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6394006070 tx=0x7f6394005b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f6392ffd700 1 -- 172.21.15.49:0/3185391817 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639400e040 con 0x7f639c1000e0 2023-10-08T09:19:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f6392ffd700 1 -- 172.21.15.49:0/3185391817 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6394003c00 con 0x7f639c1000e0 2023-10-08T09:19:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.920+0000 7f6392ffd700 1 -- 172.21.15.49:0/3185391817 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394005640 con 0x7f639c1000e0 2023-10-08T09:19:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.921+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c06e970 con 0x7f639c1000e0 2023-10-08T09:19:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.922+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c06eda0 con 0x7f639c1000e0 2023-10-08T09:19:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.926+0000 7f63aff8fb80 1 Processor -- start 2023-10-08T09:19:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.932+0000 7f63aff8fb80 1 -- start start 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.933+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704ce40 0x56252704d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.933+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704d7a0 0x56252704dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.933+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252704e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.933+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562526fb5400 con 0x56252704ce40 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.933+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562526fa6bc0 con 0x56252704d7a0 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.933+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562526fa44e0 con 0x56252704e2b0 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.934+0000 7f6392ffd700 1 -- 172.21.15.49:0/3185391817 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f63940057a0 con 0x7f639c1000e0 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.935+0000 7f63a1341700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252704e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.935+0000 7f63a1341700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704ce40 0x56252704d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.935+0000 7f63a1341700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704ce40 0x56252704d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51528/0 (socket says 172.21.15.49:51528) 2023-10-08T09:19:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.935+0000 7f63a1341700 1 -- 172.21.15.49:0/3480771032 learned_addr learned my addr 172.21.15.49:0/3480771032 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.935+0000 7f6392ffd700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f63880a1550 0x7f63880a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.935+0000 7f6392ffd700 1 -- 172.21.15.49:0/3185391817 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f63940ce6c0 con 0x7f639c1000e0 2023-10-08T09:19:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.935+0000 7f63a1b42700 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704d7a0 0x56252704dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.937+0000 7f63a2343700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f63880a1550 0x7f63880a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.938+0000 7f63a1b42700 1 -- 172.21.15.49:0/3480771032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 msgr2=0x56252704e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.938+0000 7f63a1b42700 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252704e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.938+0000 7f63a1b42700 1 -- 172.21.15.49:0/3480771032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704ce40 msgr2=0x56252704d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.938+0000 7f63a1b42700 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704ce40 0x56252704d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.938+0000 7f63a1b42700 1 -- 172.21.15.49:0/3480771032 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562526dcad80 con 0x56252704d7a0 2023-10-08T09:19:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.944+0000 7f63a2343700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f63880a1550 0x7f63880a3a10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6394005110 tx=0x7f639400b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.944+0000 7f63a1b42700 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704d7a0 0x56252704dcd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097f0 tx=0x7f638c009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7652ccd29550a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.950+0000 7f6387fff700 1 -- 172.21.15.49:0/3480771032 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c011830 con 0x56252704d7a0 2023-10-08T09:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.950+0000 7f6387fff700 1 -- 172.21.15.49:0/3480771032 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f638c011e50 con 0x56252704d7a0 2023-10-08T09:19:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.950+0000 7f6387fff700 1 -- 172.21.15.49:0/3480771032 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c010e50 con 0x56252704d7a0 2023-10-08T09:19:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.951+0000 7f63aff8fb80 1 -- 172.21.15.49:0/3480771032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704d7a0 msgr2=0x56252704dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.951+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704d7a0 0x56252704dcd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097f0 tx=0x7f638c009b00 comp rx=0 tx=0).stop 2023-10-08T09:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.951+0000 7f63aff8fb80 1 -- 172.21.15.49:0/3480771032 shutdown_connections 2023-10-08T09:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.951+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252704e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.951+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704ce40 0x56252704d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.951+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/3480771032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704d7a0 0x56252704dcd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.951+0000 7f63aff8fb80 1 -- 172.21.15.49:0/3480771032 >> 172.21.15.49:0/3480771032 conn(0x562526fc1200 msgr2=0x5625270478a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.954+0000 7f63aff8fb80 1 -- 172.21.15.49:0/3480771032 shutdown_connections 2023-10-08T09:19:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.954+0000 7f63aff8fb80 1 -- 172.21.15.49:0/3480771032 wait complete. 2023-10-08T09:19:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.954+0000 7f63aff8fb80 1 -- start start 2023-10-08T09:19:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.954+0000 7f63aff8fb80 1 Processor -- start 2023-10-08T09:19:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.955+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x5625270568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.955+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x562527056df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.955+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.955+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562526f9fdd0 con 0x56252704d7a0 2023-10-08T09:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.955+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562526f3c6b0 con 0x56252704ce40 2023-10-08T09:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.955+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562526f3c350 con 0x56252704e2b0 2023-10-08T09:19:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.955+0000 7f63aff8fb80 10 client.0 _ll_register_callbacks cb 0x562526d997a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38572/0 (socket says 172.21.15.49:38572) 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 -- 172.21.15.49:0/1835416369 learned_addr learned my addr 172.21.15.49:0/1835416369 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x5625270568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 -- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 msgr2=0x5625270568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x5625270568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 -- 172.21.15.49:0/1835416369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 msgr2=0x562527056df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x562527056df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.956+0000 7f63a1341700 1 -- 172.21.15.49:0/1835416369 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c009420 con 0x56252704e2b0 2023-10-08T09:19:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.958+0000 7f63a1341700 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705de30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6398012fb0 tx=0x7f639800cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542a1b68e0340448 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.960+0000 7f63857fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:19:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.966+0000 7f63857fa700 1 -- 172.21.15.49:0/1835416369 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398010c80 con 0x56252704e2b0 2023-10-08T09:19:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.966+0000 7f63857fa700 1 -- 172.21.15.49:0/1835416369 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6398010de0 con 0x56252704e2b0 2023-10-08T09:19:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.969+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562526f67e50 con 0x56252704e2b0 2023-10-08T09:19:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.972+0000 7f63857fa700 1 -- 172.21.15.49:0/1835416369 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6398012020 con 0x56252704e2b0 2023-10-08T09:19:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 10 client.25936 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562526db3c00 con 0x56252704e2b0 2023-10-08T09:19:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63857fa700 1 -- 172.21.15.49:0/1835416369 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562526f67e50 con 0x56252704e2b0 2023-10-08T09:19:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63857fa700 1 -- 172.21.15.49:0/1835416369 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f639801e070 con 0x56252704e2b0 2023-10-08T09:19:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 10 client.25936 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 20 client.25936 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562527062780 0x562527064ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 4 client.25936 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:19:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.976+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562526f98db0 con 0x562527062780 2023-10-08T09:19:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.980+0000 7f63a1b42700 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562527062780 0x562527064ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.980+0000 7f63a1b42700 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562527062780 0x562527064ba0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c9222f30a4a1ba server_cookie=7c2cd1eed467193 in_seq=0 out_seq=0 2023-10-08T09:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.980+0000 7f63857fa700 10 client.25936 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.981+0000 7f63857fa700 1 -- 172.21.15.49:0/1835416369 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562526f98db0 con 0x562527062780 2023-10-08T09:19:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.981+0000 7f63857fa700 10 client.25936 handle_command_reply: tid=0 2023-10-08T09:19:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.983+0000 7f63aff8fb80 1 client.25936 shutdown 2023-10-08T09:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.989+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 msgr2=0x56252705de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.989+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705de30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6398012fb0 tx=0x7f639800cb70 comp rx=0 tx=0).stop 2023-10-08T09:19:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.990+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 shutdown_connections 2023-10-08T09:19:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.990+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562527062780 0x562527064ba0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.990+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705de30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.990+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x5625270568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.990+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1835416369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x562527056df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.990+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 >> 172.21.15.49:0/1835416369 conn(0x562526fc1200 msgr2=0x562527047010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.994+0000 7f63857fa700 0 client.25936 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.994+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 shutdown_connections 2023-10-08T09:19:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:18.995+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1835416369 wait complete. 2023-10-08T09:19:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.001+0000 7f63aff8fb80 20 client.25936 trim_cache size 0 max 16384 2023-10-08T09:19:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.028+0000 7f63aff8fb80 1 Processor -- start 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63aff8fb80 1 -- start start 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x562527055dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x5625270547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562526e208c0 con 0x56252704d7a0 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252704e130 con 0x56252704ce40 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252705efb0 con 0x56252704e2b0 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51544/0 (socket says 172.21.15.49:51544) 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 -- 172.21.15.49:0/2800812118 learned_addr learned my addr 172.21.15.49:0/2800812118 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1341700 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x562527055dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1341700 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x5625270547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 -- 172.21.15.49:0/2800812118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 msgr2=0x5625270547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x5625270547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 -- 172.21.15.49:0/2800812118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 msgr2=0x562527055dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x562527055dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 -- 172.21.15.49:0/2800812118 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562526df11f0 con 0x56252704d7a0 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1341700 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x562527055dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.029+0000 7f63a1b42700 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705ddf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097f0 tx=0x7f638c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec6246ce9ec087 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.030+0000 7f6390ff9700 1 -- 172.21.15.49:0/2800812118 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c0225b0 con 0x56252704d7a0 2023-10-08T09:19:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.030+0000 7f6390ff9700 1 -- 172.21.15.49:0/2800812118 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f638c026070 con 0x56252704d7a0 2023-10-08T09:19:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.030+0000 7f6390ff9700 1 -- 172.21.15.49:0/2800812118 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c022ae0 con 0x56252704d7a0 2023-10-08T09:19:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 -- 172.21.15.49:0/2800812118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 msgr2=0x56252705ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705ddf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f638c0097f0 tx=0x7f638c009190 comp rx=0 tx=0).stop 2023-10-08T09:19:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 -- 172.21.15.49:0/2800812118 shutdown_connections 2023-10-08T09:19:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x5625270547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x562527055dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/2800812118 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705ddf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 -- 172.21.15.49:0/2800812118 >> 172.21.15.49:0/2800812118 conn(0x562526fc1200 msgr2=0x5625270694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 -- 172.21.15.49:0/2800812118 shutdown_connections 2023-10-08T09:19:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 -- 172.21.15.49:0/2800812118 wait complete. 2023-10-08T09:19:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 -- start start 2023-10-08T09:19:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.031+0000 7f63aff8fb80 1 Processor -- start 2023-10-08T09:19:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x56252705bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63aff8fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252705d610 con 0x56252704d7a0 2023-10-08T09:19:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252705d780 con 0x56252704ce40 2023-10-08T09:19:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63aff8fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56252705d940 con 0x56252704e2b0 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63aff8fb80 10 client.0 _ll_register_callbacks cb 0x7f6398012cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63a1341700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63a1b42700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63a1341700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38586/0 (socket says 172.21.15.49:38586) 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63a1341700 1 -- 172.21.15.49:0/1159439678 learned_addr learned my addr 172.21.15.49:0/1159439678 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.032+0000 7f63a1341700 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x56252705bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.033+0000 7f63a1341700 1 -- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 msgr2=0x56252705bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.033+0000 7f63a1341700 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x56252705bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.033+0000 7f63a1341700 1 -- 172.21.15.49:0/1159439678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 msgr2=0x56252705c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.033+0000 7f63a1341700 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.033+0000 7f63a1341700 1 -- 172.21.15.49:0/1159439678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562526db3c00 con 0x56252704e2b0 2023-10-08T09:19:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.034+0000 7f63a1341700 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705d0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6398001eb0 tx=0x7f6398012660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4582abacf5a074e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.034+0000 7f63857fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:19:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.034+0000 7f63857fa700 1 -- 172.21.15.49:0/1159439678 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800c830 con 0x56252704e2b0 2023-10-08T09:19:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.034+0000 7f63857fa700 1 -- 172.21.15.49:0/1159439678 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6398000e40 con 0x56252704e2b0 2023-10-08T09:19:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.034+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f638c011e50 con 0x56252704e2b0 2023-10-08T09:19:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63857fa700 1 -- 172.21.15.49:0/1159439678 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6398020890 con 0x56252704e2b0 2023-10-08T09:19:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 10 client.25942 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562526dcad80 con 0x56252704e2b0 2023-10-08T09:19:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63857fa700 1 -- 172.21.15.49:0/1159439678 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f638c011e50 con 0x56252704e2b0 2023-10-08T09:19:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63857fa700 1 -- 172.21.15.49:0/1159439678 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6398010760 con 0x56252704e2b0 2023-10-08T09:19:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 10 client.25942 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 10 client.25942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 20 client.25942 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5625270b7620 0x5625270b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 4 client.25942 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:19:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562526f98ad0 con 0x5625270b7620 2023-10-08T09:19:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.035+0000 7f63a1b42700 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5625270b7620 0x5625270b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.037+0000 7f63a1b42700 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5625270b7620 0x5625270b9a40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6ab1b063c4c99e2 server_cookie=ad52c681f584a68e in_seq=0 out_seq=0 2023-10-08T09:19:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.037+0000 7f63857fa700 10 client.25942 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63857fa700 1 -- 172.21.15.49:0/1159439678 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562526f98ad0 con 0x5625270b7620 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63857fa700 10 client.25942 handle_command_reply: tid=0 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 client.25942 shutdown 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 msgr2=0x56252705d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705d0d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6398001eb0 tx=0x7f6398012660 comp rx=0 tx=0).stop 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 shutdown_connections 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5625270b7620 0x5625270b9a40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56252704e2b0 0x56252705d0d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56252704ce40 0x56252705bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 --2- 172.21.15.49:0/1159439678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56252704d7a0 0x56252705c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 >> 172.21.15.49:0/1159439678 conn(0x562526fc1200 msgr2=0x562527051200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63857fa700 0 client.25942 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 shutdown_connections 2023-10-08T09:19:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 1 -- 172.21.15.49:0/1159439678 wait complete. 2023-10-08T09:19:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.046+0000 7f63aff8fb80 20 client.25942 trim_cache size 0 max 16384 2023-10-08T09:19:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:19:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:19:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:19:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:19:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f63880a1550 msgr2=0x7f63880a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f63880a1550 0x7f63880a3a10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6394005110 tx=0x7f639400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c1000e0 msgr2=0x7f639c06e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c1000e0 0x7f639c06e050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6394006070 tx=0x7f6394005b10 comp rx=0 tx=0).stop 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 shutdown_connections 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f63880a1550 0x7f63880a3a10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f639c1000e0 0x7f639c06e050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f639c0fe5f0 0x7f639c10b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 --2- 172.21.15.49:0/3185391817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f639c0ff720 0x7f639c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.047+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 >> 172.21.15.49:0/3185391817 conn(0x7f639c0f9ef0 msgr2=0x7f639c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.048+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 shutdown_connections 2023-10-08T09:19:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:19.048+0000 7f63a3da6700 1 -- 172.21.15.49:0/3185391817 wait complete. 2023-10-08T09:19:19.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 235.0 s, 958.8 tps, lat 106.862 ms stddev 134.991 2023-10-08T09:19:19.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:19 smithi049 ceph-mon[99485]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.4 MiB/s wr, 532 op/s 2023-10-08T09:19:19.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:19 smithi184 ceph-mon[103675]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.4 MiB/s wr, 532 op/s 2023-10-08T09:19:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:19 smithi132 ceph-mon[103389]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.4 MiB/s wr, 532 op/s 2023-10-08T09:19:21.886 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:21 smithi049 ceph-mon[99485]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 559 op/s 2023-10-08T09:19:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:21 smithi184 ceph-mon[103675]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 559 op/s 2023-10-08T09:19:22.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:21 smithi132 ceph-mon[103389]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 559 op/s 2023-10-08T09:19:23.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:23 smithi049 ceph-mon[99485]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 615 op/s 2023-10-08T09:19:23.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:23 smithi184 ceph-mon[103675]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 615 op/s 2023-10-08T09:19:24.051 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:23 smithi132 ceph-mon[103389]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 615 op/s 2023-10-08T09:19:24.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 240.0 s, 950.6 tps, lat 103.333 ms stddev 116.487 2023-10-08T09:19:24.881 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:24 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:19:24.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:24 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:19:25.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:24 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:19:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:25 smithi049 ceph-mon[99485]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 583 op/s 2023-10-08T09:19:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:25 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:19:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:25 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:19:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:25 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:19:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:25 smithi184 ceph-mon[103675]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 583 op/s 2023-10-08T09:19:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:25 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:19:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:25 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:19:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:25 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:19:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:25 smithi132 ceph-mon[103389]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 MiB/s wr, 583 op/s 2023-10-08T09:19:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:25 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:19:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:25 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:19:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:25 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:19:27.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:27 smithi049 ceph-mon[99485]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 632 op/s 2023-10-08T09:19:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:27 smithi184 ceph-mon[103675]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 632 op/s 2023-10-08T09:19:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:27 smithi132 ceph-mon[103389]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 10 MiB/s wr, 632 op/s 2023-10-08T09:19:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 245.0 s, 886.3 tps, lat 115.055 ms stddev 168.080 2023-10-08T09:19:29.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:29 smithi049 ceph-mon[99485]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 541 op/s 2023-10-08T09:19:29.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:29 smithi184 ceph-mon[103675]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 541 op/s 2023-10-08T09:19:30.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:29 smithi132 ceph-mon[103389]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.1 MiB/s wr, 541 op/s 2023-10-08T09:19:31.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:31 smithi184 ceph-mon[103675]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 608 op/s 2023-10-08T09:19:32.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:31 smithi132 ceph-mon[103389]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 608 op/s 2023-10-08T09:19:32.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:31 smithi049 ceph-mon[99485]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.2 MiB/s wr, 608 op/s 2023-10-08T09:19:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 -- 172.21.15.49:0/922539644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c100830 msgr2=0x7f591c100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 --2- 172.21.15.49:0/922539644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c100830 0x7f591c100c90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5910009990 tx=0x7f5910009ca0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 -- 172.21.15.49:0/922539644 shutdown_connections 2023-10-08T09:19:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 --2- 172.21.15.49:0/922539644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f591c1011d0 0x7f591c107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 --2- 172.21.15.49:0/922539644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c0ff640 0x7f591c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 --2- 172.21.15.49:0/922539644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c100830 0x7f591c100c90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 -- 172.21.15.49:0/922539644 >> 172.21.15.49:0/922539644 conn(0x7f591c0fabb0 msgr2=0x7f591c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 -- 172.21.15.49:0/922539644 shutdown_connections 2023-10-08T09:19:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.237+0000 7f5920f47700 1 -- 172.21.15.49:0/922539644 wait complete. 2023-10-08T09:19:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.238+0000 7f5920f47700 1 Processor -- start 2023-10-08T09:19:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.238+0000 7f5920f47700 1 -- start start 2023-10-08T09:19:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5920f47700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c0ff640 0x7f591c10aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5920f47700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c100830 0x7f591c109130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5920f47700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f591c1011d0 0x7f591c109670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5920f47700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c109e10 con 0x7f591c100830 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5920f47700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c109f80 con 0x7f591c1011d0 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5920f47700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1a06e0 con 0x7f591c0ff640 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c100830 0x7f591c109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c0ff640 0x7f591c10aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c0ff640 0x7f591c10aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38598/0 (socket says 172.21.15.49:38598) 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c100830 0x7f591c109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51566/0 (socket says 172.21.15.49:51566) 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 -- 172.21.15.49:0/3549949383 learned_addr learned my addr 172.21.15.49:0/3549949383 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:32.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591ad9d700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f591c1011d0 0x7f591c109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 -- 172.21.15.49:0/3549949383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f591c1011d0 msgr2=0x7f591c109670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f591c1011d0 0x7f591c109670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 -- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c100830 msgr2=0x7f591c109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c100830 0x7f591c109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 -- 172.21.15.49:0/3549949383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100095f0 con 0x7f591c0ff640 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f5919d9b700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c100830 0x7f591c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591ad9d700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f591c1011d0 0x7f591c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.239+0000 7f591a59c700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c0ff640 0x7f591c10aaa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f591400efc0 tx=0x7f591400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.240+0000 7f590b7fe700 1 -- 172.21.15.49:0/3549949383 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914018070 con 0x7f591c0ff640 2023-10-08T09:19:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.240+0000 7f590b7fe700 1 -- 172.21.15.49:0/3549949383 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591400f040 con 0x7f591c0ff640 2023-10-08T09:19:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.240+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c1a0910 con 0x7f591c0ff640 2023-10-08T09:19:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.240+0000 7f590b7fe700 1 -- 172.21.15.49:0/3549949383 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914004280 con 0x7f591c0ff640 2023-10-08T09:19:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.240+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c1a0e10 con 0x7f591c0ff640 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.243+0000 7f590b7fe700 1 -- 172.21.15.49:0/3549949383 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5914003a00 con 0x7f591c0ff640 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.244+0000 7f590b7fe700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f59040a1500 0x7f59040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.244+0000 7f590b7fe700 1 -- 172.21.15.49:0/3549949383 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f59140cceb0 con 0x7f591c0ff640 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 Processor -- start 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 -- start start 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f5919d9b700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f59040a1500 0x7f59040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b0ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0f2b0 0x55cc94b0f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94a76400 con 0x55cc94b0de40 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94a67bc0 con 0x55cc94b0e7a0 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f592d130b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94a654e0 con 0x55cc94b0f2b0 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f591a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f591a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b0e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51572/0 (socket says 172.21.15.49:51572) 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f591a59c700 1 -- 172.21.15.49:0/1681412789 learned_addr learned my addr 172.21.15.49:0/1681412789 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b0ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.245+0000 7f591ad9d700 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0f2b0 0x55cc94b0f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f5919d9b700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f59040a1500 0x7f59040a39c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5910006010 tx=0x7f591000bd00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f591a59c700 1 -- 172.21.15.49:0/1681412789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0f2b0 msgr2=0x55cc94b0f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f591a59c700 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0f2b0 0x55cc94b0f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f591a59c700 1 -- 172.21.15.49:0/1681412789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 msgr2=0x55cc94b0ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f591a59c700 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b0ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f591a59c700 1 -- 172.21.15.49:0/1681412789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc9488bd80 con 0x55cc94b0de40 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b0ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f591a59c700 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b0e260 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5914096430 tx=0x7f5914095c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ce8534afdf17fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f5908ff9700 1 -- 172.21.15.49:0/1681412789 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914099890 con 0x55cc94b0de40 2023-10-08T09:19:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f5908ff9700 1 -- 172.21.15.49:0/1681412789 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591409a630 con 0x55cc94b0de40 2023-10-08T09:19:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.246+0000 7f5908ff9700 1 -- 172.21.15.49:0/1681412789 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591409a190 con 0x55cc94b0de40 2023-10-08T09:19:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.247+0000 7f592d130b80 1 -- 172.21.15.49:0/1681412789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 msgr2=0x55cc94b0e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.247+0000 7f592d130b80 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b0e260 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5914096430 tx=0x7f5914095c20 comp rx=0 tx=0).stop 2023-10-08T09:19:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.247+0000 7f592d130b80 1 -- 172.21.15.49:0/1681412789 shutdown_connections 2023-10-08T09:19:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.247+0000 7f592d130b80 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0f2b0 0x55cc94b0f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.247+0000 7f592d130b80 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b0e260 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.247+0000 7f592d130b80 1 --2- 172.21.15.49:0/1681412789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b0ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.247+0000 7f592d130b80 1 -- 172.21.15.49:0/1681412789 >> 172.21.15.49:0/1681412789 conn(0x55cc94a82200 msgr2=0x55cc94b088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.248+0000 7f592d130b80 1 -- 172.21.15.49:0/1681412789 shutdown_connections 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.248+0000 7f592d130b80 1 -- 172.21.15.49:0/1681412789 wait complete. 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 -- start start 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 Processor -- start 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0de40 0x55cc94b178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b17e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94a60dd0 con 0x55cc94b0f2b0 2023-10-08T09:19:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc949fd6b0 con 0x55cc94b0e7a0 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc949fd350 con 0x55cc94b0de40 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f592d130b80 10 client.0 _ll_register_callbacks cb 0x55cc9485a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b17e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f591a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0de40 0x55cc94b178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b17e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41194/0 (socket says 172.21.15.49:41194) 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f591a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0de40 0x55cc94b178e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38612/0 (socket says 172.21.15.49:38612) 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f5919d9b700 1 -- 172.21.15.49:0/1377744960 learned_addr learned my addr 172.21.15.49:0/1377744960 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.249+0000 7f591ad9d700 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5919d9b700 1 -- 172.21.15.49:0/1377744960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0de40 msgr2=0x55cc94b178e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0de40 0x55cc94b178e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5919d9b700 1 -- 172.21.15.49:0/1377744960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 msgr2=0x55cc94b1ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5919d9b700 1 -- 172.21.15.49:0/1377744960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c1083d0 con 0x55cc94b0e7a0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b17e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5910009990 tx=0x7f5910009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98ec0d59ff13526b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5901ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5901ffb700 1 -- 172.21.15.49:0/1377744960 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000f460 con 0x55cc94b0e7a0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc94a28e50 con 0x55cc94b0e7a0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5901ffb700 1 -- 172.21.15.49:0/1377744960 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f591000f5c0 con 0x55cc94b0e7a0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.250+0000 7f5901ffb700 1 -- 172.21.15.49:0/1377744960 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f591000f780 con 0x55cc94b0e7a0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 10 client.25925 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f591c1a0e10 con 0x55cc94b0e7a0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f5901ffb700 1 -- 172.21.15.49:0/1377744960 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc94a28e50 con 0x55cc94b0e7a0 2023-10-08T09:19:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f5901ffb700 1 -- 172.21.15.49:0/1377744960 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f59100189c0 con 0x55cc94b0e7a0 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 10 client.25925 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 20 client.25925 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b23600 0x55cc94b25a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 4 client.25925 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc94a59db0 con 0x55cc94b23600 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.251+0000 7f591a59c700 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b23600 0x55cc94b25a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.252+0000 7f591a59c700 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b23600 0x55cc94b25a20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0cb0a3060fdb15 server_cookie=754d1adcdd0a9b73 in_seq=0 out_seq=0 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.252+0000 7f5901ffb700 10 client.25925 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.253+0000 7f5901ffb700 1 -- 172.21.15.49:0/1377744960 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cc94a59db0 con 0x55cc94b23600 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.253+0000 7f5901ffb700 10 client.25925 handle_command_reply: tid=0 2023-10-08T09:19:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.253+0000 7f592d130b80 1 client.25925 shutdown 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.253+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 msgr2=0x55cc94b17e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.253+0000 7f592d130b80 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b17e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5910009990 tx=0x7f5910009ca0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 shutdown_connections 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f592d130b80 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b23600 0x55cc94b25a20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f592d130b80 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0de40 0x55cc94b178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f592d130b80 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f592d130b80 1 --2- 172.21.15.49:0/1377744960 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0e7a0 0x55cc94b17e20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 >> 172.21.15.49:0/1377744960 conn(0x55cc94a82200 msgr2=0x55cc94b08010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f5901ffb700 0 client.25925 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.254+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 shutdown_connections 2023-10-08T09:19:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.255+0000 7f592d130b80 1 -- 172.21.15.49:0/1377744960 wait complete. 2023-10-08T09:19:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.255+0000 7f592d130b80 20 client.25925 trim_cache size 0 max 16384 2023-10-08T09:19:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.279+0000 7f592d130b80 1 Processor -- start 2023-10-08T09:19:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.279+0000 7f592d130b80 1 -- start start 2023-10-08T09:19:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.279+0000 7f592d130b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0de40 0x55cc94b0e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.279+0000 7f592d130b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f592d130b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f592d130b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc948e18c0 con 0x55cc94b0f2b0 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f592d130b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94a59de0 con 0x55cc94b0de40 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f592d130b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94b05830 con 0x55cc94b0e7a0 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38620/0 (socket says 172.21.15.49:38620) 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 -- 172.21.15.49:0/1868455120 learned_addr learned my addr 172.21.15.49:0/1868455120 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f591a59c700 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0de40 0x55cc94b0e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f591ad9d700 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 -- 172.21.15.49:0/1868455120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0de40 msgr2=0x55cc94b0e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0de40 0x55cc94b0e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 -- 172.21.15.49:0/1868455120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 msgr2=0x55cc94b08870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b08870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 -- 172.21.15.49:0/1868455120 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc94874c00 con 0x55cc94b0e7a0 2023-10-08T09:19:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f5919d9b700 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1ed80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5910000c00 tx=0x7f5910009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a778e3654551232b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.280+0000 7f59097fa700 1 -- 172.21.15.49:0/1868455120 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910004b70 con 0x55cc94b0e7a0 2023-10-08T09:19:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f59097fa700 1 -- 172.21.15.49:0/1868455120 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591001e040 con 0x55cc94b0e7a0 2023-10-08T09:19:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f59097fa700 1 -- 172.21.15.49:0/1868455120 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59100188b0 con 0x55cc94b0e7a0 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 -- 172.21.15.49:0/1868455120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 msgr2=0x55cc94b1ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1ed80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5910000c00 tx=0x7f5910009310 comp rx=0 tx=0).stop 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 -- 172.21.15.49:0/1868455120 shutdown_connections 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0f2b0 0x55cc94b08870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0de40 0x55cc94b0e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 --2- 172.21.15.49:0/1868455120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1ed80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 -- 172.21.15.49:0/1868455120 >> 172.21.15.49:0/1868455120 conn(0x55cc94a82200 msgr2=0x55cc94b2a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 -- 172.21.15.49:0/1868455120 shutdown_connections 2023-10-08T09:19:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.281+0000 7f592d130b80 1 -- 172.21.15.49:0/1868455120 wait complete. 2023-10-08T09:19:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.282+0000 7f592d130b80 1 -- start start 2023-10-08T09:19:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.282+0000 7f592d130b80 1 Processor -- start 2023-10-08T09:19:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.284+0000 7f592d130b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b1c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.284+0000 7f592d130b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f592d130b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f592d130b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94b1dd60 con 0x55cc94b0de40 2023-10-08T09:19:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f592d130b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94b1ded0 con 0x55cc94b0f2b0 2023-10-08T09:19:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f592d130b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc94b1e090 con 0x55cc94b0e7a0 2023-10-08T09:19:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f592d130b80 10 client.0 _ll_register_callbacks cb 0x55cc9485a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b1c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b1c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51592/0 (socket says 172.21.15.49:51592) 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 -- 172.21.15.49:0/855785660 learned_addr learned my addr 172.21.15.49:0/855785660 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 -- 172.21.15.49:0/855785660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 msgr2=0x55cc94b1c900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1c900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 -- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0f2b0 msgr2=0x55cc94b1d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.285+0000 7f591a59c700 1 -- 172.21.15.49:0/855785660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c1a0e10 con 0x55cc94b0de40 2023-10-08T09:19:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.286+0000 7f591ad9d700 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.286+0000 7f591a59c700 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b1c3c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5914099ec0 tx=0x7f591409a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2130c2bd8f6411 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.286+0000 7f5901ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.286+0000 7f5901ffb700 1 -- 172.21.15.49:0/855785660 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59140b8040 con 0x55cc94b0de40 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.286+0000 7f5901ffb700 1 -- 172.21.15.49:0/855785660 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5914078e40 con 0x55cc94b0de40 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.286+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f591409a630 con 0x55cc94b0de40 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f5901ffb700 1 -- 172.21.15.49:0/855785660 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59140b81a0 con 0x55cc94b0de40 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 10 client.16296 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55cc948b21f0 con 0x55cc94b0de40 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f5901ffb700 1 -- 172.21.15.49:0/855785660 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591409a630 con 0x55cc94b0de40 2023-10-08T09:19:32.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f5901ffb700 1 -- 172.21.15.49:0/855785660 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f59140831f0 con 0x55cc94b0de40 2023-10-08T09:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 20 client.16296 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b71590 0x55cc94b739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 4 client.16296 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.287+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc94a59ad0 con 0x55cc94b71590 2023-10-08T09:19:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.289+0000 7f5919d9b700 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b71590 0x55cc94b739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.289+0000 7f5919d9b700 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b71590 0x55cc94b739b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cdd4b8b15a09816 server_cookie=9ad09839fe34191c in_seq=0 out_seq=0 2023-10-08T09:19:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.289+0000 7f5901ffb700 10 client.16296 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f5901ffb700 1 -- 172.21.15.49:0/855785660 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc94a59ad0 con 0x55cc94b71590 2023-10-08T09:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f5901ffb700 10 client.16296 handle_command_reply: tid=0 2023-10-08T09:19:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 client.16296 shutdown 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 msgr2=0x55cc94b1c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b1c3c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5914099ec0 tx=0x7f591409a7a0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 shutdown_connections 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55cc94b71590 0x55cc94b739b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55cc94b0f2b0 0x55cc94b1d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f5901ffb700 0 client.16296 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55cc94b0de40 0x55cc94b1c3c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 --2- 172.21.15.49:0/855785660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55cc94b0e7a0 0x55cc94b1c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.299+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 >> 172.21.15.49:0/855785660 conn(0x55cc94a82200 msgr2=0x55cc94b08850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.300+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 shutdown_connections 2023-10-08T09:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.300+0000 7f592d130b80 1 -- 172.21.15.49:0/855785660 wait complete. 2023-10-08T09:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.300+0000 7f592d130b80 20 client.16296 trim_cache size 0 max 16384 2023-10-08T09:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:19:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:19:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f59040a1500 msgr2=0x7f59040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f59040a1500 0x7f59040a39c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5910006010 tx=0x7f591000bd00 comp rx=0 tx=0).stop 2023-10-08T09:19:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c0ff640 msgr2=0x7f591c10aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c0ff640 0x7f591c10aaa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f591400efc0 tx=0x7f591400c3a0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 shutdown_connections 2023-10-08T09:19:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f59040a1500 0x7f59040a39c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f591c0ff640 0x7f591c10aaa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f591c1011d0 0x7f591c109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 --2- 172.21.15.49:0/3549949383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f591c100830 0x7f591c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 >> 172.21.15.49:0/3549949383 conn(0x7f591c0fabb0 msgr2=0x7f591c103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 shutdown_connections 2023-10-08T09:19:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:32.301+0000 7f5920f47700 1 -- 172.21.15.49:0/3549949383 wait complete. 2023-10-08T09:19:33.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:33 smithi184 ceph-mon[103675]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 687 op/s 2023-10-08T09:19:34.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:33 smithi132 ceph-mon[103389]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 687 op/s 2023-10-08T09:19:34.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:33 smithi049 ceph-mon[99485]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 687 op/s 2023-10-08T09:19:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 250.0 s, 784.0 tps, lat 127.188 ms stddev 149.356 2023-10-08T09:19:35.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:35 smithi184 ceph-mon[103675]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 664 op/s 2023-10-08T09:19:36.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:35 smithi132 ceph-mon[103389]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 664 op/s 2023-10-08T09:19:36.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:35 smithi049 ceph-mon[99485]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 664 op/s 2023-10-08T09:19:37.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:37 smithi184 ceph-mon[103675]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 710 op/s 2023-10-08T09:19:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:37 smithi132 ceph-mon[103389]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 710 op/s 2023-10-08T09:19:38.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:37 smithi049 ceph-mon[99485]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 MiB/s wr, 710 op/s 2023-10-08T09:19:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 255.0 s, 945.8 tps, lat 103.127 ms stddev 114.146 2023-10-08T09:19:40.001 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:39 smithi184 ceph-mon[103675]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 659 op/s 2023-10-08T09:19:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:39 smithi132 ceph-mon[103389]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 659 op/s 2023-10-08T09:19:40.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:39 smithi049 ceph-mon[99485]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 659 op/s 2023-10-08T09:19:42.001 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:41 smithi184 ceph-mon[103675]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.8 MiB/s wr, 687 op/s 2023-10-08T09:19:42.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:41 smithi132 ceph-mon[103389]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.8 MiB/s wr, 687 op/s 2023-10-08T09:19:42.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:41 smithi049 ceph-mon[99485]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.8 MiB/s wr, 687 op/s 2023-10-08T09:19:42.787 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:19:43.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.116+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2174830724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f341077f0 msgr2=0x7f8f34109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.116+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2174830724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f341077f0 0x7f8f34109c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009990 tx=0x7f8f28009ca0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.127+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2174830724 shutdown_connections 2023-10-08T09:19:43.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.127+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2174830724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f341077f0 0x7f8f34109c20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.127+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2174830724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f3410cb30 0x7f8f3410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.127+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2174830724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3410a160 0x7f8f3410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.127+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2174830724 >> 172.21.15.49:0/2174830724 conn(0x7f8f3406d790 msgr2=0x7f8f3406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:43.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.129+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2174830724 shutdown_connections 2023-10-08T09:19:43.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.130+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2174830724 wait complete. 2023-10-08T09:19:43.132 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 Processor -- start 2023-10-08T09:19:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 -- start start 2023-10-08T09:19:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f341077f0 0x7f8f341a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3410a160 0x7f8f341a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3410cb30 0x7f8f341acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f341ad180 con 0x7f8f341077f0 2023-10-08T09:19:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3410ef20 con 0x7f8f3410a160 2023-10-08T09:19:43.133 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.131+0000 7f8f3ad31700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3410f0b0 con 0x7f8f3410cb30 2023-10-08T09:19:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.133+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3410cb30 0x7f8f341acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.133+0000 7f8f33fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3410a160 0x7f8f341a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.133+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3410cb30 0x7f8f341acbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38642/0 (socket says 172.21.15.49:38642) 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.133+0000 7f8f392ce700 1 -- 172.21.15.49:0/2945540461 learned_addr learned my addr 172.21.15.49:0/2945540461 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.133+0000 7f8f33fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3410a160 0x7f8f341a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41220/0 (socket says 172.21.15.49:41220) 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f392ce700 1 -- 172.21.15.49:0/2945540461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3410a160 msgr2=0x7f8f341a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f392ce700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3410a160 0x7f8f341a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f392ce700 1 -- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f341077f0 msgr2=0x7f8f341a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f392ce700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f341077f0 0x7f8f341a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f392ce700 1 -- 172.21.15.49:0/2945540461 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280095f0 con 0x7f8f3410cb30 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f392ce700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3410cb30 0x7f8f341acbb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00b970 tx=0x7f8f2c00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:43.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f31ffb700 1 -- 172.21.15.49:0/2945540461 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00c930 con 0x7f8f3410cb30 2023-10-08T09:19:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f31ffb700 1 -- 172.21.15.49:0/2945540461 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f2c014440 con 0x7f8f3410cb30 2023-10-08T09:19:43.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.134+0000 7f8f31ffb700 1 -- 172.21.15.49:0/2945540461 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0075d0 con 0x7f8f3410cb30 2023-10-08T09:19:43.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.140+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3410f390 con 0x7f8f3410cb30 2023-10-08T09:19:43.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.140+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3410f850 con 0x7f8f3410cb30 2023-10-08T09:19:43.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.147+0000 7f8f31ffb700 1 -- 172.21.15.49:0/2945540461 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8f2c007730 con 0x7f8f3410cb30 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.162+0000 7f8f31ffb700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8f1c0a1580 0x7f8f1c0a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.162+0000 7f8f31ffb700 1 -- 172.21.15.49:0/2945540461 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8f2c006e40 con 0x7f8f3410cb30 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.164+0000 7f8f46f1ab80 1 Processor -- start 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.164+0000 7f8f46f1ab80 1 -- start start 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x55643061b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x55643061c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061c800 0x55643061cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430583460 con 0x55643061b9c0 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430574c20 con 0x55643061b060 2023-10-08T09:19:43.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430572540 con 0x55643061c800 2023-10-08T09:19:43.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f38acd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x55643061b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f38acd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x55643061b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41226/0 (socket says 172.21.15.49:41226) 2023-10-08T09:19:43.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.165+0000 7f8f38acd700 1 -- 172.21.15.49:0/2754605609 learned_addr learned my addr 172.21.15.49:0/2754605609 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:43.169 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.168+0000 7f8f392ce700 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061c800 0x55643061cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.169+0000 7f8f38acd700 1 -- 172.21.15.49:0/2754605609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061c800 msgr2=0x55643061cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:43.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.169+0000 7f8f38acd700 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061c800 0x55643061cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.175 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.170+0000 7f8f38acd700 1 -- 172.21.15.49:0/2754605609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 msgr2=0x55643061c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.170+0000 7f8f38acd700 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x55643061c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.170+0000 7f8f38acd700 1 -- 172.21.15.49:0/2754605609 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556430398de0 con 0x55643061b060 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.170+0000 7f8f38acd700 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x55643061b480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005b40 tx=0x7f8f28005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe0097cc39dd5d4b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.171+0000 7f8f38acd700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8f1c0a1580 0x7f8f1c0a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.172+0000 7f8f38acd700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8f1c0a1580 0x7f8f1c0a3a40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f28025000 tx=0x7f8f28017a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.174+0000 7f8f1affd700 1 -- 172.21.15.49:0/2754605609 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2800b9d0 con 0x55643061b060 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.174+0000 7f8f1affd700 1 -- 172.21.15.49:0/2754605609 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f28005940 con 0x55643061b060 2023-10-08T09:19:43.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.174+0000 7f8f1affd700 1 -- 172.21.15.49:0/2754605609 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280176e0 con 0x55643061b060 2023-10-08T09:19:43.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.175+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/2754605609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 msgr2=0x55643061b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.175+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x55643061b480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f28005b40 tx=0x7f8f28005dc0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.183+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/2754605609 shutdown_connections 2023-10-08T09:19:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.183+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061c800 0x55643061cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.183+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x55643061b480 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.183+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/2754605609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x55643061c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.183+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/2754605609 >> 172.21.15.49:0/2754605609 conn(0x55643058f260 msgr2=0x556430615e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.190+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/2754605609 shutdown_connections 2023-10-08T09:19:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.191+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/2754605609 wait complete. 2023-10-08T09:19:43.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.191+0000 7f8f46f1ab80 1 -- start start 2023-10-08T09:19:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.191+0000 7f8f46f1ab80 1 Processor -- start 2023-10-08T09:19:43.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 0x556430624dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b9c0 0x556430625310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x55643062c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643056de30 con 0x55643061c800 2023-10-08T09:19:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643050a710 con 0x55643061b9c0 2023-10-08T09:19:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643050a3b0 con 0x55643061b060 2023-10-08T09:19:43.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f46f1ab80 10 client.0 _ll_register_callbacks cb 0x556430367800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x55643062c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x55643062c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51620/0 (socket says 172.21.15.49:51620) 2023-10-08T09:19:43.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f392ce700 1 -- 172.21.15.49:0/580663598 learned_addr learned my addr 172.21.15.49:0/580663598 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.192+0000 7f8f38acd700 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 0x556430624dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f33fff700 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b9c0 0x556430625310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f392ce700 1 -- 172.21.15.49:0/580663598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 msgr2=0x556430624dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f392ce700 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 0x556430624dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f392ce700 1 -- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b9c0 msgr2=0x556430625310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f392ce700 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b9c0 0x556430625310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f392ce700 1 -- 172.21.15.49:0/580663598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3410dae0 con 0x55643061c800 2023-10-08T09:19:43.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f392ce700 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x55643062c350 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c09a2e0 tx=0x7f8f2c09ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ceb58806a530e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:43.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.193+0000 7f8f13fff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:19:43.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.200+0000 7f8f13fff700 1 -- 172.21.15.49:0/580663598 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0850d0 con 0x55643061c800 2023-10-08T09:19:43.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.200+0000 7f8f13fff700 1 -- 172.21.15.49:0/580663598 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8f2c0a2970 con 0x55643061c800 2023-10-08T09:19:43.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.200+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556430535eb0 con 0x55643061c800 2023-10-08T09:19:43.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.200+0000 7f8f13fff700 1 -- 172.21.15.49:0/580663598 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f2c09a840 con 0x55643061c800 2023-10-08T09:19:43.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.200+0000 7f8f46f1ab80 10 client.16299 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:43.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.200+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x556430381c60 con 0x55643061c800 2023-10-08T09:19:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f13fff700 1 -- 172.21.15.49:0/580663598 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556430535eb0 con 0x55643061c800 2023-10-08T09:19:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f13fff700 1 -- 172.21.15.49:0/580663598 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8f2c0a24e0 con 0x55643061c800 2023-10-08T09:19:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f46f1ab80 10 client.16299 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f46f1ab80 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f46f1ab80 20 client.16299 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430630c40 0x556430633060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f46f1ab80 4 client.16299 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:19:43.204 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.201+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556430566e10 con 0x556430630c40 2023-10-08T09:19:43.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.205+0000 7f8f38acd700 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430630c40 0x556430633060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.208 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.207+0000 7f8f38acd700 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430630c40 0x556430633060 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8cb6b1b82db99e server_cookie=fa0b77912583d72e in_seq=0 out_seq=0 2023-10-08T09:19:43.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.212+0000 7f8f13fff700 10 client.16299 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:43.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.216+0000 7f8f13fff700 1 -- 172.21.15.49:0/580663598 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556430566e10 con 0x556430630c40 2023-10-08T09:19:43.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.216+0000 7f8f13fff700 10 client.16299 handle_command_reply: tid=0 2023-10-08T09:19:43.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 client.16299 shutdown 2023-10-08T09:19:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 msgr2=0x55643062c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x55643062c350 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c09a2e0 tx=0x7f8f2c09ad90 comp rx=0 tx=0).stop 2023-10-08T09:19:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 shutdown_connections 2023-10-08T09:19:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430630c40 0x556430633060 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 0x556430624dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x55643062c350 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/580663598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b9c0 0x556430625310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 >> 172.21.15.49:0/580663598 conn(0x55643058f260 msgr2=0x5564306154d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:43.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f13fff700 0 client.16299 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:43.225 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.222+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 shutdown_connections 2023-10-08T09:19:43.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.227+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/580663598 wait complete. 2023-10-08T09:19:43.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.238+0000 7f8f46f1ab80 20 client.16299 trim_cache size 0 max 16384 2023-10-08T09:19:43.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.280+0000 7f8f46f1ab80 1 Processor -- start 2023-10-08T09:19:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.283+0000 7f8f46f1ab80 1 -- start start 2023-10-08T09:19:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.283+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x556430623f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.283+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b9c0 0x55643062c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.283+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x5564306244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.283+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564303ee920 con 0x55643061c800 2023-10-08T09:19:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.283+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430566e10 con 0x55643061b060 2023-10-08T09:19:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.283+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430612ae0 con 0x55643061b9c0 2023-10-08T09:19:43.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.284+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x5564306244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.284+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x5564306244d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51626/0 (socket says 172.21.15.49:51626) 2023-10-08T09:19:43.286 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.284+0000 7f8f392ce700 1 -- 172.21.15.49:0/521623196 learned_addr learned my addr 172.21.15.49:0/521623196 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.285+0000 7f8f392ce700 1 -- 172.21.15.49:0/521623196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b9c0 msgr2=0x55643062c310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:19:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.285+0000 7f8f392ce700 1 --2- 172.21.15.49:0/521623196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b9c0 0x55643062c310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.285+0000 7f8f392ce700 1 -- 172.21.15.49:0/521623196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 msgr2=0x556430623f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:19:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.285+0000 7f8f392ce700 1 --2- 172.21.15.49:0/521623196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x556430623f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.285+0000 7f8f392ce700 1 -- 172.21.15.49:0/521623196 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564303bf250 con 0x55643061c800 2023-10-08T09:19:43.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.285+0000 7f8f392ce700 1 --2- 172.21.15.49:0/521623196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x5564306244d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c09a570 tx=0x7f8f2c09a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2804e78614233b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:43.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.288+0000 7f8f1b7fe700 1 -- 172.21.15.49:0/521623196 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0850d0 con 0x55643061c800 2023-10-08T09:19:43.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.288+0000 7f8f1b7fe700 1 -- 172.21.15.49:0/521623196 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f2c09a3b0 con 0x55643061c800 2023-10-08T09:19:43.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.288+0000 7f8f1b7fe700 1 -- 172.21.15.49:0/521623196 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c09a840 con 0x55643061c800 2023-10-08T09:19:43.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.289+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/521623196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 msgr2=0x5564306244d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.289+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/521623196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x5564306244d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c09a570 tx=0x7f8f2c09a6c0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.297+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/521623196 shutdown_connections 2023-10-08T09:19:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.297+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/521623196 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061c800 0x5564306244d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.297+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/521623196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061b060 0x556430623f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.297+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/521623196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b9c0 0x55643062c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.298+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/521623196 >> 172.21.15.49:0/521623196 conn(0x55643058f260 msgr2=0x556430624dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:43.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.303+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/521623196 shutdown_connections 2023-10-08T09:19:43.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.311+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/521623196 wait complete. 2023-10-08T09:19:43.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.311+0000 7f8f46f1ab80 1 -- start start 2023-10-08T09:19:43.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.311+0000 7f8f46f1ab80 1 Processor -- start 2023-10-08T09:19:43.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.312+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 0x55643062bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.312+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x556430628b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.312+0000 7f8f46f1ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061c800 0x5564306290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.312+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564306295e0 con 0x55643061b9c0 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.312+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430629750 con 0x55643061c800 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.312+0000 7f8f46f1ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556430684a60 con 0x55643061b060 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.312+0000 7f8f46f1ab80 10 client.0 _ll_register_callbacks cb 0x556430367800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.313+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061c800 0x5564306290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.313+0000 7f8f392ce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061c800 0x5564306290a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41244/0 (socket says 172.21.15.49:41244) 2023-10-08T09:19:43.315 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.313+0000 7f8f392ce700 1 -- 172.21.15.49:0/644151376 learned_addr learned my addr 172.21.15.49:0/644151376 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.314+0000 7f8f33fff700 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x556430628b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.314+0000 7f8f33fff700 1 -- 172.21.15.49:0/644151376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 msgr2=0x55643062bf80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:19:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.314+0000 7f8f33fff700 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 0x55643062bf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.314+0000 7f8f33fff700 1 -- 172.21.15.49:0/644151376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061c800 msgr2=0x5564306290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:43.322 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.314+0000 7f8f33fff700 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061c800 0x5564306290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.314+0000 7f8f33fff700 1 -- 172.21.15.49:0/644151376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556430381c60 con 0x55643061b9c0 2023-10-08T09:19:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.315+0000 7f8f33fff700 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x556430628b60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x556430630880 tx=0x7f8f24010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b38f2c562b8b8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.315+0000 7f8f13fff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:19:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.315+0000 7f8f13fff700 1 -- 172.21.15.49:0/644151376 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240143f0 con 0x55643061b9c0 2023-10-08T09:19:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.315+0000 7f8f13fff700 1 -- 172.21.15.49:0/644151376 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8f24014550 con 0x55643061b9c0 2023-10-08T09:19:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.315+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f28005940 con 0x55643061b9c0 2023-10-08T09:19:43.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.316+0000 7f8f13fff700 1 -- 172.21.15.49:0/644151376 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f24014710 con 0x55643061b9c0 2023-10-08T09:19:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.322+0000 7f8f46f1ab80 10 client.16305 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:43.324 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.322+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5564303972b0 con 0x55643061b9c0 2023-10-08T09:19:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f13fff700 1 -- 172.21.15.49:0/644151376 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f28005940 con 0x55643061b9c0 2023-10-08T09:19:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f13fff700 1 -- 172.21.15.49:0/644151376 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8f2401ca00 con 0x55643061b9c0 2023-10-08T09:19:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f46f1ab80 10 client.16305 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:43.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f46f1ab80 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f46f1ab80 20 client.16305 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430686520 0x556430688940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f46f1ab80 4 client.16305 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:19:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.323+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556430566b30 con 0x556430686520 2023-10-08T09:19:43.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.328+0000 7f8f38acd700 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430686520 0x556430688940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.346+0000 7f8f13fff700 10 client.16305 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.346+0000 7f8f38acd700 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430686520 0x556430688940 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30492b71e66299c8 server_cookie=1f23abd841436ed in_seq=0 out_seq=0 2023-10-08T09:19:43.349 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.346+0000 7f8f13fff700 1 -- 172.21.15.49:0/644151376 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556430566b30 con 0x556430686520 2023-10-08T09:19:43.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.346+0000 7f8f13fff700 10 client.16305 handle_command_reply: tid=0 2023-10-08T09:19:43.350 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.346+0000 7f8f46f1ab80 1 client.16305 shutdown 2023-10-08T09:19:43.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.354+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 msgr2=0x556430628b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.354+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x556430628b60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x556430630880 tx=0x7f8f24010520 comp rx=0 tx=0).stop 2023-10-08T09:19:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.358+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 shutdown_connections 2023-10-08T09:19:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.358+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556430686520 0x556430688940 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.358+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55643061b060 0x55643062bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.358+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55643061c800 0x5564306290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.363 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.358+0000 7f8f46f1ab80 1 --2- 172.21.15.49:0/644151376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55643061b9c0 0x556430628b60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.358+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 >> 172.21.15.49:0/644151376 conn(0x55643058f260 msgr2=0x55643061f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:43.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.358+0000 7f8f13fff700 0 client.16305 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:43.366 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.364+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 shutdown_connections 2023-10-08T09:19:43.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.369+0000 7f8f46f1ab80 1 -- 172.21.15.49:0/644151376 wait complete. 2023-10-08T09:19:43.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.372+0000 7f8f46f1ab80 20 client.16305 trim_cache size 0 max 16384 2023-10-08T09:19:43.373 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:19:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:19:43.374 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:19:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.388+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8f1c0a1580 msgr2=0x7f8f1c0a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.388+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8f1c0a1580 0x7f8f1c0a3a40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f28025000 tx=0x7f8f28017a30 comp rx=0 tx=0).stop 2023-10-08T09:19:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.388+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3410cb30 msgr2=0x7f8f341acbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.388+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3410cb30 0x7f8f341acbb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00b970 tx=0x7f8f2c00bc80 comp rx=0 tx=0).stop 2023-10-08T09:19:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.395+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 shutdown_connections 2023-10-08T09:19:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.395+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8f1c0a1580 0x7f8f1c0a3a40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.395+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f3410cb30 0x7f8f341acbb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.395+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8f341077f0 0x7f8f341a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.395+0000 7f8f3ad31700 1 --2- 172.21.15.49:0/2945540461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f3410a160 0x7f8f341a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:43.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.395+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 >> 172.21.15.49:0/2945540461 conn(0x7f8f3406d790 msgr2=0x7f8f3410b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.404+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 shutdown_connections 2023-10-08T09:19:43.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:19:43.404+0000 7f8f3ad31700 1 -- 172.21.15.49:0/2945540461 wait complete. 2023-10-08T09:19:43.440 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:19:43.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:43 smithi184 ceph-mon[103675]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 MiB/s wr, 704 op/s 2023-10-08T09:19:44.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:43 smithi132 ceph-mon[103389]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 MiB/s wr, 704 op/s 2023-10-08T09:19:44.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:43 smithi049 ceph-mon[99485]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.1 MiB/s wr, 704 op/s 2023-10-08T09:19:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 260.0 s, 691.0 tps, lat 148.885 ms stddev 169.820 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 -- 172.21.15.49:0/130622879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb08075510 msgr2=0x7fbb080779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 --2- 172.21.15.49:0/130622879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb08075510 0x7fbb080779a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009d40 tx=0x7fbaf800b540 comp rx=0 tx=0).stop 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 -- 172.21.15.49:0/130622879 shutdown_connections 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 --2- 172.21.15.49:0/130622879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb08077ee0 0x7fbb0807a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 --2- 172.21.15.49:0/130622879 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb080725e0 0x7fbb08072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 --2- 172.21.15.49:0/130622879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb08075510 0x7fbb080779a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 -- 172.21.15.49:0/130622879 >> 172.21.15.49:0/130622879 conn(0x7fbb0806d770 msgr2=0x7fbb0806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 -- 172.21.15.49:0/130622879 shutdown_connections 2023-10-08T09:19:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.486+0000 7fbb103dc700 1 -- 172.21.15.49:0/130622879 wait complete. 2023-10-08T09:19:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 Processor -- start 2023-10-08T09:19:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 -- start start 2023-10-08T09:19:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb080725e0 0x7fbb081ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb08075510 0x7fbb081aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb08077ee0 0x7fbb081aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081b7a00 con 0x7fbb08075510 2023-10-08T09:19:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081b7b70 con 0x7fbb08077ee0 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb103dc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081b7d30 con 0x7fbb080725e0 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb080725e0 0x7fbb081ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0d977700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb08075510 0x7fbb081aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb080725e0 0x7fbb081ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38674/0 (socket says 172.21.15.49:38674) 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e178700 1 -- 172.21.15.49:0/1610545425 learned_addr learned my addr 172.21.15.49:0/1610545425 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e979700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb08077ee0 0x7fbb081aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e178700 1 -- 172.21.15.49:0/1610545425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb08077ee0 msgr2=0x7fbb081aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e178700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb08077ee0 0x7fbb081aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e178700 1 -- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb08075510 msgr2=0x7fbb081aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.487+0000 7fbb0e178700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb08075510 0x7fbb081aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.488+0000 7fbb0e178700 1 -- 172.21.15.49:0/1610545425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf8009520 con 0x7fbb080725e0 2023-10-08T09:19:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.488+0000 7fbb0e178700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb080725e0 0x7fbb081ae510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400e9a0 tx=0x7fbb0400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.488+0000 7fbaff7fe700 1 -- 172.21.15.49:0/1610545425 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04009790 con 0x7fbb080725e0 2023-10-08T09:19:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.488+0000 7fbaff7fe700 1 -- 172.21.15.49:0/1610545425 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb04010430 con 0x7fbb080725e0 2023-10-08T09:19:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.488+0000 7fbaff7fe700 1 -- 172.21.15.49:0/1610545425 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04018680 con 0x7fbb080725e0 2023-10-08T09:19:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.491+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb081b7fb0 con 0x7fbb080725e0 2023-10-08T09:19:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.491+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb081b83e0 con 0x7fbb080725e0 2023-10-08T09:19:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.494+0000 7fbaff7fe700 1 -- 172.21.15.49:0/1610545425 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbb040105a0 con 0x7fbb080725e0 2023-10-08T09:19:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbaff7fe700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbaf40a15b0 0x7fbaf40a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 Processor -- start 2023-10-08T09:19:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbaff7fe700 1 -- 172.21.15.49:0/1610545425 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbb04014070 con 0x7fbb080725e0 2023-10-08T09:19:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb0d977700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbaf40a15b0 0x7fbaf40a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 -- start start 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d807260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d807cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8082b0 0x555c6d808730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d76f400 con 0x555c6d806e40 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d760bc0 con 0x555c6d8077a0 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d75e4e0 con 0x555c6d8082b0 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.495+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d807260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d807260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51648/0 (socket says 172.21.15.49:51648) 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e178700 1 -- 172.21.15.49:0/3930448803 learned_addr learned my addr 172.21.15.49:0/3930448803 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e979700 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8082b0 0x555c6d808730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0d977700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbaf40a15b0 0x7fbaf40a3a70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8017460 tx=0x7fbaf801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0d977700 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d807cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e178700 1 -- 172.21.15.49:0/3930448803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8082b0 msgr2=0x555c6d808730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e178700 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8082b0 0x555c6d808730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e178700 1 -- 172.21.15.49:0/3930448803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 msgr2=0x555c6d807cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e178700 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d807cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.496+0000 7fbb0e178700 1 -- 172.21.15.49:0/3930448803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c6d584d80 con 0x555c6d806e40 2023-10-08T09:19:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.497+0000 7fbb0e178700 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d807260 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbb04096d00 tx=0x7fbb0409aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06bf12c7ba0e2a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.498+0000 7fbafcd82700 1 -- 172.21.15.49:0/3930448803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04094ea0 con 0x555c6d806e40 2023-10-08T09:19:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.498+0000 7fbafcd82700 1 -- 172.21.15.49:0/3930448803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb040cd0b0 con 0x555c6d806e40 2023-10-08T09:19:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.498+0000 7fbafcd82700 1 -- 172.21.15.49:0/3930448803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0409a750 con 0x555c6d806e40 2023-10-08T09:19:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.499+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/3930448803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 msgr2=0x555c6d807260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.499+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d807260 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbb04096d00 tx=0x7fbb0409aa10 comp rx=0 tx=0).stop 2023-10-08T09:19:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.500+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/3930448803 shutdown_connections 2023-10-08T09:19:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.500+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8082b0 0x555c6d808730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.500+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d807260 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.500+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/3930448803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d807cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.500+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/3930448803 >> 172.21.15.49:0/3930448803 conn(0x555c6d77b200 msgr2=0x555c6d8018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.500+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/3930448803 shutdown_connections 2023-10-08T09:19:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.500+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/3930448803 wait complete. 2023-10-08T09:19:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 -- start start 2023-10-08T09:19:45.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 Processor -- start 2023-10-08T09:19:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d806e40 0x555c6d810760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d810ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d817ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d759dd0 con 0x555c6d8082b0 2023-10-08T09:19:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d6f66b0 con 0x555c6d8077a0 2023-10-08T09:19:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d6f6350 con 0x555c6d806e40 2023-10-08T09:19:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb1c5c5b80 10 client.0 _ll_register_callbacks cb 0x555c6d5537a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.501+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d806e40 0x555c6d810760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d806e40 0x555c6d810760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38688/0 (socket says 172.21.15.49:38688) 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0e178700 1 -- 172.21.15.49:0/2350708225 learned_addr learned my addr 172.21.15.49:0/2350708225 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0d977700 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d810ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0e979700 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d817ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0d977700 1 -- 172.21.15.49:0/2350708225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d806e40 msgr2=0x555c6d810760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0d977700 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d806e40 0x555c6d810760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0d977700 1 -- 172.21.15.49:0/2350708225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 msgr2=0x555c6d817ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0d977700 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d817ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0d977700 1 -- 172.21.15.49:0/2350708225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb081b83e0 con 0x555c6d8077a0 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbb0d977700 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d810ca0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800ff30 tx=0x7fbaf8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732d880117c391ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:45.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.502+0000 7fbaf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:19:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.503+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2350708225 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf802c070 con 0x555c6d8077a0 2023-10-08T09:19:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.503+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2350708225 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbaf8003890 con 0x555c6d8077a0 2023-10-08T09:19:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.503+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c6d721e50 con 0x555c6d8077a0 2023-10-08T09:19:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.503+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2350708225 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf801d040 con 0x555c6d8077a0 2023-10-08T09:19:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.503+0000 7fbb1c5c5b80 10 client.25952 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.503+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555c6d56dc00 con 0x555c6d8077a0 2023-10-08T09:19:45.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.503+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2350708225 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c6d721e50 con 0x555c6d8077a0 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2350708225 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fbaf8017500 con 0x555c6d8077a0 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb1c5c5b80 10 client.25952 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb1c5c5b80 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb1c5c5b80 20 client.25952 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d820990 0x555c6d822df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb1c5c5b80 4 client.25952 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c6d752db0 con 0x555c6d820990 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb0e178700 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d820990 0x555c6d822df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbaf1ffb700 10 client.25952 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.504+0000 7fbb0e178700 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d820990 0x555c6d822df0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbea9e0fe0e0fe9 server_cookie=e515ea05ef1a74db in_seq=0 out_seq=0 2023-10-08T09:19:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2350708225 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c6d752db0 con 0x555c6d820990 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbaf1ffb700 10 client.25952 handle_command_reply: tid=0 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 client.25952 shutdown 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 msgr2=0x555c6d810ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d810ca0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800ff30 tx=0x7fbaf8009260 comp rx=0 tx=0).stop 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 shutdown_connections 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d820990 0x555c6d822df0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d806e40 0x555c6d810760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d817ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbaf1ffb700 0 client.25952 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2350708225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8077a0 0x555c6d810ca0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.510 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 >> 172.21.15.49:0/2350708225 conn(0x555c6d77b200 msgr2=0x555c6d801010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 shutdown_connections 2023-10-08T09:19:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.505+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2350708225 wait complete. 2023-10-08T09:19:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.506+0000 7fbb1c5c5b80 20 client.25952 trim_cache size 0 max 16384 2023-10-08T09:19:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.529+0000 7fbb1c5c5b80 1 Processor -- start 2023-10-08T09:19:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.529+0000 7fbb1c5c5b80 1 -- start start 2023-10-08T09:19:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d806e40 0x555c6d807220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 0x555c6d817ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d80e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d5da8c0 con 0x555c6d8082b0 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d7fc2b0 con 0x555c6d806e40 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d811950 con 0x555c6d8077a0 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d80e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d80e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51662/0 (socket says 172.21.15.49:51662) 2023-10-08T09:19:45.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 -- 172.21.15.49:0/1433749274 learned_addr learned my addr 172.21.15.49:0/1433749274 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 -- 172.21.15.49:0/1433749274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 msgr2=0x555c6d817ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:19:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 --2- 172.21.15.49:0/1433749274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 0x555c6d817ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 -- 172.21.15.49:0/1433749274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d806e40 msgr2=0x555c6d807220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 --2- 172.21.15.49:0/1433749274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d806e40 0x555c6d807220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.530+0000 7fbb0e979700 1 -- 172.21.15.49:0/1433749274 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c6d5ab1f0 con 0x555c6d8082b0 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.531+0000 7fbb0e979700 1 --2- 172.21.15.49:0/1433749274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d80e750 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbb000129a0 tx=0x7fbb000099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc06bde1ed20eb19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.531+0000 7fbafd583700 1 -- 172.21.15.49:0/1433749274 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000e040 con 0x555c6d8082b0 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.531+0000 7fbafd583700 1 -- 172.21.15.49:0/1433749274 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb0000f040 con 0x555c6d8082b0 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.531+0000 7fbafd583700 1 -- 172.21.15.49:0/1433749274 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00007ba0 con 0x555c6d8082b0 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/1433749274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 msgr2=0x555c6d80e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/1433749274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d80e750 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbb000129a0 tx=0x7fbb000099e0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/1433749274 shutdown_connections 2023-10-08T09:19:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/1433749274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d8082b0 0x555c6d80e750 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/1433749274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d806e40 0x555c6d807220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/1433749274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 0x555c6d817ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/1433749274 >> 172.21.15.49:0/1433749274 conn(0x555c6d77b200 msgr2=0x555c6d827750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/1433749274 shutdown_connections 2023-10-08T09:19:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.532+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/1433749274 wait complete. 2023-10-08T09:19:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 -- start start 2023-10-08T09:19:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 Processor -- start 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d813790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 0x555c6d813cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8082b0 0x555c6d816ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d8171e0 con 0x555c6d806e40 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d817350 con 0x555c6d8082b0 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c6d817510 con 0x555c6d8077a0 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb1c5c5b80 10 client.0 _ll_register_callbacks cb 0x7fbaf8005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d813790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb0e178700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d813790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51664/0 (socket says 172.21.15.49:51664) 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb0e178700 1 -- 172.21.15.49:0/2126532053 learned_addr learned my addr 172.21.15.49:0/2126532053 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.533+0000 7fbb0e979700 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8082b0 0x555c6d816ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb0e178700 1 -- 172.21.15.49:0/2126532053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 msgr2=0x555c6d813cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb0e178700 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 0x555c6d813cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb0e178700 1 -- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8082b0 msgr2=0x555c6d816ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb0e178700 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8082b0 0x555c6d816ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb0e178700 1 -- 172.21.15.49:0/2126532053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb00012620 con 0x555c6d806e40 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb0e178700 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d813790 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbb0409a220 tx=0x7fbb0400e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafbb6b79b466a4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbaf1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:19:45.540 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2126532053 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04094ea0 con 0x555c6d806e40 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2126532053 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbb040983d0 con 0x555c6d806e40 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb040cd0b0 con 0x555c6d806e40 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2126532053 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb0409a060 con 0x555c6d806e40 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb1c5c5b80 10 client.16323 fetch_fsmap learned FSMap version 39 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555c6d583250 con 0x555c6d806e40 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2126532053 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb040cd0b0 con 0x555c6d806e40 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2126532053 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 39) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fbb0409a4f0 con 0x555c6d806e40 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb1c5c5b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.534+0000 7fbb1c5c5b80 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.535+0000 7fbb1c5c5b80 20 client.16323 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.535+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d8751b0 0x555c6d8775d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:45.541 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.535+0000 7fbb1c5c5b80 4 client.16323 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:19:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.535+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555c6d752ad0 con 0x555c6d8751b0 2023-10-08T09:19:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.535+0000 7fbb0d977700 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d8751b0 0x555c6d8775d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.536+0000 7fbaf1ffb700 10 client.16323 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:45.542 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.536+0000 7fbb0d977700 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d8751b0 0x555c6d8775d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd61ff0fe3dcc0ab server_cookie=b424c7f264f5a992 in_seq=0 out_seq=0 2023-10-08T09:19:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.546+0000 7fbaf1ffb700 1 -- 172.21.15.49:0/2126532053 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555c6d752ad0 con 0x555c6d8751b0 2023-10-08T09:19:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.546+0000 7fbaf1ffb700 10 client.16323 handle_command_reply: tid=0 2023-10-08T09:19:45.548 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 client.16323 shutdown 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 msgr2=0x555c6d813790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d813790 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbb0409a220 tx=0x7fbb0400e8f0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 shutdown_connections 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x555c6d8751b0 0x555c6d8775d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x555c6d8082b0 0x555c6d816ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x555c6d806e40 0x555c6d813790 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 --2- 172.21.15.49:0/2126532053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x555c6d8077a0 0x555c6d813cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.549 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbaf1ffb700 0 client.16323 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 >> 172.21.15.49:0/2126532053 conn(0x555c6d77b200 msgr2=0x555c6d80fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 shutdown_connections 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 1 -- 172.21.15.49:0/2126532053 wait complete. 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.547+0000 7fbb1c5c5b80 20 client.16323 trim_cache size 0 max 16384 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:19:45.550 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:19:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.561+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbaf40a15b0 msgr2=0x7fbaf40a3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.561+0000 7fbb103dc700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbaf40a15b0 0x7fbaf40a3a70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8017460 tx=0x7fbaf801a040 comp rx=0 tx=0).stop 2023-10-08T09:19:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.561+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb080725e0 msgr2=0x7fbb081ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.561+0000 7fbb103dc700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb080725e0 0x7fbb081ae510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400e9a0 tx=0x7fbb0400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.570+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 shutdown_connections 2023-10-08T09:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.570+0000 7fbb103dc700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbaf40a15b0 0x7fbaf40a3a70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.570+0000 7fbb103dc700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb080725e0 0x7fbb081ae510 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.570+0000 7fbb103dc700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb08077ee0 0x7fbb081aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.570+0000 7fbb103dc700 1 --2- 172.21.15.49:0/1610545425 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb08075510 0x7fbb081aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:45.584 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.570+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 >> 172.21.15.49:0/1610545425 conn(0x7fbb0806d770 msgr2=0x7fbb08079540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.589+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 shutdown_connections 2023-10-08T09:19:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:45.589+0000 7fbb103dc700 1 -- 172.21.15.49:0/1610545425 wait complete. 2023-10-08T09:19:45.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:45 smithi184 ceph-mon[103675]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.9 MiB/s wr, 705 op/s 2023-10-08T09:19:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:45 smithi132 ceph-mon[103389]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.9 MiB/s wr, 705 op/s 2023-10-08T09:19:46.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:45 smithi049 ceph-mon[99485]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.9 MiB/s wr, 705 op/s 2023-10-08T09:19:47.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:47 smithi184 ceph-mon[103675]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 779 op/s 2023-10-08T09:19:48.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:47 smithi132 ceph-mon[103389]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 779 op/s 2023-10-08T09:19:48.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:47 smithi049 ceph-mon[99485]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 779 op/s 2023-10-08T09:19:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 265.0 s, 969.4 tps, lat 102.035 ms stddev 114.778 2023-10-08T09:19:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:49 smithi184 ceph-mon[103675]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 MiB/s wr, 726 op/s 2023-10-08T09:19:50.052 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:49 smithi132 ceph-mon[103389]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 MiB/s wr, 726 op/s 2023-10-08T09:19:50.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:49 smithi049 ceph-mon[99485]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.1 MiB/s wr, 726 op/s 2023-10-08T09:19:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:51 smithi184 ceph-mon[103675]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 774 op/s 2023-10-08T09:19:52.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:51 smithi132 ceph-mon[103389]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 774 op/s 2023-10-08T09:19:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:51 smithi049 ceph-mon[99485]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 774 op/s 2023-10-08T09:19:53.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:52 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:52 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:53.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:52 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:19:54.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:53 smithi132 ceph-mon[103389]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 811 op/s 2023-10-08T09:19:54.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:53 smithi049 ceph-mon[99485]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 811 op/s 2023-10-08T09:19:54.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:53 smithi184 ceph-mon[103675]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 811 op/s 2023-10-08T09:19:54.319 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 270.0 s, 752.8 tps, lat 132.152 ms stddev 158.183 2023-10-08T09:19:56.059 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:55 smithi132 ceph-mon[103389]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 774 op/s 2023-10-08T09:19:56.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:55 smithi049 ceph-mon[99485]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 774 op/s 2023-10-08T09:19:56.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:55 smithi184 ceph-mon[103675]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 MiB/s wr, 774 op/s 2023-10-08T09:19:58.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:57 smithi049 ceph-mon[99485]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 722 op/s 2023-10-08T09:19:58.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:57 smithi184 ceph-mon[103675]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 722 op/s 2023-10-08T09:19:58.301 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:57 smithi132 ceph-mon[103389]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 722 op/s 2023-10-08T09:19:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.812+0000 7f3b000dd700 1 -- 172.21.15.49:0/203653383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af81077f0 msgr2=0x7f3af8109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.812+0000 7f3b000dd700 1 --2- 172.21.15.49:0/203653383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af81077f0 0x7f3af8109c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3af40098f0 tx=0x7f3af4009c00 comp rx=0 tx=0).stop 2023-10-08T09:19:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.819+0000 7f3b000dd700 1 -- 172.21.15.49:0/203653383 shutdown_connections 2023-10-08T09:19:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.819+0000 7f3b000dd700 1 --2- 172.21.15.49:0/203653383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af81077f0 0x7f3af8109c20 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3af40098f0 tx=0x7f3af4009c00 comp rx=0 tx=0).stop 2023-10-08T09:19:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.819+0000 7f3b000dd700 1 --2- 172.21.15.49:0/203653383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af810cb30 0x7f3af810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.819+0000 7f3b000dd700 1 --2- 172.21.15.49:0/203653383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3af810a160 0x7f3af810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.819+0000 7f3b000dd700 1 -- 172.21.15.49:0/203653383 >> 172.21.15.49:0/203653383 conn(0x7f3af806d770 msgr2=0x7f3af806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.829+0000 7f3b000dd700 1 -- 172.21.15.49:0/203653383 shutdown_connections 2023-10-08T09:19:58.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.834+0000 7f3b000dd700 1 -- 172.21.15.49:0/203653383 wait complete. 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.835+0000 7f3b000dd700 1 Processor -- start 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3b000dd700 1 -- start start 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3b000dd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3af810a160 0x7f3af81173d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3b000dd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af810cb30 0x7f3af8117910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3b000dd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af8118730 0x7f3af8118be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3b000dd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af81191e0 con 0x7f3af810a160 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3b000dd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8119350 con 0x7f3af8118730 2023-10-08T09:19:58.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3b000dd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8119510 con 0x7f3af810cb30 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3afe67a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af8118730 0x7f3af8118be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.836+0000 7f3afe67a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af8118730 0x7f3af8118be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41292/0 (socket says 172.21.15.49:41292) 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.837+0000 7f3afe67a700 1 -- 172.21.15.49:0/3267976795 learned_addr learned my addr 172.21.15.49:0/3267976795 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.837+0000 7f3afd678700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af810cb30 0x7f3af8117910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.837+0000 7f3afe67a700 1 -- 172.21.15.49:0/3267976795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af810cb30 msgr2=0x7f3af8117910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.837+0000 7f3afe67a700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af810cb30 0x7f3af8117910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.837+0000 7f3afe67a700 1 -- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3af810a160 msgr2=0x7f3af81173d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.837+0000 7f3afe67a700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3af810a160 0x7f3af81173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.837+0000 7f3afe67a700 1 -- 172.21.15.49:0/3267976795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4009520 con 0x7f3af8118730 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.838+0000 7f3afe67a700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af8118730 0x7f3af8118be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800e970 tx=0x7f3ae800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.838+0000 7f3aeeffd700 1 -- 172.21.15.49:0/3267976795 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8004a30 con 0x7f3af8118730 2023-10-08T09:19:58.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.838+0000 7f3aeeffd700 1 -- 172.21.15.49:0/3267976795 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ae800cc80 con 0x7f3af8118730 2023-10-08T09:19:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.838+0000 7f3aeeffd700 1 -- 172.21.15.49:0/3267976795 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8005560 con 0x7f3af8118730 2023-10-08T09:19:58.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.838+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af8078b10 con 0x7f3af8118730 2023-10-08T09:19:58.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.839+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af8078fd0 con 0x7f3af8118730 2023-10-08T09:19:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.843+0000 7f3b0c2c6b80 1 Processor -- start 2023-10-08T09:19:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3b0c2c6b80 1 -- start start 2023-10-08T09:19:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b98795e40 0x564b98796260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b98796cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b98797730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b986fe400 con 0x564b987972b0 2023-10-08T09:19:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b986efbc0 con 0x564b98795e40 2023-10-08T09:19:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b986ed4e0 con 0x564b987967a0 2023-10-08T09:19:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3afde79700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b98796cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3afde79700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b98796cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38718/0 (socket says 172.21.15.49:38718) 2023-10-08T09:19:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.844+0000 7f3afde79700 1 -- 172.21.15.49:0/2580679539 learned_addr learned my addr 172.21.15.49:0/2580679539 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.848+0000 7f3afd678700 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b98797730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.848+0000 7f3afd678700 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b98795e40 0x564b98796260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.850+0000 7f3aeeffd700 1 -- 172.21.15.49:0/3267976795 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3ae800c7c0 con 0x7f3af8118730 2023-10-08T09:19:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.851+0000 7f3aeeffd700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3ae40a1680 0x7f3ae40a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.851+0000 7f3aeeffd700 1 -- 172.21.15.49:0/3267976795 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ae8014070 con 0x7f3af8118730 2023-10-08T09:19:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.855+0000 7f3afde79700 1 -- 172.21.15.49:0/2580679539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b98795e40 msgr2=0x564b98796260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.855+0000 7f3afde79700 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b98795e40 0x564b98796260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.855+0000 7f3afde79700 1 -- 172.21.15.49:0/2580679539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 msgr2=0x564b98797730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.855+0000 7f3afde79700 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b98797730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.855+0000 7f3afde79700 1 -- 172.21.15.49:0/2580679539 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b98513d80 con 0x564b987967a0 2023-10-08T09:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.858+0000 7f3afde79700 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b98796cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009810 tx=0x7f3af400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7cf0c7bf53fc92a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.858+0000 7f3afe67a700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3ae40a1680 0x7f3ae40a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.866+0000 7f3ae3fff700 1 -- 172.21.15.49:0/2580679539 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af401c070 con 0x564b987967a0 2023-10-08T09:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.866+0000 7f3ae3fff700 1 -- 172.21.15.49:0/2580679539 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3af400b560 con 0x564b987967a0 2023-10-08T09:19:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.866+0000 7f3ae3fff700 1 -- 172.21.15.49:0/2580679539 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4005280 con 0x564b987967a0 2023-10-08T09:19:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.869+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2580679539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 msgr2=0x564b98796cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:58.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.869+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b98796cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009810 tx=0x7f3af400bef0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.870+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2580679539 shutdown_connections 2023-10-08T09:19:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.870+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b98797730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.870+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b98795e40 0x564b98796260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.870+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2580679539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b98796cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.870+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2580679539 >> 172.21.15.49:0/2580679539 conn(0x564b9870a200 msgr2=0x564b987908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.880+0000 7f3afe67a700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3ae40a1680 0x7f3ae40a3b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3ae809a130 tx=0x7f3ae8004f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:58.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.886+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2580679539 shutdown_connections 2023-10-08T09:19:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.887+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2580679539 wait complete. 2023-10-08T09:19:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.887+0000 7f3b0c2c6b80 1 -- start start 2023-10-08T09:19:58.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.887+0000 7f3b0c2c6b80 1 Processor -- start 2023-10-08T09:19:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.891+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b98795e40 0x564b9879f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.891+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b9879fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.891+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987972b0 0x564b987a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.891+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b986e8dd0 con 0x564b98795e40 2023-10-08T09:19:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.891+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b986856b0 con 0x564b987972b0 2023-10-08T09:19:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.891+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b98685350 con 0x564b987967a0 2023-10-08T09:19:58.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.891+0000 7f3b0c2c6b80 10 client.0 _ll_register_callbacks cb 0x564b984e27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987972b0 0x564b987a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b98795e40 0x564b9879f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b98795e40 0x564b9879f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51692/0 (socket says 172.21.15.49:51692) 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 -- 172.21.15.49:0/2130672290 learned_addr learned my addr 172.21.15.49:0/2130672290 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 -- 172.21.15.49:0/2130672290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 msgr2=0x564b9879fd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b9879fd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 -- 172.21.15.49:0/2130672290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987972b0 msgr2=0x564b987a6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987972b0 0x564b987a6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.893+0000 7f3afd678700 1 -- 172.21.15.49:0/2130672290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af810da70 con 0x564b98795e40 2023-10-08T09:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.894+0000 7f3afd678700 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b98795e40 0x564b9879f800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3af000bf90 tx=0x7f3af0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219e81f1d450ca17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.895+0000 7f3ae17fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.895+0000 7f3ae17fa700 1 -- 172.21.15.49:0/2130672290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af001b680 con 0x564b98795e40 2023-10-08T09:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.895+0000 7f3ae17fa700 1 -- 172.21.15.49:0/2130672290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3af0006e40 con 0x564b98795e40 2023-10-08T09:19:58.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.895+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b986b0e50 con 0x564b98795e40 2023-10-08T09:19:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.908+0000 7f3ae17fa700 1 -- 172.21.15.49:0/2130672290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3af001b7e0 con 0x564b98795e40 2023-10-08T09:19:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.914+0000 7f3b0c2c6b80 10 client.16329 fetch_fsmap learned FSMap version 40 2023-10-08T09:19:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.914+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564b984fcc00 con 0x564b98795e40 2023-10-08T09:19:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.924+0000 7f3ae17fa700 1 -- 172.21.15.49:0/2130672290 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b986b0e50 con 0x564b98795e40 2023-10-08T09:19:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.924+0000 7f3ae17fa700 1 -- 172.21.15.49:0/2130672290 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3af0023070 con 0x564b98795e40 2023-10-08T09:19:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.924+0000 7f3b0c2c6b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:19:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.924+0000 7f3b0c2c6b80 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:58.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.924+0000 7f3b0c2c6b80 20 client.16329 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.925+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ab5c0 0x564b987ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.925+0000 7f3b0c2c6b80 4 client.16329 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:19:58.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.925+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b986e1db0 con 0x564b987ab5c0 2023-10-08T09:19:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.935+0000 7f3afde79700 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ab5c0 0x564b987ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.939+0000 7f3afde79700 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ab5c0 0x564b987ad9e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c933ae791b3bb8d9 server_cookie=52dab3c25d35d94f in_seq=0 out_seq=0 2023-10-08T09:19:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.947+0000 7f3ae17fa700 10 client.16329 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.947+0000 7f3ae17fa700 1 -- 172.21.15.49:0/2130672290 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564b986e1db0 con 0x564b987ab5c0 2023-10-08T09:19:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.947+0000 7f3ae17fa700 10 client.16329 handle_command_reply: tid=0 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.947+0000 7f3b0c2c6b80 1 client.16329 shutdown 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b98795e40 msgr2=0x564b9879f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b98795e40 0x564b9879f800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3af000bf90 tx=0x7f3af0009b20 comp rx=0 tx=0).stop 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 shutdown_connections 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ab5c0 0x564b987ad9e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987972b0 0x564b987a6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b98795e40 0x564b9879f800 secure :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3af000bf90 tx=0x7f3af0009b20 comp rx=0 tx=0).stop 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/2130672290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b9879fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.948+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 >> 172.21.15.49:0/2130672290 conn(0x564b9870a200 msgr2=0x564b98790010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.958+0000 7f3ae17fa700 0 client.16329 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:58.961 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.958+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 shutdown_connections 2023-10-08T09:19:58.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.970+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/2130672290 wait complete. 2023-10-08T09:19:58.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:58.972+0000 7f3b0c2c6b80 20 client.16329 trim_cache size 0 max 16384 2023-10-08T09:19:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.001+0000 7f3b0c2c6b80 1 Processor -- start 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.002+0000 7f3b0c2c6b80 1 -- start start 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.002+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b9879edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.002+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.002+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b9879d9a0 0x564b9879f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.002+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b985698c0 con 0x564b987972b0 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.002+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b9878d830 con 0x564b9879d9a0 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.002+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b987a8b10 con 0x564b987967a0 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.003+0000 7f3afde79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.003+0000 7f3afde79700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51700/0 (socket says 172.21.15.49:51700) 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.003+0000 7f3afde79700 1 -- 172.21.15.49:0/1811186515 learned_addr learned my addr 172.21.15.49:0/1811186515 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3afde79700 1 -- 172.21.15.49:0/1811186515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 msgr2=0x564b9879edb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3afde79700 1 --2- 172.21.15.49:0/1811186515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b9879edb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3afde79700 1 -- 172.21.15.49:0/1811186515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b9879d9a0 msgr2=0x564b9879f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3afde79700 1 --2- 172.21.15.49:0/1811186515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b9879d9a0 0x564b9879f5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3afde79700 1 -- 172.21.15.49:0/1811186515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b9853a1f0 con 0x564b987972b0 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3afde79700 1 --2- 172.21.15.49:0/1811186515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a6d40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009230 tx=0x7f3af4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549866b3f09d2967 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3aecff9700 1 -- 172.21.15.49:0/1811186515 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af401c040 con 0x564b987972b0 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3aecff9700 1 -- 172.21.15.49:0/1811186515 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3af401d040 con 0x564b987972b0 2023-10-08T09:19:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.004+0000 7f3aecff9700 1 -- 172.21.15.49:0/1811186515 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af402a590 con 0x564b987972b0 2023-10-08T09:19:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.005+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/1811186515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 msgr2=0x564b987a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.005+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/1811186515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a6d40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009230 tx=0x7f3af4009260 comp rx=0 tx=0).stop 2023-10-08T09:19:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.010+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/1811186515 shutdown_connections 2023-10-08T09:19:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.010+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/1811186515 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b987967a0 0x564b9879edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.010+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/1811186515 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b9879d9a0 0x564b9879f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.010+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/1811186515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a6d40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.010+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/1811186515 >> 172.21.15.49:0/1811186515 conn(0x564b9870a200 msgr2=0x564b987b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.018+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/1811186515 shutdown_connections 2023-10-08T09:19:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.019+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/1811186515 wait complete. 2023-10-08T09:19:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.019+0000 7f3b0c2c6b80 1 -- start start 2023-10-08T09:19:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.019+0000 7f3b0c2c6b80 1 Processor -- start 2023-10-08T09:19:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.025+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987967a0 0x564b987a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.025+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.025+0000 7f3b0c2c6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b9879d9a0 0x564b987a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.025+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b987a5f50 con 0x564b987972b0 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.025+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b987a60c0 con 0x564b987967a0 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.025+0000 7f3b0c2c6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b987a6280 con 0x564b9879d9a0 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.025+0000 7f3b0c2c6b80 10 client.0 _ll_register_callbacks cb 0x7f3af000be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987967a0 0x564b987a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987967a0 0x564b987a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41320/0 (socket says 172.21.15.49:41320) 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 -- 172.21.15.49:0/3577323710 learned_addr learned my addr 172.21.15.49:0/3577323710 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b9879d9a0 0x564b987a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afde79700 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 -- 172.21.15.49:0/3577323710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b9879d9a0 msgr2=0x564b987a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b9879d9a0 0x564b987a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 -- 172.21.15.49:0/3577323710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 msgr2=0x564b987a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:19:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 -- 172.21.15.49:0/3577323710 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b98513d80 con 0x564b987967a0 2023-10-08T09:19:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3afd678700 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987967a0 0x564b987a4600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3af0004b10 tx=0x7f3af0004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f580c105da01abb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:19:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3ae17fa700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:19:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.026+0000 7f3ae17fa700 1 -- 172.21.15.49:0/3577323710 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af001b680 con 0x564b987967a0 2023-10-08T09:19:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3ae17fa700 1 -- 172.21.15.49:0/3577323710 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3af0006e40 con 0x564b987967a0 2023-10-08T09:19:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3af400b560 con 0x564b987967a0 2023-10-08T09:19:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3ae17fa700 1 -- 172.21.15.49:0/3577323710 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3af00243a0 con 0x564b987967a0 2023-10-08T09:19:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3b0c2c6b80 10 client.25967 fetch_fsmap learned FSMap version 40 2023-10-08T09:19:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564b98512250 con 0x564b987967a0 2023-10-08T09:19:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3ae17fa700 1 -- 172.21.15.49:0/3577323710 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3af400b560 con 0x564b987967a0 2023-10-08T09:19:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3ae17fa700 1 -- 172.21.15.49:0/3577323710 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3af0038070 con 0x564b987967a0 2023-10-08T09:19:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3b0c2c6b80 10 client.25967 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:19:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3b0c2c6b80 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:19:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.027+0000 7f3b0c2c6b80 20 client.25967 populate_metadata read hostname 'smithi049' 2023-10-08T09:19:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.028+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ffcc0 0x564b988020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:19:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.028+0000 7f3b0c2c6b80 4 client.25967 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:19:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.028+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564b986e1ad0 con 0x564b987ffcc0 2023-10-08T09:19:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.036+0000 7f3afde79700 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ffcc0 0x564b988020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:19:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.038+0000 7f3ae17fa700 10 client.25967 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.038+0000 7f3afde79700 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ffcc0 0x564b988020b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4151186260ec7bec server_cookie=9c3cd2eec987b1e4 in_seq=0 out_seq=0 2023-10-08T09:19:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3ae17fa700 1 -- 172.21.15.49:0/3577323710 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564b986e1ad0 con 0x564b987ffcc0 2023-10-08T09:19:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3ae17fa700 10 client.25967 handle_command_reply: tid=0 2023-10-08T09:19:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 client.25967 shutdown 2023-10-08T09:19:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987967a0 msgr2=0x564b987a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:59.059 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987967a0 0x564b987a4600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3af0004b10 tx=0x7f3af0004be0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 shutdown_connections 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564b987ffcc0 0x564b988020b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564b9879d9a0 0x564b987a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564b987967a0 0x564b987a4600 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 --2- 172.21.15.49:0/3577323710 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564b987972b0 0x564b987a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 >> 172.21.15.49:0/3577323710 conn(0x564b9870a200 msgr2=0x564b9879edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.058+0000 7f3ae17fa700 0 client.25967 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:19:59.060 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.059+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 shutdown_connections 2023-10-08T09:19:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.059+0000 7f3b0c2c6b80 1 -- 172.21.15.49:0/3577323710 wait complete. 2023-10-08T09:19:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.059+0000 7f3b0c2c6b80 20 client.25967 trim_cache size 0 max 16384 2023-10-08T09:19:59.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:19:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:19:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:19:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:19:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.060+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3ae40a1680 msgr2=0x7f3ae40a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.060+0000 7f3b000dd700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3ae40a1680 0x7f3ae40a3b40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3ae809a130 tx=0x7f3ae8004f60 comp rx=0 tx=0).stop 2023-10-08T09:19:59.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.060+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af8118730 msgr2=0x7f3af8118be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:19:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.060+0000 7f3b000dd700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af8118730 0x7f3af8118be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800e970 tx=0x7f3ae800ec80 comp rx=0 tx=0).stop 2023-10-08T09:19:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.062+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 shutdown_connections 2023-10-08T09:19:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.062+0000 7f3b000dd700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3ae40a1680 0x7f3ae40a3b40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.062+0000 7f3b000dd700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3af8118730 0x7f3af8118be0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.062+0000 7f3b000dd700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3af810a160 0x7f3af81173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.062+0000 7f3b000dd700 1 --2- 172.21.15.49:0/3267976795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3af810cb30 0x7f3af8117910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:19:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.062+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 >> 172.21.15.49:0/3267976795 conn(0x7f3af806d770 msgr2=0x7f3af81081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:19:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.095+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 shutdown_connections 2023-10-08T09:19:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:19:59.098+0000 7f3b000dd700 1 -- 172.21.15.49:0/3267976795 wait complete. 2023-10-08T09:19:59.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 275.0 s, 507.1 tps, lat 193.648 ms stddev 208.362 2023-10-08T09:20:00.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:19:59 smithi049 ceph-mon[99485]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 602 op/s 2023-10-08T09:20:00.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:19:59 smithi184 ceph-mon[103675]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 602 op/s 2023-10-08T09:20:00.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:19:59 smithi132 ceph-mon[103389]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 602 op/s 2023-10-08T09:20:01.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:00 smithi049 ceph-mon[99485]: overall HEALTH_OK 2023-10-08T09:20:01.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:00 smithi184 ceph-mon[103675]: overall HEALTH_OK 2023-10-08T09:20:01.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:00 smithi132 ceph-mon[103389]: overall HEALTH_OK 2023-10-08T09:20:02.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:01 smithi049 ceph-mon[99485]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 681 op/s 2023-10-08T09:20:02.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:01 smithi184 ceph-mon[103675]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 681 op/s 2023-10-08T09:20:02.301 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:01 smithi132 ceph-mon[103389]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 681 op/s 2023-10-08T09:20:04.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:03 smithi049 ceph-mon[99485]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 715 op/s 2023-10-08T09:20:04.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:03 smithi184 ceph-mon[103675]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 715 op/s 2023-10-08T09:20:04.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:03 smithi132 ceph-mon[103389]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 715 op/s 2023-10-08T09:20:04.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 280.0 s, 578.1 tps, lat 178.198 ms stddev 198.756 2023-10-08T09:20:06.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:05 smithi049 ceph-mon[99485]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 712 op/s 2023-10-08T09:20:06.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:05 smithi184 ceph-mon[103675]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 712 op/s 2023-10-08T09:20:06.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:05 smithi132 ceph-mon[103389]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 712 op/s 2023-10-08T09:20:08.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:07 smithi049 ceph-mon[99485]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 731 op/s 2023-10-08T09:20:08.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:07 smithi184 ceph-mon[103675]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 731 op/s 2023-10-08T09:20:08.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:07 smithi132 ceph-mon[103389]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 731 op/s 2023-10-08T09:20:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 285.0 s, 633.6 tps, lat 155.408 ms stddev 165.220 2023-10-08T09:20:10.251 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:09 smithi184 ceph-mon[103675]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 692 op/s 2023-10-08T09:20:10.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:09 smithi132 ceph-mon[103389]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 692 op/s 2023-10-08T09:20:10.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:09 smithi049 ceph-mon[99485]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 692 op/s 2023-10-08T09:20:12.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:11 smithi184 ceph-mon[103675]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 726 op/s 2023-10-08T09:20:12.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:11 smithi132 ceph-mon[103389]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 726 op/s 2023-10-08T09:20:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.312+0000 7fcdc8c01700 1 -- 172.21.15.49:0/962229354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc40725e0 msgr2=0x7fcdc4072a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.312+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/962229354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc40725e0 0x7fcdc4072a00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0090c0 tx=0x7fcdbc00c640 comp rx=0 tx=0).stop 2023-10-08T09:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.313+0000 7fcdc8c01700 1 -- 172.21.15.49:0/962229354 shutdown_connections 2023-10-08T09:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.313+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/962229354 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc4077ee0 0x7fcdc407a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.313+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/962229354 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc40725e0 0x7fcdc4072a00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.313+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/962229354 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4075510 0x7fcdc40779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.313+0000 7fcdc8c01700 1 -- 172.21.15.49:0/962229354 >> 172.21.15.49:0/962229354 conn(0x7fcdc406d770 msgr2=0x7fcdc406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.313+0000 7fcdc8c01700 1 -- 172.21.15.49:0/962229354 shutdown_connections 2023-10-08T09:20:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.313+0000 7fcdc8c01700 1 -- 172.21.15.49:0/962229354 wait complete. 2023-10-08T09:20:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.314+0000 7fcdc8c01700 1 Processor -- start 2023-10-08T09:20:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.316+0000 7fcdc8c01700 1 -- start start 2023-10-08T09:20:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc8c01700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc40725e0 0x7fcdc41b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc8c01700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc4075510 0x7fcdc41b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc8c01700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4077ee0 0x7fcdc41b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc8c01700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b9cd0 con 0x7fcdc4075510 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc8c01700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b9e10 con 0x7fcdc40725e0 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc8c01700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41b9fa0 con 0x7fcdc4077ee0 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc4075510 0x7fcdc41b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc40725e0 0x7fcdc41b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc4075510 0x7fcdc41b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51720/0 (socket says 172.21.15.49:51720) 2023-10-08T09:20:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1022662151 learned_addr learned my addr 172.21.15.49:0/1022662151 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc40725e0 0x7fcdc41b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41332/0 (socket says 172.21.15.49:41332) 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc2d9d700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4077ee0 0x7fcdc41b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1022662151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4077ee0 msgr2=0x7fcdc41b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4077ee0 0x7fcdc41b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1022662151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc40725e0 msgr2=0x7fcdc41b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc40725e0 0x7fcdc41b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1022662151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc008cf0 con 0x7fcdc4075510 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc2d9d700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4077ee0 0x7fcdc41b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.317+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc40725e0 0x7fcdc41b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.318+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc4075510 0x7fcdc41b26e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80097d0 tx=0x7fcdb8009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.318+0000 7fcdb37fe700 1 -- 172.21.15.49:0/1022662151 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8010040 con 0x7fcdc4075510 2023-10-08T09:20:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.318+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc41ba280 con 0x7fcdc4075510 2023-10-08T09:20:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.318+0000 7fcdb37fe700 1 -- 172.21.15.49:0/1022662151 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdb8013c50 con 0x7fcdc4075510 2023-10-08T09:20:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.318+0000 7fcdb37fe700 1 -- 172.21.15.49:0/1022662151 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8014dc0 con 0x7fcdc4075510 2023-10-08T09:20:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.318+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc41ba7e0 con 0x7fcdc4075510 2023-10-08T09:20:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.321+0000 7fcdb37fe700 1 -- 172.21.15.49:0/1022662151 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcdb8014420 con 0x7fcdc4075510 2023-10-08T09:20:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.321+0000 7fcdb37fe700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcdac0a1480 0x7fcdac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.322+0000 7fcdb37fe700 1 -- 172.21.15.49:0/1022662151 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcdb80cf2a0 con 0x7fcdc4075510 2023-10-08T09:20:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.322+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcdac0a1480 0x7fcdac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.322+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcdac0a1480 0x7fcdac0a3940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00ca10 tx=0x7fcdbc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.322+0000 7fcdd4deab80 1 Processor -- start 2023-10-08T09:20:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdd4deab80 1 -- start start 2023-10-08T09:20:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c792447e40 0x55c792448260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792448cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924492b0 0x55c792449730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7923b0400 con 0x55c7924492b0 2023-10-08T09:20:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7923a1bc0 con 0x55c792447e40 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79239f4e0 con 0x55c7924487a0 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c792447e40 0x55c792448260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924492b0 0x55c792449730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc1d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792448cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c792447e40 0x55c792448260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41340/0 (socket says 172.21.15.49:41340) 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 -- 172.21.15.49:0/1807066764 learned_addr learned my addr 172.21.15.49:0/1807066764 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 -- 172.21.15.49:0/1807066764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 msgr2=0x55c792448cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792448cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 -- 172.21.15.49:0/1807066764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924492b0 msgr2=0x55c792449730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924492b0 0x55c792449730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc259c700 1 -- 172.21.15.49:0/1807066764 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7921c5d80 con 0x55c792447e40 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792448cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.323+0000 7fcdc2d9d700 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924492b0 0x55c792449730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.324+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c792447e40 0x55c792448260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc015000 tx=0x7fcdbc020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56c2148264d3447 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.324+0000 7fcdb0ff9700 1 -- 172.21.15.49:0/1807066764 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00eea0 con 0x55c792447e40 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.324+0000 7fcdb0ff9700 1 -- 172.21.15.49:0/1807066764 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdbc01e040 con 0x55c792447e40 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.324+0000 7fcdb0ff9700 1 -- 172.21.15.49:0/1807066764 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc017a30 con 0x55c792447e40 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.325+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1807066764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c792447e40 msgr2=0x55c792448260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.325+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c792447e40 0x55c792448260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc015000 tx=0x7fcdbc020000 comp rx=0 tx=0).stop 2023-10-08T09:20:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.327+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1807066764 shutdown_connections 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.327+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924492b0 0x55c792449730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.327+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c792447e40 0x55c792448260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.327+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1807066764 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792448cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.327+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1807066764 >> 172.21.15.49:0/1807066764 conn(0x55c7923bc200 msgr2=0x55c7924428a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.327+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1807066764 shutdown_connections 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.327+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1807066764 wait complete. 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 -- start start 2023-10-08T09:20:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 Processor -- start 2023-10-08T09:20:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c792447e40 0x55c792451820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924487a0 0x55c792451d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792458d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79239add0 con 0x55c7924487a0 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7923376b0 con 0x55c7924492b0 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c792337350 con 0x55c792447e40 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.328+0000 7fcdd4deab80 10 client.0 _ll_register_callbacks cb 0x55c7921947a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c792447e40 0x55c792451820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc2d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792458d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c792447e40 0x55c792451820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38766/0 (socket says 172.21.15.49:38766) 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc2d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792458d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41348/0 (socket says 172.21.15.49:41348) 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 -- 172.21.15.49:0/1469136108 learned_addr learned my addr 172.21.15.49:0/1469136108 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924487a0 0x55c792451d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 -- 172.21.15.49:0/1469136108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 msgr2=0x55c792458d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792458d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 -- 172.21.15.49:0/1469136108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924487a0 msgr2=0x55c792451d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924487a0 0x55c792451d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 -- 172.21.15.49:0/1469136108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc41ba7e0 con 0x55c792447e40 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c792447e40 0x55c792451820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc008fe0 tx=0x7fcdbc017e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d87701fd7882fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcda9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1469136108 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00eea0 con 0x55c792447e40 2023-10-08T09:20:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c792362e50 con 0x55c792447e40 2023-10-08T09:20:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.329+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1469136108 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcdbc012070 con 0x55c792447e40 2023-10-08T09:20:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.330+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1469136108 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdbc023670 con 0x55c792447e40 2023-10-08T09:20:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.330+0000 7fcdd4deab80 10 client.26005 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.330+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fcdb4009990 con 0x55c792447e40 2023-10-08T09:20:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1469136108 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c792362e50 con 0x55c792447e40 2023-10-08T09:20:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1469136108 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcdbc047070 con 0x55c792447e40 2023-10-08T09:20:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcdd4deab80 10 client.26005 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcdd4deab80 10 client.26005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcdd4deab80 20 client.26005 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c79245d220 0x55c79245f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcdd4deab80 4 client.26005 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.332+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c792393db0 con 0x55c79245d220 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.333+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c79245d220 0x55c79245f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.333+0000 7fcda9ffb700 10 client.26005 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.333+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c79245d220 0x55c79245f640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd4b2ca56fbab10 server_cookie=feb10dadb931d5af in_seq=0 out_seq=0 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1469136108 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c792393db0 con 0x55c79245d220 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcda9ffb700 10 client.26005 handle_command_reply: tid=0 2023-10-08T09:20:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 client.26005 shutdown 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c792447e40 msgr2=0x55c792451820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c792447e40 0x55c792451820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc008fe0 tx=0x7fcdbc017e40 comp rx=0 tx=0).stop 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 shutdown_connections 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c79245d220 0x55c79245f640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c792447e40 0x55c792451820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792458d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcda9ffb700 0 client.26005 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1469136108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c7924487a0 0x55c792451d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 >> 172.21.15.49:0/1469136108 conn(0x55c7923bc200 msgr2=0x55c792442010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 shutdown_connections 2023-10-08T09:20:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1469136108 wait complete. 2023-10-08T09:20:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.334+0000 7fcdd4deab80 20 client.26005 trim_cache size 0 max 16384 2023-10-08T09:20:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 Processor -- start 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 -- start start 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c79245f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792458cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c79244f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c79221b8c0 con 0x55c792447e40 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c792362e50 con 0x55c7924492b0 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c792459e80 con 0x55c7924487a0 2023-10-08T09:20:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c79245f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc2d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c79244f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c79245f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51740/0 (socket says 172.21.15.49:51740) 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc259c700 1 -- 172.21.15.49:0/4127546449 learned_addr learned my addr 172.21.15.49:0/4127546449 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792458cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc2d9d700 1 -- 172.21.15.49:0/4127546449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 msgr2=0x55c792458cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc2d9d700 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792458cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc2d9d700 1 -- 172.21.15.49:0/4127546449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 msgr2=0x55c79245f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc2d9d700 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c79245f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc2d9d700 1 -- 172.21.15.49:0/4127546449 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc407ada0 con 0x55c7924492b0 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.357+0000 7fcdc259c700 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c79245f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.358+0000 7fcdc2d9d700 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c79244f890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4014400 tx=0x7fcdb4014430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5691bfea9b57d8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.358+0000 7fcdb17fa700 1 -- 172.21.15.49:0/4127546449 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4014d00 con 0x55c7924492b0 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.358+0000 7fcdb17fa700 1 -- 172.21.15.49:0/4127546449 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdb4014e60 con 0x55c7924492b0 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.358+0000 7fcdb17fa700 1 -- 172.21.15.49:0/4127546449 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4015850 con 0x55c7924492b0 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- 172.21.15.49:0/4127546449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 msgr2=0x55c79244f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c79244f890 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4014400 tx=0x7fcdb4014430 comp rx=0 tx=0).stop 2023-10-08T09:20:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- 172.21.15.49:0/4127546449 shutdown_connections 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c79244f890 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c79245f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/4127546449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792458cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- 172.21.15.49:0/4127546449 >> 172.21.15.49:0/4127546449 conn(0x55c7923bc200 msgr2=0x55c79243ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- 172.21.15.49:0/4127546449 shutdown_connections 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- 172.21.15.49:0/4127546449 wait complete. 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- start start 2023-10-08T09:20:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 Processor -- start 2023-10-08T09:20:12.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c7924566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792456c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792457c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c792458170 con 0x55c792447e40 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.359+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7924582e0 con 0x55c7924492b0 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdd4deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7924584a0 con 0x55c7924487a0 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdd4deab80 10 client.0 _ll_register_callbacks cb 0x7fcdb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792456c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792456c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38782/0 (socket says 172.21.15.49:38782) 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1140168152 learned_addr learned my addr 172.21.15.49:0/1140168152 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c7924566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc2d9d700 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792457c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 msgr2=0x55c792457c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792457c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1140168152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 msgr2=0x55c7924566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c7924566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 -- 172.21.15.49:0/1140168152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb4009990 con 0x55c7924487a0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdc1d9b700 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792456c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8095f00 tx=0x7fcdb800b2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=428cf064dbc23569 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcda9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1140168152 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8099490 con 0x55c7924487a0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1140168152 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fcdb809d5e0 con 0x55c7924487a0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.360+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdbc01e040 con 0x55c7924487a0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1140168152 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb80a1820 con 0x55c7924487a0 2023-10-08T09:20:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 10 client.26011 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fcdb4009d00 con 0x55c7924487a0 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1140168152 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdbc01e040 con 0x55c7924487a0 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1140168152 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcdb80a1420 con 0x55c7924487a0 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 10 client.26011 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 20 client.26011 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c7924b1ca0 0x55c7924b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 4 client.26011 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c792393ad0 con 0x55c7924b1ca0 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.361+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c7924b1ca0 0x55c7924b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.363+0000 7fcdc259c700 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c7924b1ca0 0x55c7924b40c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d747e3a604eabc server_cookie=f50bfa7776a55a89 in_seq=0 out_seq=0 2023-10-08T09:20:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.364+0000 7fcda9ffb700 10 client.26011 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:12.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:11 smithi049 ceph-mon[99485]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.8 MiB/s wr, 726 op/s 2023-10-08T09:20:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.404+0000 7fcda9ffb700 1 -- 172.21.15.49:0/1140168152 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c792393ad0 con 0x55c7924b1ca0 2023-10-08T09:20:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.404+0000 7fcda9ffb700 10 client.26011 handle_command_reply: tid=0 2023-10-08T09:20:12.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.404+0000 7fcdd4deab80 1 client.26011 shutdown 2023-10-08T09:20:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.416+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 msgr2=0x55c792456c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.416+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792456c20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8095f00 tx=0x7fcdb800b2d0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.425+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 shutdown_connections 2023-10-08T09:20:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.425+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c7924b1ca0 0x55c7924b40c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.425+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c7924492b0 0x55c792457c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.425+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c792447e40 0x55c7924566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.425+0000 7fcdd4deab80 1 --2- 172.21.15.49:0/1140168152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c7924487a0 0x55c792456c20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.425+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 >> 172.21.15.49:0/1140168152 conn(0x55c7923bc200 msgr2=0x55c7924515d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.433+0000 7fcda9ffb700 0 client.26011 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.433+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 shutdown_connections 2023-10-08T09:20:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.434+0000 7fcdd4deab80 1 -- 172.21.15.49:0/1140168152 wait complete. 2023-10-08T09:20:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.436+0000 7fcdd4deab80 20 client.26011 trim_cache size 0 max 16384 2023-10-08T09:20:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:20:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:20:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:20:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:20:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.455+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcdac0a1480 msgr2=0x7fcdac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.455+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcdac0a1480 0x7fcdac0a3940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00ca10 tx=0x7fcdbc01a040 comp rx=0 tx=0).stop 2023-10-08T09:20:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.455+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc4075510 msgr2=0x7fcdc41b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.455+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc4075510 0x7fcdc41b26e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80097d0 tx=0x7fcdb8009ae0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.460+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 shutdown_connections 2023-10-08T09:20:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.460+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcdac0a1480 0x7fcdac0a3940 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.460+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc4077ee0 0x7fcdc41b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.460+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdc40725e0 0x7fcdc41b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.460+0000 7fcdc8c01700 1 --2- 172.21.15.49:0/1022662151 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcdc4075510 0x7fcdc41b26e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.460+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 >> 172.21.15.49:0/1022662151 conn(0x7fcdc406d770 msgr2=0x7fcdc40764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.461+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 shutdown_connections 2023-10-08T09:20:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:12.461+0000 7fcdc8c01700 1 -- 172.21.15.49:0/1022662151 wait complete. 2023-10-08T09:20:13.441 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:20:13.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.726+0000 7f7432907700 1 -- 172.21.15.49:0/3680070592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f742c1077f0 msgr2=0x7f742c109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:13.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.726+0000 7f7432907700 1 --2- 172.21.15.49:0/3680070592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f742c1077f0 0x7f742c109c20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f742400b0a0 tx=0x7f742400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.747+0000 7f7432907700 1 -- 172.21.15.49:0/3680070592 shutdown_connections 2023-10-08T09:20:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.747+0000 7f7432907700 1 --2- 172.21.15.49:0/3680070592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f742c1077f0 0x7f742c109c20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.747+0000 7f7432907700 1 --2- 172.21.15.49:0/3680070592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c10cb30 0x7f742c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.747+0000 7f7432907700 1 --2- 172.21.15.49:0/3680070592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c10a160 0x7f742c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.748 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.747+0000 7f7432907700 1 -- 172.21.15.49:0/3680070592 >> 172.21.15.49:0/3680070592 conn(0x7f742c06d790 msgr2=0x7f742c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:13.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.757+0000 7f7432907700 1 -- 172.21.15.49:0/3680070592 shutdown_connections 2023-10-08T09:20:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.766+0000 7f7432907700 1 -- 172.21.15.49:0/3680070592 wait complete. 2023-10-08T09:20:13.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.766+0000 7f7432907700 1 Processor -- start 2023-10-08T09:20:13.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.771+0000 7f7432907700 1 -- start start 2023-10-08T09:20:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.771+0000 7f7432907700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c1077f0 0x7f742c1a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.771+0000 7f7432907700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c10a160 0x7f742c1a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.771+0000 7f7432907700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f742c10cb30 0x7f742c1acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.771+0000 7f7432907700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1ad1f0 con 0x7f742c1077f0 2023-10-08T09:20:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.771+0000 7f7432907700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1ad330 con 0x7f742c10a160 2023-10-08T09:20:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.771+0000 7f7432907700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1ad4c0 con 0x7f742c10cb30 2023-10-08T09:20:13.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.772+0000 7f742bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c1077f0 0x7f742c1a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.772+0000 7f742bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c1077f0 0x7f742c1a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51756/0 (socket says 172.21.15.49:51756) 2023-10-08T09:20:13.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.772+0000 7f742bfff700 1 -- 172.21.15.49:0/2317971114 learned_addr learned my addr 172.21.15.49:0/2317971114 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.776+0000 7f742bfff700 1 -- 172.21.15.49:0/2317971114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f742c10cb30 msgr2=0x7f742c1acc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:20:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.776+0000 7f742bfff700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f742c10cb30 0x7f742c1acc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.776+0000 7f742bfff700 1 -- 172.21.15.49:0/2317971114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c10a160 msgr2=0x7f742c1a5c50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:20:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.776+0000 7f742bfff700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c10a160 0x7f742c1a5c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.776+0000 7f742bfff700 1 -- 172.21.15.49:0/2317971114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424009d00 con 0x7f742c1077f0 2023-10-08T09:20:13.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.782+0000 7f742bfff700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c1077f0 0x7f742c1a5710 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7424009bd0 tx=0x7f7424005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:13.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.784+0000 7f74297fa700 1 -- 172.21.15.49:0/2317971114 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742400e040 con 0x7f742c1077f0 2023-10-08T09:20:13.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.784+0000 7f74297fa700 1 -- 172.21.15.49:0/2317971114 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7424003e10 con 0x7f742c1077f0 2023-10-08T09:20:13.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.784+0000 7f74297fa700 1 -- 172.21.15.49:0/2317971114 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424005740 con 0x7f742c1077f0 2023-10-08T09:20:13.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.788+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c1ad740 con 0x7f742c1077f0 2023-10-08T09:20:13.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.788+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c1adb10 con 0x7f742c1077f0 2023-10-08T09:20:13.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.793+0000 7f743eaf0b80 1 Processor -- start 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.794+0000 7f743eaf0b80 1 -- start start 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.795+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.795+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.795+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b5800 0x55d4bb5b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.795+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb51c460 con 0x55d4bb5b49c0 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.795+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb50dc20 con 0x55d4bb5b4060 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.795+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb50b540 con 0x55d4bb5b5800 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.799+0000 7f742bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.799+0000 7f742bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51760/0 (socket says 172.21.15.49:51760) 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.799+0000 7f742bfff700 1 -- 172.21.15.49:0/1400992590 learned_addr learned my addr 172.21.15.49:0/1400992590 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.799+0000 7f742b7fe700 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b5800 0x55d4bb5b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.799+0000 7f7430ea4700 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.800+0000 7f742b7fe700 1 -- 172.21.15.49:0/1400992590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 msgr2=0x55d4bb5b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.800+0000 7f742b7fe700 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.800+0000 7f742b7fe700 1 -- 172.21.15.49:0/1400992590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 msgr2=0x55d4bb5b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.800+0000 7f742b7fe700 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.800+0000 7f742b7fe700 1 -- 172.21.15.49:0/1400992590 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4bb331de0 con 0x55d4bb5b5800 2023-10-08T09:20:13.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.802+0000 7f742b7fe700 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b5800 0x55d4bb5b5c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7420009740 tx=0x7f7420009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5c6ceab47627d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.805+0000 7f74127fc700 1 -- 172.21.15.49:0/1400992590 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420011830 con 0x55d4bb5b5800 2023-10-08T09:20:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.805+0000 7f74127fc700 1 -- 172.21.15.49:0/1400992590 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7420011e50 con 0x55d4bb5b5800 2023-10-08T09:20:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.805+0000 7f74127fc700 1 -- 172.21.15.49:0/1400992590 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000f2e0 con 0x55d4bb5b5800 2023-10-08T09:20:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.806+0000 7f743eaf0b80 1 -- 172.21.15.49:0/1400992590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b5800 msgr2=0x55d4bb5b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.806+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b5800 0x55d4bb5b5c90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7420009740 tx=0x7f7420009a50 comp rx=0 tx=0).stop 2023-10-08T09:20:13.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.807+0000 7f743eaf0b80 1 -- 172.21.15.49:0/1400992590 shutdown_connections 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.807+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b5800 0x55d4bb5b5c90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.807+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.807+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/1400992590 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.807+0000 7f743eaf0b80 1 -- 172.21.15.49:0/1400992590 >> 172.21.15.49:0/1400992590 conn(0x55d4bb528260 msgr2=0x55d4bb5aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.812+0000 7f74297fa700 1 -- 172.21.15.49:0/2317971114 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7424012650 con 0x7f742c1077f0 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.813+0000 7f74297fa700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f74140a1650 0x7f74140a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.814+0000 7f74297fa700 1 -- 172.21.15.49:0/2317971114 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f74240ce9a0 con 0x7f742c1077f0 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.815+0000 7f743eaf0b80 1 -- 172.21.15.49:0/1400992590 shutdown_connections 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.815+0000 7f743eaf0b80 1 -- 172.21.15.49:0/1400992590 wait complete. 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.815+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f74140a1650 0x7f74140a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.815+0000 7f743eaf0b80 1 -- start start 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 1 Processor -- start 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f74140a1650 0x7f74140a3b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f74200109b0 tx=0x7f7420009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb506e30 con 0x55d4bb5b5800 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb4a3710 con 0x55d4bb5b4060 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb4a33b0 con 0x55d4bb5b49c0 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f743eaf0b80 10 client.0 _ll_register_callbacks cb 0x55d4bb300800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f742b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f742b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38802/0 (socket says 172.21.15.49:38802) 2023-10-08T09:20:13.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f742b7fe700 1 -- 172.21.15.49:0/3444990615 learned_addr learned my addr 172.21.15.49:0/3444990615 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.816+0000 7f742bfff700 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.817+0000 7f7430ea4700 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.817+0000 7f7430ea4700 1 -- 172.21.15.49:0/3444990615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b49c0 msgr2=0x55d4bb5cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.817+0000 7f7430ea4700 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.817+0000 7f7430ea4700 1 -- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 msgr2=0x55d4bb5cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.817+0000 7f7430ea4700 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.817+0000 7f7430ea4700 1 -- 172.21.15.49:0/3444990615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7420009370 con 0x55d4bb5b5800 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.818+0000 7f7430ea4700 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d6300 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f741c00e280 tx=0x7f741c00e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce85f63e3fc7045 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:13.822 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.818+0000 7f740b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:13.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.819+0000 7f740b7fe700 1 -- 172.21.15.49:0/3444990615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0093a0 con 0x55d4bb5b5800 2023-10-08T09:20:13.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.819+0000 7f740b7fe700 1 -- 172.21.15.49:0/3444990615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f741c00ee20 con 0x55d4bb5b5800 2023-10-08T09:20:13.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.821+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4bb4ceeb0 con 0x55d4bb5b5800 2023-10-08T09:20:13.823 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.822+0000 7f740b7fe700 1 -- 172.21.15.49:0/3444990615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f741c013470 con 0x55d4bb5b5800 2023-10-08T09:20:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 10 client.16362 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:13.824 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d4bb31ac60 con 0x55d4bb5b5800 2023-10-08T09:20:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f740b7fe700 1 -- 172.21.15.49:0/3444990615 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4bb4ceeb0 con 0x55d4bb5b5800 2023-10-08T09:20:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f740b7fe700 1 -- 172.21.15.49:0/3444990615 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f741c011070 con 0x55d4bb5b5800 2023-10-08T09:20:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 10 client.16362 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 20 client.16362 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:13.825 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb5dac10 0x55d4bb5dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 4 client.16362 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:20:13.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.823+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4bb4ffe10 con 0x55d4bb5dac10 2023-10-08T09:20:13.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.825+0000 7f742bfff700 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb5dac10 0x55d4bb5dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.826 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.825+0000 7f740b7fe700 10 client.16362 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.825+0000 7f742bfff700 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb5dac10 0x55d4bb5dd030 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2eefc5e843c138c server_cookie=a89f196e28035df3 in_seq=0 out_seq=0 2023-10-08T09:20:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.826+0000 7f740b7fe700 1 -- 172.21.15.49:0/3444990615 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d4bb4ffe10 con 0x55d4bb5dac10 2023-10-08T09:20:13.827 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.826+0000 7f740b7fe700 10 client.16362 handle_command_reply: tid=0 2023-10-08T09:20:13.828 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.826+0000 7f743eaf0b80 1 client.16362 shutdown 2023-10-08T09:20:13.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b5800 msgr2=0x55d4bb5d6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:13.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d6300 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f741c00e280 tx=0x7f741c00e590 comp rx=0 tx=0).stop 2023-10-08T09:20:13.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 shutdown_connections 2023-10-08T09:20:13.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb5dac10 0x55d4bb5dd030 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d6300 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f740b7fe700 0 client.16362 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b4060 0x55d4bb5cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/3444990615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.827+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 >> 172.21.15.49:0/3444990615 conn(0x55d4bb528260 msgr2=0x55d4bb5ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.828+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 shutdown_connections 2023-10-08T09:20:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.828+0000 7f743eaf0b80 1 -- 172.21.15.49:0/3444990615 wait complete. 2023-10-08T09:20:13.830 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.828+0000 7f743eaf0b80 20 client.16362 trim_cache size 0 max 16384 2023-10-08T09:20:13.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.852+0000 7f743eaf0b80 1 Processor -- start 2023-10-08T09:20:13.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.852+0000 7f743eaf0b80 1 -- start start 2023-10-08T09:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 0x55d4bb5aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb387920 con 0x55d4bb5b49c0 2023-10-08T09:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb5ce940 con 0x55d4bb5b5800 2023-10-08T09:20:13.855 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb5af310 con 0x55d4bb5b4060 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51776/0 (socket says 172.21.15.49:51776) 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 -- 172.21.15.49:0/2875748403 learned_addr learned my addr 172.21.15.49:0/2875748403 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742bfff700 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 -- 172.21.15.49:0/2875748403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 msgr2=0x55d4bb5ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 -- 172.21.15.49:0/2875748403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 msgr2=0x55d4bb5aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f7430ea4700 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 0x55d4bb5aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 0x55d4bb5aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.853+0000 7f742b7fe700 1 -- 172.21.15.49:0/2875748403 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4bb358250 con 0x55d4bb5b49c0 2023-10-08T09:20:13.856 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.854+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d6220 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7420010ae0 tx=0x7f7420020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e221863a858a6a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.854+0000 7f7412ffd700 1 -- 172.21.15.49:0/2875748403 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420011630 con 0x55d4bb5b49c0 2023-10-08T09:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.854+0000 7f7412ffd700 1 -- 172.21.15.49:0/2875748403 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f742001e070 con 0x55d4bb5b49c0 2023-10-08T09:20:13.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.854+0000 7f7412ffd700 1 -- 172.21.15.49:0/2875748403 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420030c40 con 0x55d4bb5b49c0 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.855+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2875748403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 msgr2=0x55d4bb5d6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.855+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d6220 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7420010ae0 tx=0x7f7420020000 comp rx=0 tx=0).stop 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2875748403 shutdown_connections 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 0x55d4bb5aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2875748403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d6220 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2875748403 >> 172.21.15.49:0/2875748403 conn(0x55d4bb528260 msgr2=0x55d4bb5e19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:13.860 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2875748403 shutdown_connections 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2875748403 wait complete. 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 -- start start 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.856+0000 7f743eaf0b80 1 Processor -- start 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f743eaf0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb5d3530 con 0x55d4bb5b49c0 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb5d36a0 con 0x55d4bb5b5800 2023-10-08T09:20:13.861 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f743eaf0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bb62ea50 con 0x55d4bb5b4060 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f743eaf0b80 10 client.0 _ll_register_callbacks cb 0x7f741c0098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5d6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38814/0 (socket says 172.21.15.49:38814) 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 -- 172.21.15.49:0/2816274494 learned_addr learned my addr 172.21.15.49:0/2816274494 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 -- 172.21.15.49:0/2816274494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 msgr2=0x55d4bb5d2ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d2ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 -- 172.21.15.49:0/2816274494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 msgr2=0x55d4bb5d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 -- 172.21.15.49:0/2816274494 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4bb331de0 con 0x55d4bb5b4060 2023-10-08T09:20:13.862 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f742bfff700 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5d6010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7424007bd0 tx=0x7f742400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=251dea33097e751d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.857+0000 7f740b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f740b7fe700 1 -- 172.21.15.49:0/2816274494 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240a3cc0 con 0x55d4bb5b4060 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f740b7fe700 1 -- 172.21.15.49:0/2816274494 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7424012d80 con 0x55d4bb5b4060 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7420011e50 con 0x55d4bb5b4060 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f740b7fe700 1 -- 172.21.15.49:0/2816274494 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74240b4210 con 0x55d4bb5b4060 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 10 client.26026 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d4bb3302b0 con 0x55d4bb5b4060 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f740b7fe700 1 -- 172.21.15.49:0/2816274494 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7420011e50 con 0x55d4bb5b4060 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f740b7fe700 1 -- 172.21.15.49:0/2816274494 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f74240bb0b0 con 0x55d4bb5b4060 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 10 client.26026 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:13.863 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:13.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 20 client.26026 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:13.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb630210 0x55d4bb632630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:13.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 4 client.26026 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:20:13.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.858+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4bb4ffb30 con 0x55d4bb630210 2023-10-08T09:20:13.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.859+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb630210 0x55d4bb632630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:13.864 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.859+0000 7f740b7fe700 10 client.26026 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.859+0000 7f742b7fe700 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb630210 0x55d4bb632630 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da78210e1fa76a47 server_cookie=3dc73e26d066af0 in_seq=0 out_seq=0 2023-10-08T09:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.859+0000 7f740b7fe700 1 -- 172.21.15.49:0/2816274494 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d4bb4ffb30 con 0x55d4bb630210 2023-10-08T09:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.859+0000 7f740b7fe700 10 client.26026 handle_command_reply: tid=0 2023-10-08T09:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.859+0000 7f743eaf0b80 1 client.26026 shutdown 2023-10-08T09:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 msgr2=0x55d4bb5d6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5d6010 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7424007bd0 tx=0x7f742400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.865 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 shutdown_connections 2023-10-08T09:20:13.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d4bb630210 0x55d4bb632630 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d4bb5b4060 0x55d4bb5d6010 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d4bb5b5800 0x55d4bb5d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 --2- 172.21.15.49:0/2816274494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d4bb5b49c0 0x55d4bb5d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f740b7fe700 0 client.26026 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:13.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 >> 172.21.15.49:0/2816274494 conn(0x55d4bb528260 msgr2=0x55d4bb5aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:13.866 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.863+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 shutdown_connections 2023-10-08T09:20:13.869 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f743eaf0b80 1 -- 172.21.15.49:0/2816274494 wait complete. 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f743eaf0b80 20 client.26026 trim_cache size 0 max 16384 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:20:13.870 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:20:13.871 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:20:13.871 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:20:13.871 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:20:13.871 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f74140a1650 msgr2=0x7f74140a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:13.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f74140a1650 0x7f74140a3b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f74200109b0 tx=0x7f7420009250 comp rx=0 tx=0).stop 2023-10-08T09:20:13.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c1077f0 msgr2=0x7f742c1a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:13.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c1077f0 0x7f742c1a5710 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7424009bd0 tx=0x7f7424005b90 comp rx=0 tx=0).stop 2023-10-08T09:20:13.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 shutdown_connections 2023-10-08T09:20:13.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f74140a1650 0x7f74140a3b10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f742c10cb30 0x7f742c1acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.872 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f742c1077f0 0x7f742c1a5710 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 --2- 172.21.15.49:0/2317971114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f742c10a160 0x7f742c1a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:13.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.866+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 >> 172.21.15.49:0/2317971114 conn(0x7f742c06d790 msgr2=0x7f742c10b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:13.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.867+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 shutdown_connections 2023-10-08T09:20:13.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:13.867+0000 7f7432907700 1 -- 172.21.15.49:0/2317971114 wait complete. 2023-10-08T09:20:13.888 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:20:14.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:13 smithi184 ceph-mon[103675]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-08T09:20:14.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:13 smithi132 ceph-mon[103389]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-08T09:20:14.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 290.0 s, 624.4 tps, lat 162.557 ms stddev 168.869 2023-10-08T09:20:14.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:13 smithi049 ceph-mon[99485]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 705 op/s 2023-10-08T09:20:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:15 smithi184 ceph-mon[103675]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.6 MiB/s wr, 627 op/s 2023-10-08T09:20:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:15 smithi132 ceph-mon[103389]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.6 MiB/s wr, 627 op/s 2023-10-08T09:20:16.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:15 smithi049 ceph-mon[99485]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.6 MiB/s wr, 627 op/s 2023-10-08T09:20:17.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:17 smithi184 ceph-mon[103675]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 MiB/s wr, 640 op/s 2023-10-08T09:20:18.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:17 smithi132 ceph-mon[103389]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 MiB/s wr, 640 op/s 2023-10-08T09:20:18.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:17 smithi049 ceph-mon[99485]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 MiB/s wr, 640 op/s 2023-10-08T09:20:19.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 295.0 s, 650.8 tps, lat 152.243 ms stddev 167.098 2023-10-08T09:20:19.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:19 smithi184 ceph-mon[103675]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.3 MiB/s wr, 590 op/s 2023-10-08T09:20:20.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:19 smithi132 ceph-mon[103389]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.3 MiB/s wr, 590 op/s 2023-10-08T09:20:20.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:19 smithi049 ceph-mon[99485]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 9.3 MiB/s wr, 590 op/s 2023-10-08T09:20:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:21 smithi184 ceph-mon[103675]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.5 MiB/s wr, 644 op/s 2023-10-08T09:20:22.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:21 smithi132 ceph-mon[103389]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.5 MiB/s wr, 644 op/s 2023-10-08T09:20:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:21 smithi049 ceph-mon[99485]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.5 MiB/s wr, 644 op/s 2023-10-08T09:20:23.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:23 smithi184 ceph-mon[103675]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 667 op/s 2023-10-08T09:20:24.053 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:23 smithi132 ceph-mon[103389]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 667 op/s 2023-10-08T09:20:24.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:23 smithi049 ceph-mon[99485]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 667 op/s 2023-10-08T09:20:24.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 300.0 s, 672.0 tps, lat 151.242 ms stddev 175.010 2023-10-08T09:20:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.705+0000 7fd10b281700 1 -- 172.21.15.49:0/694560694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1040feff0 msgr2=0x7fd104101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.705+0000 7fd10b281700 1 --2- 172.21.15.49:0/694560694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1040feff0 0x7fd104101420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd0f401a9a0 tx=0x7fd0f401acb0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.707+0000 7fd10b281700 1 -- 172.21.15.49:0/694560694 shutdown_connections 2023-10-08T09:20:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.707+0000 7fd10b281700 1 --2- 172.21.15.49:0/694560694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd104104330 0x7fd1041047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.707+0000 7fd10b281700 1 --2- 172.21.15.49:0/694560694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd1040feff0 0x7fd104101420 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.707+0000 7fd10b281700 1 --2- 172.21.15.49:0/694560694 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd104101960 0x7fd104103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.707+0000 7fd10b281700 1 -- 172.21.15.49:0/694560694 >> 172.21.15.49:0/694560694 conn(0x7fd1040fabf0 msgr2=0x7fd1040fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.707+0000 7fd10b281700 1 -- 172.21.15.49:0/694560694 shutdown_connections 2023-10-08T09:20:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.707+0000 7fd10b281700 1 -- 172.21.15.49:0/694560694 wait complete. 2023-10-08T09:20:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 Processor -- start 2023-10-08T09:20:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 -- start start 2023-10-08T09:20:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1040feff0 0x7fd1041b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd104101960 0x7fd1041b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd104104330 0x7fd1041bdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041be1d0 con 0x7fd104104330 2023-10-08T09:20:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041be340 con 0x7fd1040feff0 2023-10-08T09:20:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.708+0000 7fd10b281700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041be500 con 0x7fd104101960 2023-10-08T09:20:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.710+0000 7fd10881c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd104101960 0x7fd1041b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.710+0000 7fd10881c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd104101960 0x7fd1041b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38830/0 (socket says 172.21.15.49:38830) 2023-10-08T09:20:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.710+0000 7fd10881c700 1 -- 172.21.15.49:0/2471768065 learned_addr learned my addr 172.21.15.49:0/2471768065 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.710+0000 7fd10901d700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1040feff0 0x7fd1041b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10901d700 1 -- 172.21.15.49:0/2471768065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd104101960 msgr2=0x7fd1041b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10901d700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd104101960 0x7fd1041b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10901d700 1 -- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd104104330 msgr2=0x7fd1041bdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10981e700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd104104330 0x7fd1041bdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10901d700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd104104330 0x7fd1041bdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10901d700 1 -- 172.21.15.49:0/2471768065 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f401a600 con 0x7fd1040feff0 2023-10-08T09:20:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10981e700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd104104330 0x7fd1041bdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:20:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd10901d700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1040feff0 0x7fd1041b6d70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4003780 tx=0x7fd0f4005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd0fa7fc700 1 -- 172.21.15.49:0/2471768065 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4004070 con 0x7fd1040feff0 2023-10-08T09:20:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd0fa7fc700 1 -- 172.21.15.49:0/2471768065 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f40041d0 con 0x7fd1040feff0 2023-10-08T09:20:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.711+0000 7fd0fa7fc700 1 -- 172.21.15.49:0/2471768065 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f401f650 con 0x7fd1040feff0 2023-10-08T09:20:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.712+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1041be720 con 0x7fd1040feff0 2023-10-08T09:20:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.712+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1041beaf0 con 0x7fd1040feff0 2023-10-08T09:20:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.718+0000 7fd11746ab80 1 Processor -- start 2023-10-08T09:20:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd11746ab80 1 -- start start 2023-10-08T09:20:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9c27a0 0x55e3eb9c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c32b0 0x55e3eb9c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd11746ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb92a400 con 0x55e3eb9c32b0 2023-10-08T09:20:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd11746ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb91bbc0 con 0x55e3eb9c1e40 2023-10-08T09:20:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd11746ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb9194e0 con 0x55e3eb9c27a0 2023-10-08T09:20:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd10881c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c32b0 0x55e3eb9c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd10981e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd10881c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c32b0 0x55e3eb9c3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51800/0 (socket says 172.21.15.49:51800) 2023-10-08T09:20:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.723+0000 7fd10881c700 1 -- 172.21.15.49:0/530764492 learned_addr learned my addr 172.21.15.49:0/530764492 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.726+0000 7fd0fa7fc700 1 -- 172.21.15.49:0/2471768065 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd0f400f480 con 0x7fd1040feff0 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd10981e700 1 -- 172.21.15.49:0/530764492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9c27a0 msgr2=0x55e3eb9c2cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd10981e700 1 --2- 172.21.15.49:0/530764492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9c27a0 0x55e3eb9c2cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd10981e700 1 -- 172.21.15.49:0/530764492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c32b0 msgr2=0x55e3eb9c3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd10981e700 1 --2- 172.21.15.49:0/530764492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c32b0 0x55e3eb9c3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd10981e700 1 -- 172.21.15.49:0/530764492 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3eb73fd80 con 0x55e3eb9c1e40 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd0fa7fc700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f00a1680 0x7fd0f00a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd0fa7fc700 1 -- 172.21.15.49:0/2471768065 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd0f40ce720 con 0x7fd1040feff0 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd10981e700 1 --2- 172.21.15.49:0/530764492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9c2260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd10000a900 tx=0x7fd10000ac10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bd374f07ffbfbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd0ef7fe700 1 -- 172.21.15.49:0/530764492 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000ca90 con 0x55e3eb9c1e40 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd0ef7fe700 1 -- 172.21.15.49:0/530764492 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd10000ae50 con 0x55e3eb9c1e40 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.727+0000 7fd0ef7fe700 1 -- 172.21.15.49:0/530764492 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100018360 con 0x55e3eb9c1e40 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.728+0000 7fd11746ab80 1 -- 172.21.15.49:0/530764492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 msgr2=0x55e3eb9c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.728+0000 7fd11746ab80 1 --2- 172.21.15.49:0/530764492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9c2260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd10000a900 tx=0x7fd10000ac10 comp rx=0 tx=0).stop 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.728+0000 7fd11746ab80 1 -- 172.21.15.49:0/530764492 shutdown_connections 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.728+0000 7fd11746ab80 1 --2- 172.21.15.49:0/530764492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c32b0 0x55e3eb9c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.728+0000 7fd11746ab80 1 --2- 172.21.15.49:0/530764492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9c2260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.728+0000 7fd11746ab80 1 --2- 172.21.15.49:0/530764492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9c27a0 0x55e3eb9c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.728+0000 7fd11746ab80 1 -- 172.21.15.49:0/530764492 >> 172.21.15.49:0/530764492 conn(0x55e3eb936200 msgr2=0x55e3eb9bc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.730+0000 7fd11746ab80 1 -- 172.21.15.49:0/530764492 shutdown_connections 2023-10-08T09:20:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.730+0000 7fd10901d700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f00a1680 0x7fd0f00a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.730+0000 7fd11746ab80 1 -- 172.21.15.49:0/530764492 wait complete. 2023-10-08T09:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 -- start start 2023-10-08T09:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 Processor -- start 2023-10-08T09:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb914dd0 con 0x55e3eb9dd050 2023-10-08T09:20:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb8b16b0 con 0x55e3eb9c1e40 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb8b1350 con 0x55e3eb9dd4d0 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.731+0000 7fd11746ab80 10 client.0 _ll_register_callbacks cb 0x55e3eb70e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.732+0000 7fd10881c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.732+0000 7fd10881c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51808/0 (socket says 172.21.15.49:51808) 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.732+0000 7fd10881c700 1 -- 172.21.15.49:0/2596987477 learned_addr learned my addr 172.21.15.49:0/2596987477 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.732+0000 7fd10981e700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.732+0000 7fd10981e700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd10881c700 1 -- 172.21.15.49:0/2596987477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 msgr2=0x55e3eb9e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd10881c700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd10881c700 1 -- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 msgr2=0x55e3eb9dcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd10881c700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9dcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd10881c700 1 -- 172.21.15.49:0/2596987477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10000a530 con 0x55e3eb9dd050 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd10981e700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd10881c700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fe0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc007b50 tx=0x7fd0fc007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75de05d12f29a435 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd0ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2596987477 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00e030 con 0x55e3eb9dd050 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2596987477 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd0fc00f070 con 0x55e3eb9dd050 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3eb8dce50 con 0x55e3eb9dd050 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2596987477 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0fc011640 con 0x55e3eb9dd050 2023-10-08T09:20:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd11746ab80 10 client.16377 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.733+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e3eb728c00 con 0x55e3eb9dd050 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2596987477 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3eb8dce50 con 0x55e3eb9dd050 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2596987477 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd0fc014070 con 0x55e3eb9dd050 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd11746ab80 10 client.16377 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd11746ab80 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd11746ab80 20 client.16377 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eb9e9190 0x55e3eb9eb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd11746ab80 4 client.16377 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3eb90ddb0 con 0x55e3eb9e9190 2023-10-08T09:20:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.734+0000 7fd10981e700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eb9e9190 0x55e3eb9eb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.742+0000 7fd0ecff9700 10 client.16377 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.742+0000 7fd10981e700 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eb9e9190 0x55e3eb9eb5b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9116f669b0bd975d server_cookie=ad8eca47bccbce62 in_seq=0 out_seq=0 2023-10-08T09:20:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.742+0000 7fd10901d700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f00a1680 0x7fd0f00a3b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4097cd0 tx=0x7fd0f40038f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.742+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2596987477 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3eb90ddb0 con 0x55e3eb9e9190 2023-10-08T09:20:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.742+0000 7fd0ecff9700 10 client.16377 handle_command_reply: tid=0 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.742+0000 7fd11746ab80 1 client.16377 shutdown 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd050 msgr2=0x55e3eb9e1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fe0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc007b50 tx=0x7fd0fc007e60 comp rx=0 tx=0).stop 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 shutdown_connections 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eb9e9190 0x55e3eb9eb5b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd0ecff9700 0 client.16377 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9dcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2596987477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fe0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 >> 172.21.15.49:0/2596987477 conn(0x55e3eb936200 msgr2=0x55e3eb9bc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 shutdown_connections 2023-10-08T09:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 1 -- 172.21.15.49:0/2596987477 wait complete. 2023-10-08T09:20:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.743+0000 7fd11746ab80 20 client.16377 trim_cache size 0 max 16384 2023-10-08T09:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 Processor -- start 2023-10-08T09:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 -- start start 2023-10-08T09:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb7958c0 con 0x55e3eb9dd4d0 2023-10-08T09:20:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb9b9830 con 0x55e3eb9c1e40 2023-10-08T09:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd11746ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eb9dc680 con 0x55e3eb9dd050 2023-10-08T09:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd10981e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.762+0000 7fd10881c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51820/0 (socket says 172.21.15.49:51820) 2023-10-08T09:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 -- 172.21.15.49:0/2588504397 learned_addr learned my addr 172.21.15.49:0/2588504397 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 -- 172.21.15.49:0/2588504397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd050 msgr2=0x55e3eb9e1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 -- 172.21.15.49:0/2588504397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 msgr2=0x55e3eb9eb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9eb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 -- 172.21.15.49:0/2588504397 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3eb7661f0 con 0x55e3eb9dd4d0 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10881c700 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd10981e700 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9c2220 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd10000c430 tx=0x7fd100013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f56ff5793e4fed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd0effff700 1 -- 172.21.15.49:0/2588504397 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100020040 con 0x55e3eb9dd4d0 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd0effff700 1 -- 172.21.15.49:0/2588504397 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd10002a410 con 0x55e3eb9dd4d0 2023-10-08T09:20:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.763+0000 7fd0effff700 1 -- 172.21.15.49:0/2588504397 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10002ae50 con 0x55e3eb9dd4d0 2023-10-08T09:20:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.764+0000 7fd11746ab80 1 -- 172.21.15.49:0/2588504397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd4d0 msgr2=0x55e3eb9c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.764+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9c2220 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd10000c430 tx=0x7fd100013040 comp rx=0 tx=0).stop 2023-10-08T09:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 -- 172.21.15.49:0/2588504397 shutdown_connections 2023-10-08T09:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9dd4d0 0x55e3eb9c2220 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9eb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2588504397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd050 0x55e3eb9e1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 -- 172.21.15.49:0/2588504397 >> 172.21.15.49:0/2588504397 conn(0x55e3eb936200 msgr2=0x55e3eb9eff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 -- 172.21.15.49:0/2588504397 shutdown_connections 2023-10-08T09:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 -- 172.21.15.49:0/2588504397 wait complete. 2023-10-08T09:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 -- start start 2023-10-08T09:20:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.768+0000 7fd11746ab80 1 Processor -- start 2023-10-08T09:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9f8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9dd050 0x55e3eb9f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd11746ab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eba3b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd11746ab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eba3bc20 con 0x55e3eb9c1e40 2023-10-08T09:20:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd11746ab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eba3bd60 con 0x55e3eb9dd050 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd11746ab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3eba3bea0 con 0x55e3eb9dd4d0 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd11746ab80 10 client.0 _ll_register_callbacks cb 0x7fd0fc007710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51822/0 (socket says 172.21.15.49:51822) 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 -- 172.21.15.49:0/2930914182 learned_addr learned my addr 172.21.15.49:0/2930914182 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eba3b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10881c700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9dd050 0x55e3eb9f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 -- 172.21.15.49:0/2930914182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 msgr2=0x55e3eba3b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eba3b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 -- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9dd050 msgr2=0x55e3eb9f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9dd050 0x55e3eb9f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10981e700 1 -- 172.21.15.49:0/2930914182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3eb728c00 con 0x55e3eb9c1e40 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.769+0000 7fd10881c700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9dd050 0x55e3eb9f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd10981e700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9f8040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd100009000 tx=0x7fd1000116d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6a8c219a859f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd0ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2930914182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100020040 con 0x55e3eb9c1e40 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2930914182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd100011740 con 0x55e3eb9c1e40 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd10000ae50 con 0x55e3eb9c1e40 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2930914182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd100011900 con 0x55e3eb9c1e40 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd11746ab80 10 client.16383 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.770+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fd100023040 con 0x55e3eb9c1e40 2023-10-08T09:20:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2930914182 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd10000ae50 con 0x55e3eb9c1e40 2023-10-08T09:20:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2930914182 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd100003c70 con 0x55e3eb9c1e40 2023-10-08T09:20:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd11746ab80 10 client.16383 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd11746ab80 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd11746ab80 20 client.16383 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eba3d420 0x55e3eba3f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd11746ab80 4 client.16383 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:20:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.771+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3eb90dad0 con 0x55e3eba3d420 2023-10-08T09:20:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.772+0000 7fd10881c700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eba3d420 0x55e3eba3f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.772+0000 7fd0ecff9700 10 client.16383 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.772+0000 7fd10881c700 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eba3d420 0x55e3eba3f840 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c9a923a5dab9b3 server_cookie=fd10d834c585740a in_seq=0 out_seq=0 2023-10-08T09:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd0ecff9700 1 -- 172.21.15.49:0/2930914182 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3eb90dad0 con 0x55e3eba3d420 2023-10-08T09:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd0ecff9700 10 client.16383 handle_command_reply: tid=0 2023-10-08T09:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 client.16383 shutdown 2023-10-08T09:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c1e40 msgr2=0x55e3eb9f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9f8040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd100009000 tx=0x7fd1000116d0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 shutdown_connections 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e3eba3d420 0x55e3eba3f840 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e3eb9dd4d0 0x55e3eba3b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e3eb9c1e40 0x55e3eb9f8040 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 --2- 172.21.15.49:0/2930914182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e3eb9dd050 0x55e3eb9f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 >> 172.21.15.49:0/2930914182 conn(0x55e3eb936200 msgr2=0x55e3eb9e4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd0ecff9700 0 client.16383 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 shutdown_connections 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.788+0000 7fd11746ab80 1 -- 172.21.15.49:0/2930914182 wait complete. 2023-10-08T09:20:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd11746ab80 20 client.16383 trim_cache size 0 max 16384 2023-10-08T09:20:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:20:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:20:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:20:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:20:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f00a1680 msgr2=0x7fd0f00a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f00a1680 0x7fd0f00a3b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4097cd0 tx=0x7fd0f40038f0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1040feff0 msgr2=0x7fd1041b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1040feff0 0x7fd1041b6d70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4003780 tx=0x7fd0f4005ab0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 shutdown_connections 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd0f00a1680 0x7fd0f00a3b40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd104104330 0x7fd1041bdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1040feff0 0x7fd1041b6d70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 --2- 172.21.15.49:0/2471768065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd104101960 0x7fd1041b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.789+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 >> 172.21.15.49:0/2471768065 conn(0x7fd1040fabf0 msgr2=0x7fd1040fafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.790+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 shutdown_connections 2023-10-08T09:20:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:25.790+0000 7fd10b281700 1 -- 172.21.15.49:0/2471768065 wait complete. 2023-10-08T09:20:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:25 smithi184 ceph-mon[103675]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.0 MiB/s wr, 569 op/s 2023-10-08T09:20:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:25 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:20:26.051 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:25 smithi132 ceph-mon[103389]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.0 MiB/s wr, 569 op/s 2023-10-08T09:20:26.052 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:25 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:20:26.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:25 smithi049 ceph-mon[99485]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.0 MiB/s wr, 569 op/s 2023-10-08T09:20:26.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:25 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:20:26.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:26 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:20:26.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:26 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:20:26.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:26 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:20:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:26 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:20:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:26 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:20:27.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:26 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:20:27.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:26 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:20:27.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:26 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:20:27.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:26 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:20:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:27 smithi184 ceph-mon[103675]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.4 MiB/s wr, 647 op/s 2023-10-08T09:20:28.051 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:27 smithi132 ceph-mon[103389]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.4 MiB/s wr, 647 op/s 2023-10-08T09:20:28.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:27 smithi049 ceph-mon[99485]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.4 MiB/s wr, 647 op/s 2023-10-08T09:20:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 305.0 s, 501.8 tps, lat 196.118 ms stddev 209.390 2023-10-08T09:20:29.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:29 smithi184 ceph-mon[103675]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2023-10-08T09:20:30.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:29 smithi132 ceph-mon[103389]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2023-10-08T09:20:30.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:29 smithi049 ceph-mon[99485]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2023-10-08T09:20:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:31 smithi132 ceph-mon[103389]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 644 op/s 2023-10-08T09:20:32.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:31 smithi049 ceph-mon[99485]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 644 op/s 2023-10-08T09:20:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:31 smithi184 ceph-mon[103675]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.3 MiB/s wr, 644 op/s 2023-10-08T09:20:34.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:33 smithi132 ceph-mon[103389]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.5 MiB/s wr, 683 op/s 2023-10-08T09:20:34.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:33 smithi049 ceph-mon[99485]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.5 MiB/s wr, 683 op/s 2023-10-08T09:20:34.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:33 smithi184 ceph-mon[103675]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.5 MiB/s wr, 683 op/s 2023-10-08T09:20:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 310.0 s, 691.6 tps, lat 144.687 ms stddev 162.139 2023-10-08T09:20:36.051 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:35 smithi132 ceph-mon[103389]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 595 op/s 2023-10-08T09:20:36.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:35 smithi049 ceph-mon[99485]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 595 op/s 2023-10-08T09:20:36.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:35 smithi184 ceph-mon[103675]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 595 op/s 2023-10-08T09:20:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:37 smithi132 ceph-mon[103389]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 716 op/s 2023-10-08T09:20:38.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:37 smithi049 ceph-mon[99485]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 716 op/s 2023-10-08T09:20:38.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:37 smithi184 ceph-mon[103675]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 716 op/s 2023-10-08T09:20:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.007+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1686874794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3741084e0 msgr2=0x7fc374071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.007+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1686874794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3741084e0 0x7fc374071900 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc3640098f0 tx=0x7fc364009c00 comp rx=0 tx=0).stop 2023-10-08T09:20:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.011+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1686874794 shutdown_connections 2023-10-08T09:20:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.011+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1686874794 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc3741084e0 0x7fc374071900 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.011+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1686874794 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc37410ec10 0x7fc3741110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.011+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1686874794 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc374071ed0 0x7fc374072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.011+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1686874794 >> 172.21.15.49:0/1686874794 conn(0x7fc37406d770 msgr2=0x7fc37406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.011+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1686874794 shutdown_connections 2023-10-08T09:20:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.011+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1686874794 wait complete. 2023-10-08T09:20:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.012+0000 7fc37b7ad700 1 Processor -- start 2023-10-08T09:20:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.012+0000 7fc37b7ad700 1 -- start start 2023-10-08T09:20:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc37b7ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc374071ed0 0x7fc37419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc37b7ad700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3741084e0 0x7fc37419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc37b7ad700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc37410ec10 0x7fc3741a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc37b7ad700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741a4850 con 0x7fc3741084e0 2023-10-08T09:20:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc37b7ad700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741a49c0 con 0x7fc374071ed0 2023-10-08T09:20:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc37b7ad700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3741a4b80 con 0x7fc37410ec10 2023-10-08T09:20:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc37410ec10 0x7fc3741a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc37410ec10 0x7fc3741a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38874/0 (socket says 172.21.15.49:38874) 2023-10-08T09:20:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 -- 172.21.15.49:0/1142310283 learned_addr learned my addr 172.21.15.49:0/1142310283 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379549700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc374071ed0 0x7fc37419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 -- 172.21.15.49:0/1142310283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc374071ed0 msgr2=0x7fc37419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc374071ed0 0x7fc37419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 -- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3741084e0 msgr2=0x7fc37419d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3741084e0 0x7fc37419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.013+0000 7fc379d4a700 1 -- 172.21.15.49:0/1142310283 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364009520 con 0x7fc37410ec10 2023-10-08T09:20:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.014+0000 7fc379d4a700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc37410ec10 0x7fc3741a4280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00b970 tx=0x7fc36c00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.017+0000 7fc36a7fc700 1 -- 172.21.15.49:0/1142310283 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c00c9b0 con 0x7fc37410ec10 2023-10-08T09:20:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.017+0000 7fc36a7fc700 1 -- 172.21.15.49:0/1142310283 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36c014440 con 0x7fc37410ec10 2023-10-08T09:20:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.017+0000 7fc36a7fc700 1 -- 172.21.15.49:0/1142310283 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0075d0 con 0x7fc37410ec10 2023-10-08T09:20:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.023+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3741a4e60 con 0x7fc37410ec10 2023-10-08T09:20:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.023+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3741a5230 con 0x7fc37410ec10 2023-10-08T09:20:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.028+0000 7fc387996b80 1 Processor -- start 2023-10-08T09:20:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc387996b80 1 -- start start 2023-10-08T09:20:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc387996b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d16ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc387996b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d16d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc387996b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 0x56151d16df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc387996b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d0cc400 con 0x56151d16c650 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc387996b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d0bdbc0 con 0x56151d16cfb0 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc387996b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d0bb4e0 con 0x56151d16dac0 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.034+0000 7fc379549700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d16d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d16d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41456/0 (socket says 172.21.15.49:41456) 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 -- 172.21.15.49:0/1577252486 learned_addr learned my addr 172.21.15.49:0/1577252486 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 -- 172.21.15.49:0/1577252486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 msgr2=0x56151d16df40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:20:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 --2- 172.21.15.49:0/1577252486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 0x56151d16df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 -- 172.21.15.49:0/1577252486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 msgr2=0x56151d16ca70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:20:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 --2- 172.21.15.49:0/1577252486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d16ca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 -- 172.21.15.49:0/1577252486 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56151cee1d80 con 0x56151d16cfb0 2023-10-08T09:20:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc36a7fc700 1 -- 172.21.15.49:0/1142310283 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc36c0145b0 con 0x7fc37410ec10 2023-10-08T09:20:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.035+0000 7fc379549700 1 --2- 172.21.15.49:0/1577252486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d16d4e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc364009810 tx=0x7fc36400bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc82f36a209ebcc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.036+0000 7fc35f7fe700 1 -- 172.21.15.49:0/1577252486 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36401c040 con 0x56151d16cfb0 2023-10-08T09:20:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.036+0000 7fc36a7fc700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc3600a1600 0x7fc3600a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.036+0000 7fc35f7fe700 1 -- 172.21.15.49:0/1577252486 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36400f7e0 con 0x56151d16cfb0 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.036+0000 7fc35f7fe700 1 -- 172.21.15.49:0/1577252486 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640056e0 con 0x56151d16cfb0 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.036+0000 7fc36a7fc700 1 -- 172.21.15.49:0/1142310283 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc36c006e40 con 0x7fc37410ec10 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc378d48700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc3600a1600 0x7fc3600a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 -- 172.21.15.49:0/1577252486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 msgr2=0x56151d16d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 --2- 172.21.15.49:0/1577252486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d16d4e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc364009810 tx=0x7fc36400bef0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 -- 172.21.15.49:0/1577252486 shutdown_connections 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 --2- 172.21.15.49:0/1577252486 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 0x56151d16df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 --2- 172.21.15.49:0/1577252486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d16ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 --2- 172.21.15.49:0/1577252486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d16d4e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 -- 172.21.15.49:0/1577252486 >> 172.21.15.49:0/1577252486 conn(0x56151d0d8200 msgr2=0x56151d15e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc378d48700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc3600a1600 0x7fc3600a3ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc370005950 tx=0x7fc3700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.037+0000 7fc387996b80 1 -- 172.21.15.49:0/1577252486 shutdown_connections 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 -- 172.21.15.49:0/1577252486 wait complete. 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 -- start start 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 Processor -- start 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d165960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 0x56151d16a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d0b6dd0 con 0x56151d16c650 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d0536b0 con 0x56151d16cfb0 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d053350 con 0x56151d16dac0 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.038+0000 7fc387996b80 10 client.0 _ll_register_callbacks cb 0x56151ceb07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d165420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51846/0 (socket says 172.21.15.49:51846) 2023-10-08T09:20:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 -- 172.21.15.49:0/1277637791 learned_addr learned my addr 172.21.15.49:0/1277637791 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc378d48700 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379d4a700 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 0x56151d16a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 -- 172.21.15.49:0/1277637791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 msgr2=0x56151d16a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 0x56151d16a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 -- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 msgr2=0x56151d165960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d165960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 -- 172.21.15.49:0/1277637791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc374111c90 con 0x56151d16c650 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc379549700 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d165420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc364005920 tx=0x7fc36400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed95332e067f6b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc35cff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc35cff9700 1 -- 172.21.15.49:0/1277637791 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36401c070 con 0x56151d16c650 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc35cff9700 1 -- 172.21.15.49:0/1277637791 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc36400ba90 con 0x56151d16c650 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.039+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56151d07ee50 con 0x56151d16c650 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc35cff9700 1 -- 172.21.15.49:0/1277637791 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3640046d0 con 0x56151d16c650 2023-10-08T09:20:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 10 client.16386 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56151cecac00 con 0x56151d16c650 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc35cff9700 1 -- 172.21.15.49:0/1277637791 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56151d07ee50 con 0x56151d16c650 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc35cff9700 1 -- 172.21.15.49:0/1277637791 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc364015020 con 0x56151d16c650 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 20 client.16386 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d18a180 0x56151d18c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 4 client.16386 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56151d0afdb0 con 0x56151d18a180 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.040+0000 7fc378d48700 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d18a180 0x56151d18c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.041+0000 7fc35cff9700 10 client.16386 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.041+0000 7fc378d48700 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d18a180 0x56151d18c5a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3339b186d4861dde server_cookie=5597dd91f4492099 in_seq=0 out_seq=0 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc35cff9700 1 -- 172.21.15.49:0/1277637791 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56151d0afdb0 con 0x56151d18a180 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc35cff9700 10 client.16386 handle_command_reply: tid=0 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 client.16386 shutdown 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 msgr2=0x56151d165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d165420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc364005920 tx=0x7fc36400b790 comp rx=0 tx=0).stop 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 shutdown_connections 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d18a180 0x56151d18c5a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16dac0 0x56151d16a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc35cff9700 0 client.16386 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16c650 0x56151d165420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 --2- 172.21.15.49:0/1277637791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16cfb0 0x56151d165960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 >> 172.21.15.49:0/1277637791 conn(0x56151d0d8200 msgr2=0x56151d15e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 shutdown_connections 2023-10-08T09:20:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 1 -- 172.21.15.49:0/1277637791 wait complete. 2023-10-08T09:20:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.042+0000 7fc387996b80 20 client.16386 trim_cache size 0 max 16384 2023-10-08T09:20:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 Processor -- start 2023-10-08T09:20:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 -- start start 2023-10-08T09:20:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d16b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d16a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d16dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151cf378c0 con 0x56151d16dac0 2023-10-08T09:20:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d1592b0 con 0x56151d16c650 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc387996b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d16bb10 con 0x56151d16cfb0 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc379d4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d16dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.063+0000 7fc379d4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d16dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51856/0 (socket says 172.21.15.49:51856) 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379549700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d16b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379d4a700 1 -- 172.21.15.49:0/2493340835 learned_addr learned my addr 172.21.15.49:0/2493340835 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc378d48700 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d16a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379d4a700 1 -- 172.21.15.49:0/2493340835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 msgr2=0x56151d16a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379d4a700 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d16a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379d4a700 1 -- 172.21.15.49:0/2493340835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 msgr2=0x56151d16b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379d4a700 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d16b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379d4a700 1 -- 172.21.15.49:0/2493340835 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56151cf081f0 con 0x56151d16dac0 2023-10-08T09:20:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc379d4a700 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d16dea0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0c9c10 tx=0x7fc36c095040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed521da39f27031 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc35ffff700 1 -- 172.21.15.49:0/2493340835 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c09d580 con 0x56151d16dac0 2023-10-08T09:20:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc35ffff700 1 -- 172.21.15.49:0/2493340835 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36c0a17c0 con 0x56151d16dac0 2023-10-08T09:20:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.064+0000 7fc35ffff700 1 -- 172.21.15.49:0/2493340835 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c099690 con 0x56151d16dac0 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 -- 172.21.15.49:0/2493340835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 msgr2=0x56151d16dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d16dea0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0c9c10 tx=0x7fc36c095040 comp rx=0 tx=0).stop 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 -- 172.21.15.49:0/2493340835 shutdown_connections 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d16dea0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d16b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 --2- 172.21.15.49:0/2493340835 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d16a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 -- 172.21.15.49:0/2493340835 >> 172.21.15.49:0/2493340835 conn(0x56151d0d8200 msgr2=0x56151d190f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 -- 172.21.15.49:0/2493340835 shutdown_connections 2023-10-08T09:20:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 -- 172.21.15.49:0/2493340835 wait complete. 2023-10-08T09:20:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.065+0000 7fc387996b80 1 -- start start 2023-10-08T09:20:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 1 Processor -- start 2023-10-08T09:20:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d1da6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d1dcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d1dd3a0 con 0x56151d16dac0 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d1dd510 con 0x56151d16c650 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d1dd6d0 con 0x56151d16cfb0 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc387996b80 10 client.0 _ll_register_callbacks cb 0x7fc3700091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc379d4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d1dcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc379d4a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d1dcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51860/0 (socket says 172.21.15.49:51860) 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc379d4a700 1 -- 172.21.15.49:0/985334862 learned_addr learned my addr 172.21.15.49:0/985334862 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc378d48700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d1da6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.066+0000 7fc379549700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc379d4a700 1 -- 172.21.15.49:0/985334862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 msgr2=0x56151d1da6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc379d4a700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d1da6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc379d4a700 1 -- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 msgr2=0x56151d199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc379d4a700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc379d4a700 1 -- 172.21.15.49:0/985334862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3741a5230 con 0x56151d16dac0 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc379549700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc379d4a700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d1dcc90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0993c0 tx=0x7fc36c098c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18677bf78259d6ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc35cff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc35cff9700 1 -- 172.21.15.49:0/985334862 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c099200 con 0x56151d16dac0 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc35cff9700 1 -- 172.21.15.49:0/985334862 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc36c0148d0 con 0x56151d16dac0 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc36400f7e0 con 0x56151d16dac0 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc35cff9700 1 -- 172.21.15.49:0/985334862 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc36c09d580 con 0x56151d16dac0 2023-10-08T09:20:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc387996b80 10 client.16392 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56151cee0250 con 0x56151d16dac0 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.067+0000 7fc35cff9700 1 -- 172.21.15.49:0/985334862 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc36c07d020 con 0x56151d16dac0 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc35cff9700 1 -- 172.21.15.49:0/985334862 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc36c0ac070 con 0x56151d16dac0 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc387996b80 10 client.16392 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc387996b80 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc387996b80 20 client.16392 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc387996b80 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d1def50 0x56151d1e1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc387996b80 4 client.16392 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56151d0afad0 con 0x56151d1def50 2023-10-08T09:20:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.068+0000 7fc379549700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d1def50 0x56151d1e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.086+0000 7fc379549700 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d1def50 0x56151d1e1370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b50032f76c2e9f5 server_cookie=3c12fb55941a69c8 in_seq=0 out_seq=0 2023-10-08T09:20:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.096+0000 7fc35cff9700 10 client.16392 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc35cff9700 1 -- 172.21.15.49:0/985334862 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56151d0afad0 con 0x56151d1def50 2023-10-08T09:20:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc35cff9700 10 client.16392 handle_command_reply: tid=0 2023-10-08T09:20:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 client.16392 shutdown 2023-10-08T09:20:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 msgr2=0x56151d1dcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d1dcc90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0993c0 tx=0x7fc36c098c90 comp rx=0 tx=0).stop 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 shutdown_connections 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56151d1def50 0x56151d1e1370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56151d16dac0 0x56151d1dcc90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc35cff9700 0 client.16392 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56151d16c650 0x56151d199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 --2- 172.21.15.49:0/985334862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56151d16cfb0 0x56151d1da6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.152+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 >> 172.21.15.49:0/985334862 conn(0x56151d0d8200 msgr2=0x56151d174010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 shutdown_connections 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc387996b80 1 -- 172.21.15.49:0/985334862 wait complete. 2023-10-08T09:20:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc387996b80 20 client.16392 trim_cache size 0 max 16384 2023-10-08T09:20:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:20:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:20:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:20:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc3600a1600 msgr2=0x7fc3600a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc3600a1600 0x7fc3600a3ac0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc370005950 tx=0x7fc3700058e0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc37410ec10 msgr2=0x7fc3741a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc37410ec10 0x7fc3741a4280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc36c00b970 tx=0x7fc36c00bd30 comp rx=0 tx=0).stop 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 shutdown_connections 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc3600a1600 0x7fc3600a3ac0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc37410ec10 0x7fc3741a4280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc374071ed0 0x7fc37419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 --2- 172.21.15.49:0/1142310283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3741084e0 0x7fc37419d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 >> 172.21.15.49:0/1142310283 conn(0x7fc37406d770 msgr2=0x7fc37410ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 shutdown_connections 2023-10-08T09:20:39.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:39.153+0000 7fc37b7ad700 1 -- 172.21.15.49:0/1142310283 wait complete. 2023-10-08T09:20:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 315.0 s, 656.1 tps, lat 153.797 ms stddev 166.269 2023-10-08T09:20:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:39 smithi132 ceph-mon[103389]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.5 MiB/s wr, 621 op/s 2023-10-08T09:20:40.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:39 smithi049 ceph-mon[99485]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.5 MiB/s wr, 621 op/s 2023-10-08T09:20:40.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:39 smithi184 ceph-mon[103675]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 8.5 MiB/s wr, 621 op/s 2023-10-08T09:20:42.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:41 smithi049 ceph-mon[99485]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 675 op/s 2023-10-08T09:20:42.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:41 smithi184 ceph-mon[103675]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 675 op/s 2023-10-08T09:20:42.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:41 smithi132 ceph-mon[103389]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.0 MiB/s wr, 675 op/s 2023-10-08T09:20:43.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:20:44.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:43 smithi049 ceph-mon[99485]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 MiB/s wr, 686 op/s 2023-10-08T09:20:44.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.213+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/920837995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc10a160 msgr2=0x7ff2cc10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.213+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/920837995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc10a160 0x7ff2cc10c5f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc01a9a0 tx=0x7ff2bc01acb0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.224+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/920837995 shutdown_connections 2023-10-08T09:20:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.224+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/920837995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2cc10cb30 0x7ff2cc10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.224+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/920837995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2cc1077f0 0x7ff2cc109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.224+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/920837995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc10a160 0x7ff2cc10c5f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.224+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/920837995 >> 172.21.15.49:0/920837995 conn(0x7ff2cc06d790 msgr2=0x7ff2cc06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:44.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.229+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/920837995 shutdown_connections 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.234+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/920837995 wait complete. 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.234+0000 7ff2d2cf6700 1 Processor -- start 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d2cf6700 1 -- start start 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d2cf6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc1077f0 0x7ff2cc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d2cf6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2cc10a160 0x7ff2cc19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d2cf6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2cc10cb30 0x7ff2cc1a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d2cf6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1a4990 con 0x7ff2cc10a160 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d2cf6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1a4ad0 con 0x7ff2cc1077f0 2023-10-08T09:20:44.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d2cf6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1a4c60 con 0x7ff2cc10cb30 2023-10-08T09:20:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc1077f0 0x7ff2cc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc1077f0 0x7ff2cc19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41488/0 (socket says 172.21.15.49:41488) 2023-10-08T09:20:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2d0a92700 1 -- 172.21.15.49:0/264949630 learned_addr learned my addr 172.21.15.49:0/264949630 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:44.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.235+0000 7ff2cbfff700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2cc10a160 0x7ff2cc19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2d0a92700 1 -- 172.21.15.49:0/264949630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2cc10cb30 msgr2=0x7ff2cc1a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2cc10cb30 0x7ff2cc1a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2d0a92700 1 -- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2cc10a160 msgr2=0x7ff2cc19d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2cc10a160 0x7ff2cc19d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2d0a92700 1 -- 172.21.15.49:0/264949630 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc01a600 con 0x7ff2cc1077f0 2023-10-08T09:20:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc1077f0 0x7ff2cc19ce60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000b530 tx=0x7ff2c000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:44.238 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2c9ffb700 1 -- 172.21.15.49:0/264949630 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0011830 con 0x7ff2cc1077f0 2023-10-08T09:20:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc1a4f40 con 0x7ff2cc1077f0 2023-10-08T09:20:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2c9ffb700 1 -- 172.21.15.49:0/264949630 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c0011e50 con 0x7ff2cc1077f0 2023-10-08T09:20:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.236+0000 7ff2c9ffb700 1 -- 172.21.15.49:0/264949630 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c000f2e0 con 0x7ff2cc1077f0 2023-10-08T09:20:44.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.237+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc1a5370 con 0x7ff2cc1077f0 2023-10-08T09:20:44.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:43 smithi184 ceph-mon[103675]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 MiB/s wr, 686 op/s 2023-10-08T09:20:44.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.244+0000 7ff2deedfb80 1 Processor -- start 2023-10-08T09:20:44.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.244+0000 7ff2c9ffb700 1 -- 172.21.15.49:0/264949630 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff2c0010380 con 0x7ff2cc1077f0 2023-10-08T09:20:44.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.244+0000 7ff2c9ffb700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2b40a1500 0x7ff2b40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.244+0000 7ff2c9ffb700 1 -- 172.21.15.49:0/264949630 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff2c00ccbf0 con 0x7ff2cc1077f0 2023-10-08T09:20:44.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.244+0000 7ff2deedfb80 1 -- start start 2023-10-08T09:20:44.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.245+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de2060 0x55ffd5de2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.254 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.245+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5de3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.245+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 0x55ffd5de3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.245+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5d4a460 con 0x55ffd5de29c0 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.245+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5d3bc20 con 0x55ffd5de3800 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.245+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5d39540 con 0x55ffd5de2060 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.247+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de2060 0x55ffd5de2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.247+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de2060 0x55ffd5de2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38916/0 (socket says 172.21.15.49:38916) 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.247+0000 7ff2d0a92700 1 -- 172.21.15.49:0/476195222 learned_addr learned my addr 172.21.15.49:0/476195222 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:44.255 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.252+0000 7ff2d1293700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2b40a1500 0x7ff2b40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.253+0000 7ff2cbfff700 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5de3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.253+0000 7ff2cbfff700 1 -- 172.21.15.49:0/476195222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de2060 msgr2=0x55ffd5de2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.253+0000 7ff2cbfff700 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de2060 0x55ffd5de2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.253+0000 7ff2cbfff700 1 -- 172.21.15.49:0/476195222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 msgr2=0x55ffd5de3c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:20:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.253+0000 7ff2cbfff700 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 0x55ffd5de3c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.256 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.253+0000 7ff2cbfff700 1 -- 172.21.15.49:0/476195222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd5b5fde0 con 0x55ffd5de29c0 2023-10-08T09:20:44.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.254+0000 7ff2cbfff700 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5de3270 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc01a8f0 tx=0x7ff2bc01aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901e1f5192be216e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:44.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.254+0000 7ff2b2ffd700 1 -- 172.21.15.49:0/476195222 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc02d050 con 0x55ffd5de29c0 2023-10-08T09:20:44.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.255+0000 7ff2b2ffd700 1 -- 172.21.15.49:0/476195222 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2bc02e040 con 0x55ffd5de29c0 2023-10-08T09:20:44.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.255+0000 7ff2b2ffd700 1 -- 172.21.15.49:0/476195222 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc004260 con 0x55ffd5de29c0 2023-10-08T09:20:44.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.255+0000 7ff2deedfb80 1 -- 172.21.15.49:0/476195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 msgr2=0x55ffd5de3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.255+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5de3270 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc01a8f0 tx=0x7ff2bc01aef0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.256+0000 7ff2deedfb80 1 -- 172.21.15.49:0/476195222 shutdown_connections 2023-10-08T09:20:44.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.256+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de2060 0x55ffd5de2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.256+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 0x55ffd5de3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.256+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/476195222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5de3270 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.256+0000 7ff2deedfb80 1 -- 172.21.15.49:0/476195222 >> 172.21.15.49:0/476195222 conn(0x55ffd5d56260 msgr2=0x55ffd5ddce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:44.259 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.257+0000 7ff2d1293700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2b40a1500 0x7ff2b40a39c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400ac50 tx=0x7ff2c4009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.265+0000 7ff2deedfb80 1 -- 172.21.15.49:0/476195222 shutdown_connections 2023-10-08T09:20:44.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.274+0000 7ff2deedfb80 1 -- 172.21.15.49:0/476195222 wait complete. 2023-10-08T09:20:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.274+0000 7ff2deedfb80 1 -- start start 2023-10-08T09:20:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.274+0000 7ff2deedfb80 1 Processor -- start 2023-10-08T09:20:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.275+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.275+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 0x55ffd5e0e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.275+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.275+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5d34e30 con 0x55ffd5e13480 2023-10-08T09:20:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.275+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5cd1710 con 0x55ffd5de3800 2023-10-08T09:20:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.275+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5cd13b0 con 0x55ffd5de29c0 2023-10-08T09:20:44.277 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.275+0000 7ff2deedfb80 10 client.0 _ll_register_callbacks cb 0x55ffd5b2e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51890/0 (socket says 172.21.15.49:51890) 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 -- 172.21.15.49:0/3464778998 learned_addr learned my addr 172.21.15.49:0/3464778998 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 -- 172.21.15.49:0/3464778998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 msgr2=0x55ffd5e0df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 -- 172.21.15.49:0/3464778998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 msgr2=0x55ffd5e0e450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 0x55ffd5e0e450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 -- 172.21.15.49:0/3464778998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc10dae0 con 0x55ffd5e13480 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2d1293700 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400d9a0 tx=0x7ff2c4004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe6df9ebd2c17df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:44.278 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.276+0000 7ff2abfff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:44.279 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2abfff700 1 -- 172.21.15.49:0/3464778998 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4003ca0 con 0x55ffd5e13480 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2abfff700 1 -- 172.21.15.49:0/3464778998 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff2c401e070 con 0x55ffd5e13480 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffd5cfceb0 con 0x55ffd5e13480 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2abfff700 1 -- 172.21.15.49:0/3464778998 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2c4003e00 con 0x55ffd5e13480 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2deedfb80 10 client.16401 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ffd5b48c60 con 0x55ffd5e13480 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2abfff700 1 -- 172.21.15.49:0/3464778998 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffd5cfceb0 con 0x55ffd5e13480 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.277+0000 7ff2abfff700 1 -- 172.21.15.49:0/3464778998 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff2c4004750 con 0x55ffd5e13480 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2deedfb80 10 client.16401 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2deedfb80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:44.280 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2deedfb80 20 client.16401 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e1a1e0 0x55ffd5e1c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2deedfb80 4 client.16401 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:20:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffd5d2de10 con 0x55ffd5e1a1e0 2023-10-08T09:20:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e1a1e0 0x55ffd5e1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2abfff700 10 client.16401 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.278+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e1a1e0 0x55ffd5e1c680 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d33c883d0bcd92 server_cookie=e58af17bf3bfff6f in_seq=0 out_seq=0 2023-10-08T09:20:44.281 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.280+0000 7ff2abfff700 1 -- 172.21.15.49:0/3464778998 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ffd5d2de10 con 0x55ffd5e1a1e0 2023-10-08T09:20:44.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.280+0000 7ff2abfff700 10 client.16401 handle_command_reply: tid=0 2023-10-08T09:20:44.282 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.280+0000 7ff2deedfb80 1 client.16401 shutdown 2023-10-08T09:20:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.280+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 msgr2=0x55ffd5e15870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.283 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.280+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400d9a0 tx=0x7ff2c4004bd0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.282+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 shutdown_connections 2023-10-08T09:20:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.282+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e1a1e0 0x55ffd5e1c680 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.282+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.282+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.282+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3464778998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de3800 0x55ffd5e0e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.282+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 >> 172.21.15.49:0/3464778998 conn(0x55ffd5d56260 msgr2=0x55ffd5ddc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:44.284 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.282+0000 7ff2abfff700 0 client.16401 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:44.291 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.284+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 shutdown_connections 2023-10-08T09:20:44.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:43 smithi132 ceph-mon[103389]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.9 MiB/s wr, 686 op/s 2023-10-08T09:20:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.292+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3464778998 wait complete. 2023-10-08T09:20:44.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.300+0000 7ff2deedfb80 20 client.16401 trim_cache size 0 max 16384 2023-10-08T09:20:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 320.0 s, 492.5 tps, lat 187.526 ms stddev 222.272 2023-10-08T09:20:44.346 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 Processor -- start 2023-10-08T09:20:44.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 -- start start 2023-10-08T09:20:44.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de9ea0 0x55ffd5e29ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5bb5920 con 0x55ffd5e13480 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5de2e90 con 0x55ffd5de9ea0 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.345+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5dde990 con 0x55ffd5de29c0 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.346+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.346+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38928/0 (socket says 172.21.15.49:38928) 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.346+0000 7ff2d0a92700 1 -- 172.21.15.49:0/1039141552 learned_addr learned my addr 172.21.15.49:0/1039141552 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.347+0000 7ff2d0a92700 1 -- 172.21.15.49:0/1039141552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de9ea0 msgr2=0x55ffd5e29ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.347+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/1039141552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de9ea0 0x55ffd5e29ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.354 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.347+0000 7ff2d0a92700 1 -- 172.21.15.49:0/1039141552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 msgr2=0x55ffd5e15870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:20:44.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.347+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/1039141552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.347+0000 7ff2d0a92700 1 -- 172.21.15.49:0/1039141552 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd5b86250 con 0x55ffd5de29c0 2023-10-08T09:20:44.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.353+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/1039141552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0ca90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0099880 tx=0x7ff2c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983afc91b04e1fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:44.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.354+0000 7ff2b37fe700 1 -- 172.21.15.49:0/1039141552 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0099c00 con 0x55ffd5de29c0 2023-10-08T09:20:44.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.354+0000 7ff2b37fe700 1 -- 172.21.15.49:0/1039141552 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2c009dcd0 con 0x55ffd5de29c0 2023-10-08T09:20:44.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.354+0000 7ff2b37fe700 1 -- 172.21.15.49:0/1039141552 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00b2260 con 0x55ffd5de29c0 2023-10-08T09:20:44.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.355+0000 7ff2deedfb80 1 -- 172.21.15.49:0/1039141552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 msgr2=0x55ffd5e0ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.355+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/1039141552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0ca90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0099880 tx=0x7ff2c0004750 comp rx=0 tx=0).stop 2023-10-08T09:20:44.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.355+0000 7ff2deedfb80 1 -- 172.21.15.49:0/1039141552 shutdown_connections 2023-10-08T09:20:44.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.355+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/1039141552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de29c0 0x55ffd5e0ca90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.355+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/1039141552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5de9ea0 0x55ffd5e29ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.355+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/1039141552 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5e13480 0x55ffd5e15870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.357 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.355+0000 7ff2deedfb80 1 -- 172.21.15.49:0/1039141552 >> 172.21.15.49:0/1039141552 conn(0x55ffd5d56260 msgr2=0x55ffd5e21020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:44.364 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.359+0000 7ff2deedfb80 1 -- 172.21.15.49:0/1039141552 shutdown_connections 2023-10-08T09:20:44.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.368+0000 7ff2deedfb80 1 -- 172.21.15.49:0/1039141552 wait complete. 2023-10-08T09:20:44.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.368+0000 7ff2deedfb80 1 -- start start 2023-10-08T09:20:44.370 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.368+0000 7ff2deedfb80 1 Processor -- start 2023-10-08T09:20:44.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5e689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de9ea0 0x55ffd5e6af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2deedfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5e13480 0x55ffd5e6d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5e6db80 con 0x55ffd5de29c0 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5e6dcf0 con 0x55ffd5e13480 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2deedfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5e6deb0 con 0x55ffd5de9ea0 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2deedfb80 10 client.0 _ll_register_callbacks cb 0x7ff2c400cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5e689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5e689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51898/0 (socket says 172.21.15.49:51898) 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 -- 172.21.15.49:0/3493381165 learned_addr learned my addr 172.21.15.49:0/3493381165 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:44.375 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 -- 172.21.15.49:0/3493381165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de9ea0 msgr2=0x55ffd5e6af10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de9ea0 0x55ffd5e6af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 -- 172.21.15.49:0/3493381165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5e13480 msgr2=0x55ffd5e6d460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5e13480 0x55ffd5e6d460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.369+0000 7ff2d0a92700 1 -- 172.21.15.49:0/3493381165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc1a5370 con 0x55ffd5de29c0 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.370+0000 7ff2d0a92700 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5e689c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000b500 tx=0x7ff2c0099ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5240367a2271883b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.372+0000 7ff2abfff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.372+0000 7ff2abfff700 1 -- 172.21.15.49:0/3493381165 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00b2460 con 0x55ffd5de29c0 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.372+0000 7ff2abfff700 1 -- 172.21.15.49:0/3493381165 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff2c0099de0 con 0x55ffd5de29c0 2023-10-08T09:20:44.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.372+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2bc02e040 con 0x55ffd5de29c0 2023-10-08T09:20:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.380+0000 7ff2abfff700 1 -- 172.21.15.49:0/3493381165 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2c0099fa0 con 0x55ffd5de29c0 2023-10-08T09:20:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.387+0000 7ff2deedfb80 10 client.16404 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.387+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ffd5b5e2b0 con 0x55ffd5de29c0 2023-10-08T09:20:44.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2abfff700 1 -- 172.21.15.49:0/3493381165 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2bc02e040 con 0x55ffd5de29c0 2023-10-08T09:20:44.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2abfff700 1 -- 172.21.15.49:0/3493381165 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff2c0065180 con 0x55ffd5de29c0 2023-10-08T09:20:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2deedfb80 10 client.16404 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2deedfb80 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2deedfb80 20 client.16404 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e6f3a0 0x55ffd5e71790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2deedfb80 4 client.16404 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:20:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.388+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ffd5d2db30 con 0x55ffd5e6f3a0 2023-10-08T09:20:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.389+0000 7ff2cbfff700 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e6f3a0 0x55ffd5e71790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:44.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.392+0000 7ff2cbfff700 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e6f3a0 0x55ffd5e71790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593e10388e3efbf8 server_cookie=ce0dd07ea1b263ea in_seq=0 out_seq=0 2023-10-08T09:20:44.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.394+0000 7ff2abfff700 10 client.16404 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:44.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.394+0000 7ff2abfff700 1 -- 172.21.15.49:0/3493381165 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ffd5d2db30 con 0x55ffd5e6f3a0 2023-10-08T09:20:44.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.394+0000 7ff2abfff700 10 client.16404 handle_command_reply: tid=0 2023-10-08T09:20:44.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.394+0000 7ff2deedfb80 1 client.16404 shutdown 2023-10-08T09:20:44.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.395+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 msgr2=0x55ffd5e689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.395+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5e689c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000b500 tx=0x7ff2c0099ce0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.401+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 shutdown_connections 2023-10-08T09:20:44.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.401+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ffd5e6f3a0 0x55ffd5e71790 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.401+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ffd5e13480 0x55ffd5e6d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.401+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ffd5de29c0 0x55ffd5e689c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.401+0000 7ff2deedfb80 1 --2- 172.21.15.49:0/3493381165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ffd5de9ea0 0x55ffd5e6af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.401+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 >> 172.21.15.49:0/3493381165 conn(0x55ffd5d56260 msgr2=0x55ffd5dd8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:44.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.401+0000 7ff2abfff700 0 client.16404 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:44.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.408+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 shutdown_connections 2023-10-08T09:20:44.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.411+0000 7ff2deedfb80 1 -- 172.21.15.49:0/3493381165 wait complete. 2023-10-08T09:20:44.423 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.421+0000 7ff2deedfb80 20 client.16404 trim_cache size 0 max 16384 2023-10-08T09:20:44.431 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:20:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:20:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:20:44.431 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:20:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:20:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:20:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:20:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:20:44.432 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:20:44.432 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:20:44.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.447+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2b40a1500 msgr2=0x7ff2b40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.447+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2b40a1500 0x7ff2b40a39c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400ac50 tx=0x7ff2c4009250 comp rx=0 tx=0).stop 2023-10-08T09:20:44.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.447+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc1077f0 msgr2=0x7ff2cc19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:44.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.447+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc1077f0 0x7ff2cc19ce60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000b530 tx=0x7ff2c000b840 comp rx=0 tx=0).stop 2023-10-08T09:20:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.455+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 shutdown_connections 2023-10-08T09:20:44.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.455+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff2b40a1500 0x7ff2b40a39c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.455+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff2cc10cb30 0x7ff2cc1a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.455+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2cc1077f0 0x7ff2cc19ce60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.455+0000 7ff2d2cf6700 1 --2- 172.21.15.49:0/264949630 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2cc10a160 0x7ff2cc19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:44.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.455+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 >> 172.21.15.49:0/264949630 conn(0x7ff2cc06d790 msgr2=0x7ff2cc10a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.456+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 shutdown_connections 2023-10-08T09:20:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:20:44.458+0000 7ff2d2cf6700 1 -- 172.21.15.49:0/264949630 wait complete. 2023-10-08T09:20:44.485 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:20:46.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:45 smithi049 ceph-mon[99485]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.2 MiB/s wr, 552 op/s 2023-10-08T09:20:46.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:45 smithi184 ceph-mon[103675]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.2 MiB/s wr, 552 op/s 2023-10-08T09:20:46.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:45 smithi132 ceph-mon[103389]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 7.2 MiB/s wr, 552 op/s 2023-10-08T09:20:48.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:47 smithi049 ceph-mon[99485]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-10-08T09:20:48.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:47 smithi184 ceph-mon[103675]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-10-08T09:20:48.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:47 smithi132 ceph-mon[103389]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-10-08T09:20:49.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 325.0 s, 737.4 tps, lat 145.028 ms stddev 204.584 2023-10-08T09:20:50.250 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:49 smithi184 ceph-mon[103675]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-10-08T09:20:50.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:49 smithi132 ceph-mon[103389]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-10-08T09:20:50.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:49 smithi049 ceph-mon[99485]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-10-08T09:20:52.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:51 smithi184 ceph-mon[103675]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-10-08T09:20:52.301 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:51 smithi132 ceph-mon[103389]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-10-08T09:20:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.357+0000 7f4760f77700 1 -- 172.21.15.49:0/197204246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f475c10cb30 msgr2=0x7f475c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.357+0000 7f4760f77700 1 --2- 172.21.15.49:0/197204246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f475c10cb30 0x7f475c10cfb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f474c01a9a0 tx=0x7f474c01acb0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.358+0000 7f4760f77700 1 -- 172.21.15.49:0/197204246 shutdown_connections 2023-10-08T09:20:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.358+0000 7f4760f77700 1 --2- 172.21.15.49:0/197204246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f475c10cb30 0x7f475c10cfb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.358+0000 7f4760f77700 1 --2- 172.21.15.49:0/197204246 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f475c1077f0 0x7f475c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.358+0000 7f4760f77700 1 --2- 172.21.15.49:0/197204246 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f475c10a160 0x7f475c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.358+0000 7f4760f77700 1 -- 172.21.15.49:0/197204246 >> 172.21.15.49:0/197204246 conn(0x7f475c06d770 msgr2=0x7f475c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.376+0000 7f4760f77700 1 -- 172.21.15.49:0/197204246 shutdown_connections 2023-10-08T09:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.379+0000 7f4760f77700 1 -- 172.21.15.49:0/197204246 wait complete. 2023-10-08T09:20:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 Processor -- start 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 -- start start 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f475c1077f0 0x7f475c1181f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f475c10cb30 0x7f475c118730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f475c1c4480 0x7f475c1c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1c6e40 con 0x7f475c10cb30 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1c6f80 con 0x7f475c1077f0 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4760f77700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c1c7140 con 0x7f475c1c4480 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4759d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f475c10cb30 0x7f475c118730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4759d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f475c10cb30 0x7f475c118730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51906/0 (socket says 172.21.15.49:51906) 2023-10-08T09:20:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f4759d9b700 1 -- 172.21.15.49:0/2177955163 learned_addr learned my addr 172.21.15.49:0/2177955163 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.380+0000 7f475ad9d700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f475c1c4480 0x7f475c1c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f475ad9d700 1 -- 172.21.15.49:0/2177955163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f475c1077f0 msgr2=0x7f475c1181f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f475ad9d700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f475c1077f0 0x7f475c1181f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f475ad9d700 1 -- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f475c10cb30 msgr2=0x7f475c118730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f475ad9d700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f475c10cb30 0x7f475c118730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f475ad9d700 1 -- 172.21.15.49:0/2177955163 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c01a600 con 0x7f475c1c4480 2023-10-08T09:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f475ad9d700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f475c1c4480 0x7f475c1c6870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f474c003820 tx=0x7f474c003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f474b7fe700 1 -- 172.21.15.49:0/2177955163 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c004050 con 0x7f475c1c4480 2023-10-08T09:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c1c73c0 con 0x7f475c1c4480 2023-10-08T09:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.381+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c1c7730 con 0x7f475c1c4480 2023-10-08T09:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.382+0000 7f474b7fe700 1 -- 172.21.15.49:0/2177955163 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f474c0041b0 con 0x7f475c1c4480 2023-10-08T09:20:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.382+0000 7f474b7fe700 1 -- 172.21.15.49:0/2177955163 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c00d630 con 0x7f475c1c4480 2023-10-08T09:20:52.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:51 smithi049 ceph-mon[99485]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.31k op/s 2023-10-08T09:20:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.387+0000 7f476d160b80 1 Processor -- start 2023-10-08T09:20:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f476d160b80 1 -- start start 2023-10-08T09:20:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f476d160b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1846a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f476d160b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1846acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f476d160b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f1846b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f476d160b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f183d2400 con 0x564f1846a7a0 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f476d160b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f183c3bc0 con 0x564f18469e40 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f476d160b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f183c14e0 con 0x564f1846b2b0 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f1846b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f1846b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38946/0 (socket says 172.21.15.49:38946) 2023-10-08T09:20:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 -- 172.21.15.49:0/3790208855 learned_addr learned my addr 172.21.15.49:0/3790208855 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1846a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 -- 172.21.15.49:0/3790208855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 msgr2=0x564f1846a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1846a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 -- 172.21.15.49:0/3790208855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 msgr2=0x564f1846acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1846acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.389+0000 7f4759d9b700 1 -- 172.21.15.49:0/3790208855 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f181e7d80 con 0x564f1846b2b0 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.390+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f1846b730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4754008b70 tx=0x7f4754009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f392afc8786a849 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.392+0000 7f474b7fe700 1 -- 172.21.15.49:0/2177955163 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f474c00e430 con 0x7f475c1c4480 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.393+0000 7f474b7fe700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f47440a15d0 0x7f47440a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.393+0000 7f474b7fe700 1 -- 172.21.15.49:0/2177955163 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f474c0ce420 con 0x7f475c1c4480 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.394+0000 7f4748ff9700 1 -- 172.21.15.49:0/3790208855 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754008e80 con 0x564f1846b2b0 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.394+0000 7f4748ff9700 1 -- 172.21.15.49:0/3790208855 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4754015c50 con 0x564f1846b2b0 2023-10-08T09:20:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.394+0000 7f4748ff9700 1 -- 172.21.15.49:0/3790208855 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475401ab60 con 0x564f1846b2b0 2023-10-08T09:20:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 -- 172.21.15.49:0/3790208855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 msgr2=0x564f1846b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f1846b730 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4754008b70 tx=0x7f4754009bf0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 -- 172.21.15.49:0/3790208855 shutdown_connections 2023-10-08T09:20:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f1846b730 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1846a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 --2- 172.21.15.49:0/3790208855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1846acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 -- 172.21.15.49:0/3790208855 >> 172.21.15.49:0/3790208855 conn(0x564f183de200 msgr2=0x564f184648a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.395+0000 7f476d160b80 1 -- 172.21.15.49:0/3790208855 shutdown_connections 2023-10-08T09:20:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.396+0000 7f476d160b80 1 -- 172.21.15.49:0/3790208855 wait complete. 2023-10-08T09:20:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.396+0000 7f475a59c700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f47440a15d0 0x7f47440a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.396+0000 7f476d160b80 1 -- start start 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.396+0000 7f476d160b80 1 Processor -- start 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.397+0000 7f476d160b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f18473900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.397+0000 7f476d160b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18473e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.397+0000 7f476d160b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846b2b0 0x564f1847ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.397+0000 7f476d160b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f183bcdd0 con 0x564f1846b2b0 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.397+0000 7f476d160b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f183596b0 con 0x564f18469e40 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.397+0000 7f476d160b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f18359350 con 0x564f1846a7a0 2023-10-08T09:20:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.397+0000 7f476d160b80 10 client.0 _ll_register_callbacks cb 0x564f181b67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846b2b0 0x564f1847ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846b2b0 0x564f1847ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51920/0 (socket says 172.21.15.49:51920) 2023-10-08T09:20:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 -- 172.21.15.49:0/70977461 learned_addr learned my addr 172.21.15.49:0/70977461 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f4759d9b700 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18473e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 -- 172.21.15.49:0/70977461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 msgr2=0x564f18473e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18473e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 -- 172.21.15.49:0/70977461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 msgr2=0x564f18473900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f18473900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.398+0000 7f475ad9d700 1 -- 172.21.15.49:0/70977461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4754011040 con 0x564f1846b2b0 2023-10-08T09:20:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.399+0000 7f475a59c700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f47440a15d0 0x7f47440a3a90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f475c1193c0 tx=0x7f4750006c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.399+0000 7f475ad9d700 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846b2b0 0x564f1847ae80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f474c09b1d0 tx=0x7f474c09b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba16f7138b1b9b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.402+0000 7f4741ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.402+0000 7f4741ffb700 1 -- 172.21.15.49:0/70977461 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c09bbd0 con 0x564f1846b2b0 2023-10-08T09:20:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.402+0000 7f4741ffb700 1 -- 172.21.15.49:0/70977461 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f474c09b4d0 con 0x564f1846b2b0 2023-10-08T09:20:52.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.402+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f18384e50 con 0x564f1846b2b0 2023-10-08T09:20:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.407+0000 7f4741ffb700 1 -- 172.21.15.49:0/70977461 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f474c09f620 con 0x564f1846b2b0 2023-10-08T09:20:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.411+0000 7f476d160b80 10 client.16410 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.411+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564f181d0c00 con 0x564f1846b2b0 2023-10-08T09:20:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f4741ffb700 1 -- 172.21.15.49:0/70977461 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f18384e50 con 0x564f1846b2b0 2023-10-08T09:20:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f4741ffb700 1 -- 172.21.15.49:0/70977461 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f474c0a3340 con 0x564f1846b2b0 2023-10-08T09:20:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f476d160b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f476d160b80 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f476d160b80 20 client.16410 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f476d160b80 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f1847f3a0 0x564f184817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f476d160b80 4 client.16410 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:20:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.412+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f183b5db0 con 0x564f1847f3a0 2023-10-08T09:20:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.413+0000 7f475a59c700 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f1847f3a0 0x564f184817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.414+0000 7f4741ffb700 10 client.16410 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:52.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.414+0000 7f475a59c700 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f1847f3a0 0x564f184817c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49573159d1766232 server_cookie=804c2c04108d560 in_seq=0 out_seq=0 2023-10-08T09:20:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.415+0000 7f4741ffb700 1 -- 172.21.15.49:0/70977461 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564f183b5db0 con 0x564f1847f3a0 2023-10-08T09:20:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.415+0000 7f4741ffb700 10 client.16410 handle_command_reply: tid=0 2023-10-08T09:20:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.415+0000 7f476d160b80 1 client.16410 shutdown 2023-10-08T09:20:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846b2b0 msgr2=0x564f1847ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846b2b0 0x564f1847ae80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f474c09b1d0 tx=0x7f474c09b3b0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 shutdown_connections 2023-10-08T09:20:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f1847f3a0 0x564f184817c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846b2b0 0x564f1847ae80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f18473900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 --2- 172.21.15.49:0/70977461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18473e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 >> 172.21.15.49:0/70977461 conn(0x564f183de200 msgr2=0x564f18468b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f4741ffb700 0 client.16410 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 shutdown_connections 2023-10-08T09:20:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.417+0000 7f476d160b80 1 -- 172.21.15.49:0/70977461 wait complete. 2023-10-08T09:20:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.422+0000 7f476d160b80 20 client.16410 trim_cache size 0 max 16384 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 Processor -- start 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 -- start start 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1847b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1847ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f18464830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1823d8c0 con 0x564f1846a7a0 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f18384e50 con 0x564f18469e40 2023-10-08T09:20:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.450+0000 7f476d160b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f18466330 con 0x564f1846b2b0 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f475a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1847b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f475a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1847b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41540/0 (socket says 172.21.15.49:41540) 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f4759d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1847ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f475a59c700 1 -- 172.21.15.49:0/3766382399 learned_addr learned my addr 172.21.15.49:0/3766382399 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f475ad9d700 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f18464830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f4759d9b700 1 -- 172.21.15.49:0/3766382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 msgr2=0x564f18464830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f18464830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f4759d9b700 1 -- 172.21.15.49:0/3766382399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 msgr2=0x564f1847b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1847b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f4759d9b700 1 -- 172.21.15.49:0/3766382399 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f1820e1f0 con 0x564f1846a7a0 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f475a59c700 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1847b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f475ad9d700 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f18464830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.451+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1847ae40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4754008470 tx=0x7f4754009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e769359dbe7a97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.452+0000 7f47497fa700 1 -- 172.21.15.49:0/3766382399 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754008e80 con 0x564f1846a7a0 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.452+0000 7f47497fa700 1 -- 172.21.15.49:0/3766382399 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f475401a460 con 0x564f1846a7a0 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.452+0000 7f47497fa700 1 -- 172.21.15.49:0/3766382399 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475401a970 con 0x564f1846a7a0 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.452+0000 7f476d160b80 1 -- 172.21.15.49:0/3766382399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 msgr2=0x564f1847ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1847ae40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4754008470 tx=0x7f4754009a70 comp rx=0 tx=0).stop 2023-10-08T09:20:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 -- 172.21.15.49:0/3766382399 shutdown_connections 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846b2b0 0x564f18464830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f18469e40 0x564f1847b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 --2- 172.21.15.49:0/3766382399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f1846a7a0 0x564f1847ae40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 -- 172.21.15.49:0/3766382399 >> 172.21.15.49:0/3766382399 conn(0x564f183de200 msgr2=0x564f18486120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 -- 172.21.15.49:0/3766382399 shutdown_connections 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 -- 172.21.15.49:0/3766382399 wait complete. 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 -- start start 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.453+0000 7f476d160b80 1 Processor -- start 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f18469e40 0x564f184786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18478c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f1846b2b0 0x564f18479b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1847a0b0 con 0x564f18469e40 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1847a220 con 0x564f1846b2b0 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1847a3e0 con 0x564f1846a7a0 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 10 client.0 _ll_register_callbacks cb 0x7f474c097830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f18469e40 0x564f184786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f18469e40 0x564f184786c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51932/0 (socket says 172.21.15.49:51932) 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 -- 172.21.15.49:0/3853527659 learned_addr learned my addr 172.21.15.49:0/3853527659 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18478c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475ad9d700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f1846b2b0 0x564f18479b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 -- 172.21.15.49:0/3853527659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 msgr2=0x564f18478c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18478c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 -- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f1846b2b0 msgr2=0x564f18479b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f1846b2b0 0x564f18479b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 -- 172.21.15.49:0/3853527659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47500135c0 con 0x564f18469e40 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18478c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f475a59c700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f18469e40 0x564f184786c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4750020d20 tx=0x7f4750004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e90f5a1239a1d32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f4741ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f4741ffb700 1 -- 172.21.15.49:0/3853527659 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475004a5f0 con 0x564f18469e40 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.454+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4754015c50 con 0x564f18469e40 2023-10-08T09:20:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f4741ffb700 1 -- 172.21.15.49:0/3853527659 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f475004a750 con 0x564f18469e40 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f4741ffb700 1 -- 172.21.15.49:0/3853527659 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475004a910 con 0x564f18469e40 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 10 client.16416 fetch_fsmap learned FSMap version 40 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564f181d0c00 con 0x564f18469e40 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f4741ffb700 1 -- 172.21.15.49:0/3853527659 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4754015c50 con 0x564f18469e40 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f4741ffb700 1 -- 172.21.15.49:0/3853527659 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f475001fa40 con 0x564f18469e40 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 10 client.16416 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 20 client.16416 populate_metadata read hostname 'smithi049' 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f184d3d80 0x564f184d61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:20:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 4 client.16416 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:20:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564f183b5ad0 con 0x564f184d3d80 2023-10-08T09:20:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.455+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f184d3d80 0x564f184d61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:20:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.456+0000 7f4741ffb700 10 client.16416 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.456+0000 7f4759d9b700 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f184d3d80 0x564f184d61a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f46b0b036daccd9 server_cookie=a6c0a603c8a67fbf in_seq=0 out_seq=0 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f4741ffb700 1 -- 172.21.15.49:0/3853527659 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564f183b5ad0 con 0x564f184d3d80 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f4741ffb700 10 client.16416 handle_command_reply: tid=0 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 client.16416 shutdown 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f18469e40 msgr2=0x564f184786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f18469e40 0x564f184786c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4750020d20 tx=0x7f4750004800 comp rx=0 tx=0).stop 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 shutdown_connections 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564f184d3d80 0x564f184d61a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564f1846b2b0 0x564f18479b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564f18469e40 0x564f184786c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f4741ffb700 0 client.16416 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 --2- 172.21.15.49:0/3853527659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564f1846a7a0 0x564f18478c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 >> 172.21.15.49:0/3853527659 conn(0x564f183de200 msgr2=0x564f184647f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 shutdown_connections 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 1 -- 172.21.15.49:0/3853527659 wait complete. 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.465+0000 7f476d160b80 20 client.16416 trim_cache size 0 max 16384 2023-10-08T09:20:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:20:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:20:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:20:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:20:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f47440a15d0 msgr2=0x7f47440a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f47440a15d0 0x7f47440a3a90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f475c1193c0 tx=0x7f4750006c00 comp rx=0 tx=0).stop 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f475c1c4480 msgr2=0x7f475c1c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f475c1c4480 0x7f475c1c6870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f474c003820 tx=0x7f474c003c20 comp rx=0 tx=0).stop 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 shutdown_connections 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f47440a15d0 0x7f47440a3a90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f475c1c4480 0x7f475c1c6870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f475c1077f0 0x7f475c1181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 --2- 172.21.15.49:0/2177955163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f475c10cb30 0x7f475c118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:20:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 >> 172.21.15.49:0/2177955163 conn(0x7f475c06d770 msgr2=0x7f475c108040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:20:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 shutdown_connections 2023-10-08T09:20:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:20:52.466+0000 7f4760f77700 1 -- 172.21.15.49:0/2177955163 wait complete. 2023-10-08T09:20:54.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:53 smithi184 ceph-mon[103675]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-10-08T09:20:54.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:53 smithi132 ceph-mon[103389]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-10-08T09:20:54.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 330.0 s, 707.6 tps, lat 137.092 ms stddev 173.597 2023-10-08T09:20:54.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:53 smithi049 ceph-mon[99485]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-10-08T09:20:56.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:55 smithi184 ceph-mon[103675]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-10-08T09:20:56.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:55 smithi132 ceph-mon[103389]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-10-08T09:20:56.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:55 smithi049 ceph-mon[99485]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 1.22k op/s 2023-10-08T09:20:58.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:20:58 smithi132 ceph-mon[103389]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-10-08T09:20:58.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:20:58 smithi049 ceph-mon[99485]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-10-08T09:20:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:20:58 smithi184 ceph-mon[103675]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 1.32k op/s 2023-10-08T09:20:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 335.0 s, 715.0 tps, lat 144.963 ms stddev 178.598 2023-10-08T09:21:00.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:00 smithi049 ceph-mon[99485]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 917 op/s 2023-10-08T09:21:00.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:00 smithi184 ceph-mon[103675]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 917 op/s 2023-10-08T09:21:00.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:00 smithi132 ceph-mon[103389]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 917 op/s 2023-10-08T09:21:01.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:01 smithi049 ceph-mon[99485]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.8 MiB/s wr, 722 op/s 2023-10-08T09:21:01.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:01 smithi184 ceph-mon[103675]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.8 MiB/s wr, 722 op/s 2023-10-08T09:21:01.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:01 smithi132 ceph-mon[103389]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.8 MiB/s wr, 722 op/s 2023-10-08T09:21:03.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:03 smithi184 ceph-mon[103675]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-10-08T09:21:04.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:03 smithi132 ceph-mon[103389]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-10-08T09:21:04.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:03 smithi049 ceph-mon[99485]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.8 MiB/s wr, 640 op/s 2023-10-08T09:21:04.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 340.0 s, 650.4 tps, lat 152.925 ms stddev 165.008 2023-10-08T09:21:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.673+0000 7fb009095700 1 -- 172.21.15.49:0/3500164837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0041077f0 msgr2=0x7fb004109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.673+0000 7fb009095700 1 --2- 172.21.15.49:0/3500164837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0041077f0 0x7fb004109c20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faffc01c0d0 tx=0x7faffc01c3e0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.674+0000 7fb009095700 1 -- 172.21.15.49:0/3500164837 shutdown_connections 2023-10-08T09:21:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.674+0000 7fb009095700 1 --2- 172.21.15.49:0/3500164837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb00410cb30 0x7fb00410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.674+0000 7fb009095700 1 --2- 172.21.15.49:0/3500164837 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0041077f0 0x7fb004109c20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.674+0000 7fb009095700 1 --2- 172.21.15.49:0/3500164837 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb00410a160 0x7fb00410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.674+0000 7fb009095700 1 -- 172.21.15.49:0/3500164837 >> 172.21.15.49:0/3500164837 conn(0x7fb00406d770 msgr2=0x7fb00406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.675+0000 7fb009095700 1 -- 172.21.15.49:0/3500164837 shutdown_connections 2023-10-08T09:21:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.675+0000 7fb009095700 1 -- 172.21.15.49:0/3500164837 wait complete. 2023-10-08T09:21:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.676+0000 7fb009095700 1 Processor -- start 2023-10-08T09:21:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb009095700 1 -- start start 2023-10-08T09:21:05.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb009095700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0041077f0 0x7fb00419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb009095700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb00410cb30 0x7fb00419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb009095700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0041a23e0 0x7fb0041a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb009095700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041a4da0 con 0x7fb0041a23e0 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb009095700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041a4ee0 con 0x7fb0041077f0 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb009095700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041a5070 con 0x7fb00410cb30 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0041077f0 0x7fb00419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0041077f0 0x7fb00419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41560/0 (socket says 172.21.15.49:41560) 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb002d9d700 1 -- 172.21.15.49:0/2903974247 learned_addr learned my addr 172.21.15.49:0/2903974247 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.677+0000 7fb00259c700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb00410cb30 0x7fb00419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7fb002d9d700 1 -- 172.21.15.49:0/2903974247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb00410cb30 msgr2=0x7fb00419d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7fb002d9d700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb00410cb30 0x7fb00419d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7fb002d9d700 1 -- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0041a23e0 msgr2=0x7fb0041a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7fb002d9d700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0041a23e0 0x7fb0041a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7fb002d9d700 1 -- 172.21.15.49:0/2903974247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc01ad10 con 0x7fb0041077f0 2023-10-08T09:21:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7fb002d9d700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0041077f0 0x7fb00419ce90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faffc018940 tx=0x7faffc018a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7faffbfff700 1 -- 172.21.15.49:0/2903974247 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc01f040 con 0x7fb0041077f0 2023-10-08T09:21:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7faffbfff700 1 -- 172.21.15.49:0/2903974247 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faffc004e60 con 0x7fb0041077f0 2023-10-08T09:21:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.678+0000 7faffbfff700 1 -- 172.21.15.49:0/2903974247 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc02c9f0 con 0x7fb0041077f0 2023-10-08T09:21:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.681+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0041a52f0 con 0x7fb0041077f0 2023-10-08T09:21:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.681+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0041a5720 con 0x7fb0041077f0 2023-10-08T09:21:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.685+0000 7faffbfff700 1 -- 172.21.15.49:0/2903974247 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7faffc02ecc0 con 0x7fb0041077f0 2023-10-08T09:21:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.685+0000 7faffbfff700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7faff00a1550 0x7faff00a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.686+0000 7faffbfff700 1 -- 172.21.15.49:0/2903974247 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faffc0df020 con 0x7fb0041077f0 2023-10-08T09:21:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.686+0000 7fb00259c700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7faff00a1550 0x7faff00a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.686+0000 7fb01527eb80 1 Processor -- start 2023-10-08T09:21:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb01527eb80 1 -- start start 2023-10-08T09:21:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb00259c700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7faff00a1550 0x7faff00a3a10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb004071c70 tx=0x7faff4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d46e40 0x564200d47260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d47cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb01527eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200caf400 con 0x564200d477a0 2023-10-08T09:21:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb01527eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200ca0bc0 con 0x564200d482b0 2023-10-08T09:21:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.687+0000 7fb01527eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200c9e4e0 con 0x564200d46e40 2023-10-08T09:21:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.688+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d46e40 0x564200d47260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.688+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d46e40 0x564200d47260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:38988/0 (socket says 172.21.15.49:38988) 2023-10-08T09:21:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.688+0000 7fb002d9d700 1 -- 172.21.15.49:0/2407452778 learned_addr learned my addr 172.21.15.49:0/2407452778 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.688+0000 7fb00359e700 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7fb002d9d700 1 -- 172.21.15.49:0/2407452778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 msgr2=0x564200d48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7fb002d9d700 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d48730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7fb002d9d700 1 -- 172.21.15.49:0/2407452778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 msgr2=0x564200d47cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7fb002d9d700 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d47cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7fb002d9d700 1 -- 172.21.15.49:0/2407452778 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564200ac4d80 con 0x564200d46e40 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7fb002d9d700 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d46e40 0x564200d47260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faffc0abda0 tx=0x7faffc02c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd70b11e0d4cf08 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7faff97fa700 1 -- 172.21.15.49:0/2407452778 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0b43a0 con 0x564200d46e40 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7faff97fa700 1 -- 172.21.15.49:0/2407452778 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faffc0aa440 con 0x564200d46e40 2023-10-08T09:21:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.689+0000 7faff97fa700 1 -- 172.21.15.49:0/2407452778 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc0b2310 con 0x564200d46e40 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 -- 172.21.15.49:0/2407452778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d46e40 msgr2=0x564200d47260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d46e40 0x564200d47260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faffc0abda0 tx=0x7faffc02c4f0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 -- 172.21.15.49:0/2407452778 shutdown_connections 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d46e40 0x564200d47260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2407452778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d47cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 -- 172.21.15.49:0/2407452778 >> 172.21.15.49:0/2407452778 conn(0x564200cbb200 msgr2=0x564200d418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 -- 172.21.15.49:0/2407452778 shutdown_connections 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.690+0000 7fb01527eb80 1 -- 172.21.15.49:0/2407452778 wait complete. 2023-10-08T09:21:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 -- start start 2023-10-08T09:21:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 Processor -- start 2023-10-08T09:21:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d50c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d482b0 0x564200d57c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200c99dd0 con 0x564200d482b0 2023-10-08T09:21:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200c366b0 con 0x564200d46e40 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200c36350 con 0x564200d477a0 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb01527eb80 10 client.0 _ll_register_callbacks cb 0x564200a937a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb00359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d482b0 0x564200d57c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb00359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d482b0 0x564200d57c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51964/0 (socket says 172.21.15.49:51964) 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb00259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d50c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41574/0 (socket says 172.21.15.49:41574) 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.691+0000 7fb00359e700 1 -- 172.21.15.49:0/1028230548 learned_addr learned my addr 172.21.15.49:0/1028230548 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb00359e700 1 -- 172.21.15.49:0/1028230548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 msgr2=0x564200d50c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb00359e700 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d50c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb00359e700 1 -- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 msgr2=0x564200d506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb00359e700 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb00359e700 1 -- 172.21.15.49:0/1028230548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0041a5720 con 0x564200d482b0 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb002d9d700 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb00359e700 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d482b0 0x564200d57c20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fafec00f760 tx=0x7fafec00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d610a54fe56b2531 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fafea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fafea7fc700 1 -- 172.21.15.49:0/1028230548 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00a760 con 0x564200d482b0 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fafea7fc700 1 -- 172.21.15.49:0/1028230548 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fafec010960 con 0x564200d482b0 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564200c61e50 con 0x564200d482b0 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fafea7fc700 1 -- 172.21.15.49:0/1028230548 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafec010b20 con 0x564200d482b0 2023-10-08T09:21:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb01527eb80 10 client.16422 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.692+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564200aadc00 con 0x564200d482b0 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fafea7fc700 1 -- 172.21.15.49:0/1028230548 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564200c61e50 con 0x564200d482b0 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fafea7fc700 1 -- 172.21.15.49:0/1028230548 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fafec0173a0 con 0x564200d482b0 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fb01527eb80 10 client.16422 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fb01527eb80 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fb01527eb80 20 client.16422 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fb01527eb80 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200d5c560 0x564200d5e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fb01527eb80 4 client.16422 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564200c92db0 con 0x564200d5c560 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.693+0000 7fb002d9d700 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200d5c560 0x564200d5e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb002d9d700 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200d5c560 0x564200d5e980 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cab4d40520dab63 server_cookie=2ca4953d1ff3126 in_seq=0 out_seq=0 2023-10-08T09:21:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fafea7fc700 10 client.16422 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fafea7fc700 1 -- 172.21.15.49:0/1028230548 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564200c92db0 con 0x564200d5c560 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fafea7fc700 10 client.16422 handle_command_reply: tid=0 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 client.16422 shutdown 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d482b0 msgr2=0x564200d57c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d482b0 0x564200d57c20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fafec00f760 tx=0x7fafec00fa70 comp rx=0 tx=0).stop 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 shutdown_connections 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200d5c560 0x564200d5e980 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d482b0 0x564200d57c20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fafea7fc700 0 client.16422 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 --2- 172.21.15.49:0/1028230548 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d50c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.694+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 >> 172.21.15.49:0/1028230548 conn(0x564200cbb200 msgr2=0x564200d41010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.695+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 shutdown_connections 2023-10-08T09:21:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.695+0000 7fb01527eb80 1 -- 172.21.15.49:0/1028230548 wait complete. 2023-10-08T09:21:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.695+0000 7fb01527eb80 20 client.16422 trim_cache size 0 max 16384 2023-10-08T09:21:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.711+0000 7fb01527eb80 1 Processor -- start 2023-10-08T09:21:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.711+0000 7fb01527eb80 1 -- start start 2023-10-08T09:21:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d48690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d482b0 0x564200d57b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb01527eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200b1a8c0 con 0x564200d477a0 2023-10-08T09:21:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb01527eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200d3e860 con 0x564200d46e40 2023-10-08T09:21:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb01527eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200d39670 con 0x564200d482b0 2023-10-08T09:21:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d48690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d482b0 0x564200d57b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d4fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41582/0 (socket says 172.21.15.49:41582) 2023-10-08T09:21:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d48690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51970/0 (socket says 172.21.15.49:51970) 2023-10-08T09:21:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb002d9d700 1 -- 172.21.15.49:0/2053557783 learned_addr learned my addr 172.21.15.49:0/2053557783 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00259c700 1 -- 172.21.15.49:0/2053557783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d482b0 msgr2=0x564200d57b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00259c700 1 --2- 172.21.15.49:0/2053557783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d482b0 0x564200d57b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00259c700 1 -- 172.21.15.49:0/2053557783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 msgr2=0x564200d4fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00259c700 1 --2- 172.21.15.49:0/2053557783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d4fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.712+0000 7fb00259c700 1 -- 172.21.15.49:0/2053557783 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564200aeb1f0 con 0x564200d477a0 2023-10-08T09:21:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.713+0000 7fb00259c700 1 --2- 172.21.15.49:0/2053557783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d48690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb004071bc0 tx=0x7faff4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d82506ad6d2b1e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.713+0000 7faff9ffb700 1 -- 172.21.15.49:0/2053557783 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff40116e0 con 0x564200d477a0 2023-10-08T09:21:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.713+0000 7faff9ffb700 1 -- 172.21.15.49:0/2053557783 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff4011d00 con 0x564200d477a0 2023-10-08T09:21:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.713+0000 7faff9ffb700 1 -- 172.21.15.49:0/2053557783 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff401fdb0 con 0x564200d477a0 2023-10-08T09:21:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.713+0000 7fb01527eb80 1 -- 172.21.15.49:0/2053557783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 msgr2=0x564200d48690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.713+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2053557783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d48690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb004071bc0 tx=0x7faff4004790 comp rx=0 tx=0).stop 2023-10-08T09:21:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- 172.21.15.49:0/2053557783 shutdown_connections 2023-10-08T09:21:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2053557783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d482b0 0x564200d57b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2053557783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d46e40 0x564200d4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 --2- 172.21.15.49:0/2053557783 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d477a0 0x564200d48690 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- 172.21.15.49:0/2053557783 >> 172.21.15.49:0/2053557783 conn(0x564200cbb200 msgr2=0x564200d632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- 172.21.15.49:0/2053557783 shutdown_connections 2023-10-08T09:21:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- 172.21.15.49:0/2053557783 wait complete. 2023-10-08T09:21:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- start start 2023-10-08T09:21:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 Processor -- start 2023-10-08T09:21:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d46e40 0x564200d53520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d53a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d56a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200d56fc0 con 0x564200d46e40 2023-10-08T09:21:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200d57130 con 0x564200d482b0 2023-10-08T09:21:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.714+0000 7fb01527eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564200d572f0 con 0x564200d477a0 2023-10-08T09:21:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb01527eb80 10 client.0 _ll_register_callbacks cb 0x7fafec00ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d46e40 0x564200d53520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d46e40 0x564200d53520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51974/0 (socket says 172.21.15.49:51974) 2023-10-08T09:21:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 -- 172.21.15.49:0/4205147899 learned_addr learned my addr 172.21.15.49:0/4205147899 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb00259c700 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d53a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb00359e700 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d56a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 -- 172.21.15.49:0/4205147899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 msgr2=0x564200d53a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d53a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 -- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 msgr2=0x564200d56a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d56a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 -- 172.21.15.49:0/4205147899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff4013750 con 0x564200d46e40 2023-10-08T09:21:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb002d9d700 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d46e40 0x564200d53520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7faffc0dcb60 tx=0x7faffc02c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73377dfcb3c1674 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fafea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:21:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fafea7fc700 1 -- 172.21.15.49:0/4205147899 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc035310 con 0x564200d46e40 2023-10-08T09:21:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fafea7fc700 1 -- 172.21.15.49:0/4205147899 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7faffc0ca1b0 con 0x564200d46e40 2023-10-08T09:21:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faffc0aa440 con 0x564200d46e40 2023-10-08T09:21:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fafea7fc700 1 -- 172.21.15.49:0/4205147899 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7faffc035470 con 0x564200d46e40 2023-10-08T09:21:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb01527eb80 10 client.16428 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.715+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564200ac3250 con 0x564200d46e40 2023-10-08T09:21:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fafea7fc700 1 -- 172.21.15.49:0/4205147899 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faffc0aa440 con 0x564200d46e40 2023-10-08T09:21:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fafea7fc700 1 -- 172.21.15.49:0/4205147899 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7faffc0ab590 con 0x564200d46e40 2023-10-08T09:21:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb01527eb80 10 client.16428 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb01527eb80 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb01527eb80 20 client.16428 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb01527eb80 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200db0bb0 0x564200db2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb01527eb80 4 client.16428 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:21:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564200c92ad0 con 0x564200db0bb0 2023-10-08T09:21:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb00259c700 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200db0bb0 0x564200db2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fafea7fc700 10 client.16428 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.716+0000 7fb00259c700 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200db0bb0 0x564200db2fd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c9bdd60a78ba27 server_cookie=33e4502ad505e5f5 in_seq=0 out_seq=0 2023-10-08T09:21:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fafea7fc700 1 -- 172.21.15.49:0/4205147899 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564200c92ad0 con 0x564200db0bb0 2023-10-08T09:21:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fafea7fc700 10 client.16428 handle_command_reply: tid=0 2023-10-08T09:21:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 client.16428 shutdown 2023-10-08T09:21:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d46e40 msgr2=0x564200d53520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d46e40 0x564200d53520 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7faffc0dcb60 tx=0x7faffc02c460 comp rx=0 tx=0).stop 2023-10-08T09:21:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 shutdown_connections 2023-10-08T09:21:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564200db0bb0 0x564200db2fd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564200d482b0 0x564200d56a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564200d46e40 0x564200d53520 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fafea7fc700 0 client.16428 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 --2- 172.21.15.49:0/4205147899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564200d477a0 0x564200d53a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 >> 172.21.15.49:0/4205147899 conn(0x564200cbb200 msgr2=0x564200d50690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 shutdown_connections 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 1 -- 172.21.15.49:0/4205147899 wait complete. 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.729+0000 7fb01527eb80 20 client.16428 trim_cache size 0 max 16384 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:21:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:21:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:21:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7faff00a1550 msgr2=0x7faff00a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7faff00a1550 0x7faff00a3a10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb004071c70 tx=0x7faff4008040 comp rx=0 tx=0).stop 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0041077f0 msgr2=0x7fb00419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0041077f0 0x7fb00419ce90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faffc018940 tx=0x7faffc018a20 comp rx=0 tx=0).stop 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 shutdown_connections 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7faff00a1550 0x7faff00a3a10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0041a23e0 0x7fb0041a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0041077f0 0x7fb00419ce90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 --2- 172.21.15.49:0/2903974247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb00410cb30 0x7fb00419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 >> 172.21.15.49:0/2903974247 conn(0x7fb00406d770 msgr2=0x7fb004108000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 shutdown_connections 2023-10-08T09:21:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:05.730+0000 7fb009095700 1 -- 172.21.15.49:0/2903974247 wait complete. 2023-10-08T09:21:06.004 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:05 smithi184 ceph-mon[103675]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-10-08T09:21:06.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:05 smithi132 ceph-mon[103389]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-10-08T09:21:06.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:05 smithi049 ceph-mon[99485]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 549 op/s 2023-10-08T09:21:08.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:07 smithi184 ceph-mon[103675]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 695 op/s 2023-10-08T09:21:08.058 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:07 smithi132 ceph-mon[103389]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 695 op/s 2023-10-08T09:21:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:07 smithi049 ceph-mon[99485]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 695 op/s 2023-10-08T09:21:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 345.0 s, 620.2 tps, lat 157.983 ms stddev 160.433 2023-10-08T09:21:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:09 smithi132 ceph-mon[103389]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.9 MiB/s wr, 638 op/s 2023-10-08T09:21:10.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:09 smithi049 ceph-mon[99485]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.9 MiB/s wr, 638 op/s 2023-10-08T09:21:10.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:09 smithi184 ceph-mon[103675]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.9 MiB/s wr, 638 op/s 2023-10-08T09:21:12.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:11 smithi132 ceph-mon[103389]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 650 op/s 2023-10-08T09:21:12.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:11 smithi049 ceph-mon[99485]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 650 op/s 2023-10-08T09:21:12.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:11 smithi184 ceph-mon[103675]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 650 op/s 2023-10-08T09:21:14.051 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:13 smithi132 ceph-mon[103389]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.4 MiB/s wr, 656 op/s 2023-10-08T09:21:14.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:13 smithi049 ceph-mon[99485]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.4 MiB/s wr, 656 op/s 2023-10-08T09:21:14.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:13 smithi184 ceph-mon[103675]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.4 MiB/s wr, 656 op/s 2023-10-08T09:21:14.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 350.0 s, 650.2 tps, lat 153.297 ms stddev 176.251 2023-10-08T09:21:14.486 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:21:14.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 -- 172.21.15.49:0/814735104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 msgr2=0x7f8238100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 --2- 172.21.15.49:0/814735104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 0x7f8238100d00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f822c0099c0 tx=0x7f822c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 -- 172.21.15.49:0/814735104 shutdown_connections 2023-10-08T09:21:14.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 --2- 172.21.15.49:0/814735104 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82380ff6b0 0x7f82380ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 --2- 172.21.15.49:0/814735104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 0x7f8238107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 --2- 172.21.15.49:0/814735104 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 0x7f8238100d00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 -- 172.21.15.49:0/814735104 >> 172.21.15.49:0/814735104 conn(0x7f82380fac40 msgr2=0x7f82380fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:14.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 -- 172.21.15.49:0/814735104 shutdown_connections 2023-10-08T09:21:14.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.756+0000 7f823d8ab700 1 -- 172.21.15.49:0/814735104 wait complete. 2023-10-08T09:21:14.758 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 Processor -- start 2023-10-08T09:21:14.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 -- start start 2023-10-08T09:21:14.759 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82380ff6b0 0x7f8238198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 0x7f8238198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 0x7f823819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381a0480 con 0x7f82381008a0 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381a05c0 con 0x7f8238101240 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f823d8ab700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381a0750 con 0x7f82380ff6b0 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f82377fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 0x7f823819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82380ff6b0 0x7f8238198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f82367fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 0x7f8238198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f82377fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 0x7f823819feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41604/0 (socket says 172.21.15.49:41604) 2023-10-08T09:21:14.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82380ff6b0 0x7f8238198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39022/0 (socket says 172.21.15.49:39022) 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f82367fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 0x7f8238198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51992/0 (socket says 172.21.15.49:51992) 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.757+0000 7f82377fe700 1 -- 172.21.15.49:0/1014227859 learned_addr learned my addr 172.21.15.49:0/1014227859 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f82377fe700 1 -- 172.21.15.49:0/1014227859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82380ff6b0 msgr2=0x7f8238198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f82377fe700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82380ff6b0 0x7f8238198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f82377fe700 1 -- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 msgr2=0x7f8238198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f82377fe700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 0x7f8238198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f82377fe700 1 -- 172.21.15.49:0/1014227859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c0095f0 con 0x7f8238101240 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f82377fe700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 0x7f823819feb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f823000ef30 tx=0x7f823000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f823c8a9700 1 -- 172.21.15.49:0/1014227859 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230018070 con 0x7f8238101240 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82381a09d0 con 0x7f8238101240 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f823c8a9700 1 -- 172.21.15.49:0/1014227859 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f823000f040 con 0x7f8238101240 2023-10-08T09:21:14.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f823c8a9700 1 -- 172.21.15.49:0/1014227859 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000cd00 con 0x7f8238101240 2023-10-08T09:21:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.758+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82381a0f30 con 0x7f8238101240 2023-10-08T09:21:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.761+0000 7f823c8a9700 1 -- 172.21.15.49:0/1014227859 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8230003a00 con 0x7f8238101240 2023-10-08T09:21:14.762 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.762+0000 7f823c8a9700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f82200a1480 0x7f82200a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.762+0000 7f823c8a9700 1 -- 172.21.15.49:0/1014227859 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f82300ccb00 con 0x7f8238101240 2023-10-08T09:21:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.762+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f82200a1480 0x7f82200a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.763+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f82200a1480 0x7f82200a3940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8238100700 tx=0x7f822800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.764+0000 7f8249a94b80 1 Processor -- start 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.764+0000 7f8249a94b80 1 -- start start 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde1060 0x562e5cde1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde2800 0x562e5cde2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8249a94b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5cd49460 con 0x562e5cde1060 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8249a94b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5cd3ac20 con 0x562e5cde2800 2023-10-08T09:21:14.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8249a94b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5cd38540 con 0x562e5cde19c0 2023-10-08T09:21:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde1060 0x562e5cde1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde1060 0x562e5cde1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:51996/0 (socket says 172.21.15.49:51996) 2023-10-08T09:21:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 -- 172.21.15.49:0/1778869899 learned_addr learned my addr 172.21.15.49:0/1778869899 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f82367fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f82377fe700 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde2800 0x562e5cde2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.767 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 -- 172.21.15.49:0/1778869899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 msgr2=0x562e5cde2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 -- 172.21.15.49:0/1778869899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde2800 msgr2=0x562e5cde2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde2800 0x562e5cde2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.765+0000 7f8236ffd700 1 -- 172.21.15.49:0/1778869899 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e5cb5ede0 con 0x562e5cde1060 2023-10-08T09:21:14.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.766+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde1060 0x562e5cde1480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8228009c70 tx=0x7f8228015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f153df57b0b3e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.766+0000 7f821dffb700 1 -- 172.21.15.49:0/1778869899 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228019a20 con 0x562e5cde1060 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.766+0000 7f821dffb700 1 -- 172.21.15.49:0/1778869899 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8228013040 con 0x562e5cde1060 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.766+0000 7f821dffb700 1 -- 172.21.15.49:0/1778869899 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280173c0 con 0x562e5cde1060 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 -- 172.21.15.49:0/1778869899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde1060 msgr2=0x562e5cde1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde1060 0x562e5cde1480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8228009c70 tx=0x7f8228015000 comp rx=0 tx=0).stop 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 -- 172.21.15.49:0/1778869899 shutdown_connections 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde2800 0x562e5cde2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde1060 0x562e5cde1480 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1778869899 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 -- 172.21.15.49:0/1778869899 >> 172.21.15.49:0/1778869899 conn(0x562e5cd55260 msgr2=0x562e5cddbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 -- 172.21.15.49:0/1778869899 shutdown_connections 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 -- 172.21.15.49:0/1778869899 wait complete. 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.767+0000 7f8249a94b80 1 -- start start 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 1 Processor -- start 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde1060 0x562e5cdeae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cdeb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5cd33e30 con 0x562e5cde2800 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5ccd0710 con 0x562e5cde1060 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5ccd03b0 con 0x562e5cde19c0 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8249a94b80 10 client.0 _ll_register_callbacks cb 0x562e5cb2d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:14.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde1060 0x562e5cdeae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde1060 0x562e5cdeae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41616/0 (socket says 172.21.15.49:41616) 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8236ffd700 1 -- 172.21.15.49:0/1858180401 learned_addr learned my addr 172.21.15.49:0/1858180401 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f82367fc700 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cdeb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f82377fe700 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8236ffd700 1 -- 172.21.15.49:0/1858180401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 msgr2=0x562e5cdeb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.768+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cdeb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8236ffd700 1 -- 172.21.15.49:0/1858180401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 msgr2=0x562e5cdf2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8236ffd700 1 -- 172.21.15.49:0/1858180401 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82280099e0 con 0x562e5cde1060 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde1060 0x562e5cdeae00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82280196f0 tx=0x7f8228024e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2000ecfcb98a5a9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8216ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:21:14.774 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8216ffd700 1 -- 172.21.15.49:0/1858180401 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280054d0 con 0x562e5cde1060 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8216ffd700 1 -- 172.21.15.49:0/1858180401 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8228005630 con 0x562e5cde1060 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e5ccfbeb0 con 0x562e5cde1060 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8216ffd700 1 -- 172.21.15.49:0/1858180401 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8228019820 con 0x562e5cde1060 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8249a94b80 10 client.26066 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562e5cb47c60 con 0x562e5cde1060 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.769+0000 7f8216ffd700 1 -- 172.21.15.49:0/1858180401 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562e5ccfbeb0 con 0x562e5cde1060 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f8216ffd700 1 -- 172.21.15.49:0/1858180401 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f822802e410 con 0x562e5cde1060 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f8249a94b80 10 client.26066 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f8249a94b80 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f8249a94b80 20 client.26066 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5cdf6a40 0x562e5cdf8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f8249a94b80 4 client.26066 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:21:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e5cd2ce10 con 0x562e5cdf6a40 2023-10-08T09:21:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.770+0000 7f82367fc700 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5cdf6a40 0x562e5cdf8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.771+0000 7f82367fc700 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5cdf6a40 0x562e5cdf8e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504bd232027bb16a server_cookie=990f37e608034307 in_seq=0 out_seq=0 2023-10-08T09:21:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.771+0000 7f8216ffd700 10 client.26066 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.771+0000 7f8216ffd700 1 -- 172.21.15.49:0/1858180401 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e5cd2ce10 con 0x562e5cdf6a40 2023-10-08T09:21:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.771+0000 7f8216ffd700 10 client.26066 handle_command_reply: tid=0 2023-10-08T09:21:14.776 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 client.26066 shutdown 2023-10-08T09:21:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde1060 msgr2=0x562e5cdeae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde1060 0x562e5cdeae00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82280196f0 tx=0x7f8228024e90 comp rx=0 tx=0).stop 2023-10-08T09:21:14.777 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 shutdown_connections 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5cdf6a40 0x562e5cdf8e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cde1060 0x562e5cdeae00 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f82280196f0 tx=0x7f8228024e90 comp rx=0 tx=0).stop 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1858180401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cdeb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 >> 172.21.15.49:0/1858180401 conn(0x562e5cd55260 msgr2=0x562e5cddb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.772+0000 7f8216ffd700 0 client.26066 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.776+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 shutdown_connections 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.776+0000 7f8249a94b80 1 -- 172.21.15.49:0/1858180401 wait complete. 2023-10-08T09:21:14.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.776+0000 7f8249a94b80 20 client.26066 trim_cache size 0 max 16384 2023-10-08T09:21:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.800+0000 7f8249a94b80 1 Processor -- start 2023-10-08T09:21:14.801 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.800+0000 7f8249a94b80 1 -- start start 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5cde5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f8249a94b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5cbb4920 con 0x562e5cde2800 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f8249a94b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5cd2ce10 con 0x562e5cdfd7c0 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f8249a94b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5ccfbeb0 con 0x562e5cde19c0 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5cde5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5cde5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41624/0 (socket says 172.21.15.49:41624) 2023-10-08T09:21:14.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 -- 172.21.15.49:0/4000114033 learned_addr learned my addr 172.21.15.49:0/4000114033 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:14.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f8236ffd700 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82367fc700 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 -- 172.21.15.49:0/4000114033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 msgr2=0x562e5cde5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 -- 172.21.15.49:0/4000114033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 msgr2=0x562e5cdf2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.801+0000 7f82377fe700 1 -- 172.21.15.49:0/4000114033 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e5cb85250 con 0x562e5cdfd7c0 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.802+0000 7f82367fc700 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.802+0000 7f8236ffd700 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.802+0000 7f82377fe700 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5cde5ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82300ca2d0 tx=0x7f8230099560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d79e352649053d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.802+0000 7f821e7fc700 1 -- 172.21.15.49:0/4000114033 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823009a280 con 0x562e5cdfd7c0 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.802+0000 7f821e7fc700 1 -- 172.21.15.49:0/4000114033 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f823000ce60 con 0x562e5cdfd7c0 2023-10-08T09:21:14.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.802+0000 7f821e7fc700 1 -- 172.21.15.49:0/4000114033 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230099de0 con 0x562e5cdfd7c0 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 -- 172.21.15.49:0/4000114033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 msgr2=0x562e5cde5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5cde5ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82300ca2d0 tx=0x7f8230099560 comp rx=0 tx=0).stop 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 -- 172.21.15.49:0/4000114033 shutdown_connections 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cde5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5cde5ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 --2- 172.21.15.49:0/4000114033 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cdf2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 -- 172.21.15.49:0/4000114033 >> 172.21.15.49:0/4000114033 conn(0x562e5cd55260 msgr2=0x562e5cdea340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 -- 172.21.15.49:0/4000114033 shutdown_connections 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 -- 172.21.15.49:0/4000114033 wait complete. 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.803+0000 7f8249a94b80 1 -- start start 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 1 Processor -- start 2023-10-08T09:21:14.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cded170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cded6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5ce64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5cdedd00 con 0x562e5cde2800 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5ce645c0 con 0x562e5cdfd7c0 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e5ce64700 con 0x562e5cde19c0 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8249a94b80 10 client.0 _ll_register_callbacks cb 0x7f82300097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cded170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cded170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39050/0 (socket says 172.21.15.49:39050) 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f8236ffd700 1 -- 172.21.15.49:0/1357539575 learned_addr learned my addr 172.21.15.49:0/1357539575 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f82377fe700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5ce64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.810 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.804+0000 7f82367fc700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cded6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8236ffd700 1 -- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 msgr2=0x562e5ce64080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5ce64080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8236ffd700 1 -- 172.21.15.49:0/1357539575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 msgr2=0x562e5cded6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cded6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8236ffd700 1 -- 172.21.15.49:0/1357539575 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82381a0f30 con 0x562e5cde19c0 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f82367fc700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cded6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f82377fe700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5ce64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8236ffd700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cded170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x562e5cdebca0 tx=0x7f822802ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97bef9b49734506 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8216ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8216ffd700 1 -- 172.21.15.49:0/1357539575 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822800d410 con 0x562e5cde19c0 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8216ffd700 1 -- 172.21.15.49:0/1357539575 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f822802ea90 con 0x562e5cde19c0 2023-10-08T09:21:14.811 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8228013040 con 0x562e5cde19c0 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8216ffd700 1 -- 172.21.15.49:0/1357539575 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8228024850 con 0x562e5cde19c0 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8249a94b80 10 client.26104 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562e5cb5d2b0 con 0x562e5cde19c0 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.805+0000 7f8216ffd700 1 -- 172.21.15.49:0/1357539575 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8228013040 con 0x562e5cde19c0 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8216ffd700 1 -- 172.21.15.49:0/1357539575 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f82280197a0 con 0x562e5cde19c0 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8249a94b80 10 client.26104 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8249a94b80 10 client.26104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8249a94b80 20 client.26104 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5ce65ce0 0x562e5ce68100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8249a94b80 4 client.26104 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e5cd2cb30 con 0x562e5ce65ce0 2023-10-08T09:21:14.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f82367fc700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5ce65ce0 0x562e5ce68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f8216ffd700 10 client.26104 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.806+0000 7f82367fc700 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5ce65ce0 0x562e5ce68100 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158348f8664e303 server_cookie=25ef5808f13cb7d0 in_seq=0 out_seq=0 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8216ffd700 1 -- 172.21.15.49:0/1357539575 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562e5cd2cb30 con 0x562e5ce65ce0 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8216ffd700 10 client.26104 handle_command_reply: tid=0 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 client.26104 shutdown 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 msgr2=0x562e5cded170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cded170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x562e5cdebca0 tx=0x7f822802ff80 comp rx=0 tx=0).stop 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 shutdown_connections 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x562e5ce65ce0 0x562e5ce68100 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562e5cde19c0 0x562e5cded170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562e5cdfd7c0 0x562e5ce64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 --2- 172.21.15.49:0/1357539575 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562e5cde2800 0x562e5cded6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8216ffd700 0 client.26104 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 >> 172.21.15.49:0/1357539575 conn(0x562e5cd55260 msgr2=0x562e5cde5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 shutdown_connections 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 1 -- 172.21.15.49:0/1357539575 wait complete. 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.807+0000 7f8249a94b80 20 client.26104 trim_cache size 0 max 16384 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:21:14.814 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:21:14.815 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:21:14.815 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:21:14.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:21:14.815 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:21:14.815 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:21:14.815 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:21:14.815 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:21:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f82200a1480 msgr2=0x7f82200a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f82200a1480 0x7f82200a3940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8238100700 tx=0x7f822800f040 comp rx=0 tx=0).stop 2023-10-08T09:21:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 msgr2=0x7f823819feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 0x7f823819feb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f823000ef30 tx=0x7f823000c3a0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 shutdown_connections 2023-10-08T09:21:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f82200a1480 0x7f82200a3940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82380ff6b0 0x7f8238198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8238101240 0x7f823819feb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 --2- 172.21.15.49:0/1014227859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f82381008a0 0x7f8238198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:14.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 >> 172.21.15.49:0/1014227859 conn(0x7f82380fac40 msgr2=0x7f8238103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:14.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 shutdown_connections 2023-10-08T09:21:14.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:14.808+0000 7f823d8ab700 1 -- 172.21.15.49:0/1014227859 wait complete. 2023-10-08T09:21:14.823 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:21:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:15 smithi132 ceph-mon[103389]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.1 MiB/s wr, 521 op/s 2023-10-08T09:21:16.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:15 smithi049 ceph-mon[99485]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.1 MiB/s wr, 521 op/s 2023-10-08T09:21:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:15 smithi184 ceph-mon[103675]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.1 MiB/s wr, 521 op/s 2023-10-08T09:21:18.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:17 smithi049 ceph-mon[99485]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 625 op/s 2023-10-08T09:21:18.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:17 smithi184 ceph-mon[103675]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 625 op/s 2023-10-08T09:21:18.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:17 smithi132 ceph-mon[103389]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.7 MiB/s wr, 625 op/s 2023-10-08T09:21:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.965+0000 7fbd62c90700 1 -- 172.21.15.49:0/1669392823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10a160 msgr2=0x7fbd5c10c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.965+0000 7fbd62c90700 1 --2- 172.21.15.49:0/1669392823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10a160 0x7fbd5c10c5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009990 tx=0x7fbd54009ca0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.965+0000 7fbd62c90700 1 -- 172.21.15.49:0/1669392823 shutdown_connections 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.965+0000 7fbd62c90700 1 --2- 172.21.15.49:0/1669392823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd5c10cb30 0x7fbd5c10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.965+0000 7fbd62c90700 1 --2- 172.21.15.49:0/1669392823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd5c1077f0 0x7fbd5c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.965+0000 7fbd62c90700 1 --2- 172.21.15.49:0/1669392823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10a160 0x7fbd5c10c5f0 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009990 tx=0x7fbd54009ca0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.965+0000 7fbd62c90700 1 -- 172.21.15.49:0/1669392823 >> 172.21.15.49:0/1669392823 conn(0x7fbd5c06d770 msgr2=0x7fbd5c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.966+0000 7fbd62c90700 1 -- 172.21.15.49:0/1669392823 shutdown_connections 2023-10-08T09:21:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.966+0000 7fbd62c90700 1 -- 172.21.15.49:0/1669392823 wait complete. 2023-10-08T09:21:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 Processor -- start 2023-10-08T09:21:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 -- start start 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10cb30 0x7fbd5c1a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd5c1a5dd0 0x7fbd5c1aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd5c1a6250 0x7fbd5c1ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1adb30 con 0x7fbd5c1a6250 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1adca0 con 0x7fbd5c1a5dd0 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.967+0000 7fbd62c90700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1ade60 con 0x7fbd5c10cb30 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd6122d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd5c1a6250 0x7fbd5c1ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd60a2c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10cb30 0x7fbd5c1a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd6122d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd5c1a6250 0x7fbd5c1ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52030/0 (socket says 172.21.15.49:52030) 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd6122d700 1 -- 172.21.15.49:0/771717895 learned_addr learned my addr 172.21.15.49:0/771717895 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd5bfff700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd5c1a5dd0 0x7fbd5c1aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd60a2c700 1 -- 172.21.15.49:0/771717895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd5c1a5dd0 msgr2=0x7fbd5c1aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd5c1a5dd0 0x7fbd5c1aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd60a2c700 1 -- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd5c1a6250 msgr2=0x7fbd5c1ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd5c1a6250 0x7fbd5c1ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd60a2c700 1 -- 172.21.15.49:0/771717895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540095f0 con 0x7fbd5c10cb30 2023-10-08T09:21:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.968+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10cb30 0x7fbd5c1a5890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0719a0 tx=0x7fbd4c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.969+0000 7fbd59ffb700 1 -- 172.21.15.49:0/771717895 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c004db0 con 0x7fbd5c10cb30 2023-10-08T09:21:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.969+0000 7fbd59ffb700 1 -- 172.21.15.49:0/771717895 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4c00f040 con 0x7fbd5c10cb30 2023-10-08T09:21:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.969+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5c1ae140 con 0x7fbd5c10cb30 2023-10-08T09:21:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.969+0000 7fbd59ffb700 1 -- 172.21.15.49:0/771717895 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c005800 con 0x7fbd5c10cb30 2023-10-08T09:21:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.969+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c1ae6d0 con 0x7fbd5c10cb30 2023-10-08T09:21:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.974+0000 7fbd59ffb700 1 -- 172.21.15.49:0/771717895 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbd4c00db20 con 0x7fbd5c10cb30 2023-10-08T09:21:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.978+0000 7fbd6ee79b80 1 Processor -- start 2023-10-08T09:21:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.978+0000 7fbd59ffb700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd480a1580 0x7fbd480a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd59ffb700 1 -- 172.21.15.49:0/771717895 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbd4c0cc7d0 con 0x7fbd5c10cb30 2023-10-08T09:21:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd6ee79b80 1 -- start start 2023-10-08T09:21:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613b5e40 0x5639613b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 0x5639613b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396131e400 con 0x5639613b67a0 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396130fbc0 con 0x5639613b72b0 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.979+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396130d4e0 con 0x5639613b5e40 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613b5e40 0x5639613b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613b5e40 0x5639613b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39072/0 (socket says 172.21.15.49:39072) 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 -- 172.21.15.49:0/2286087871 learned_addr learned my addr 172.21.15.49:0/2286087871 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 -- 172.21.15.49:0/2286087871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 msgr2=0x5639613b7730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/2286087871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613b7730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 -- 172.21.15.49:0/2286087871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 msgr2=0x5639613b6cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/2286087871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 0x5639613b6cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 -- 172.21.15.49:0/2286087871 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563961133d80 con 0x5639613b5e40 2023-10-08T09:21:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.980+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/2286087871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613b5e40 0x5639613b6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c099460 tx=0x7fbd4c009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ea300b0c4a5c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.981+0000 7fbd6122d700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd480a1580 0x7fbd480a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.981+0000 7fbd52ffd700 1 -- 172.21.15.49:0/2286087871 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c099e60 con 0x5639613b5e40 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.981+0000 7fbd52ffd700 1 -- 172.21.15.49:0/2286087871 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4c099760 con 0x5639613b5e40 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.981+0000 7fbd52ffd700 1 -- 172.21.15.49:0/2286087871 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c097b90 con 0x5639613b5e40 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.982+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/2286087871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613b5e40 msgr2=0x5639613b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.982+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/2286087871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613b5e40 0x5639613b6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c099460 tx=0x7fbd4c009e20 comp rx=0 tx=0).stop 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.984+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/2286087871 shutdown_connections 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.984+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/2286087871 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613b5e40 0x5639613b6260 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c099460 tx=0x7fbd4c009e20 comp rx=0 tx=0).stop 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.984+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/2286087871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.984+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/2286087871 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 0x5639613b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.986 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.984+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/2286087871 >> 172.21.15.49:0/2286087871 conn(0x56396132a200 msgr2=0x5639613b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.989+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/2286087871 shutdown_connections 2023-10-08T09:21:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.991+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/2286087871 wait complete. 2023-10-08T09:21:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.991+0000 7fbd6122d700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd480a1580 0x7fbd480a3a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c1a6650 tx=0x7fbd44009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.991+0000 7fbd6ee79b80 1 -- start start 2023-10-08T09:21:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.991+0000 7fbd6ee79b80 1 Processor -- start 2023-10-08T09:21:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613d8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563961308dd0 con 0x5639613b72b0 2023-10-08T09:21:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639612a56b0 con 0x5639613b67a0 2023-10-08T09:21:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639612a5350 con 0x5639613d6020 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6ee79b80 10 client.0 _ll_register_callbacks cb 0x5639611027a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6122d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613d8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6122d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613d8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39074/0 (socket says 172.21.15.49:39074) 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd6122d700 1 -- 172.21.15.49:0/46072147 learned_addr learned my addr 172.21.15.49:0/46072147 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.992+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd6122d700 1 -- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 msgr2=0x5639613d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd6122d700 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd6122d700 1 -- 172.21.15.49:0/46072147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 msgr2=0x5639613d1040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd6122d700 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd6122d700 1 -- 172.21.15.49:0/46072147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c1ae6d0 con 0x5639613d6020 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd6122d700 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613d8410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd44015480 tx=0x7fbd440047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4cffc31322f0178 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:21:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd3bfff700 1 -- 172.21.15.49:0/46072147 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44015aa0 con 0x5639613d6020 2023-10-08T09:21:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd3bfff700 1 -- 172.21.15.49:0/46072147 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbd44015c00 con 0x5639613d6020 2023-10-08T09:21:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.993+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639612d0e50 con 0x5639613d6020 2023-10-08T09:21:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd3bfff700 1 -- 172.21.15.49:0/46072147 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd44015dc0 con 0x5639613d6020 2023-10-08T09:21:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 10 client.26116 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56396111cc00 con 0x5639613d6020 2023-10-08T09:21:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd3bfff700 1 -- 172.21.15.49:0/46072147 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639612d0e50 con 0x5639613d6020 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd3bfff700 1 -- 172.21.15.49:0/46072147 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd4401f4a0 con 0x5639613d6020 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 20 client.26116 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639613dce50 0x5639613df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 4 client.26116 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.994+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563961301db0 con 0x5639613dce50 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.995+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639613dce50 0x5639613df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.995+0000 7fbd3bfff700 10 client.26116 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.995+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639613dce50 0x5639613df270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c11747fabd575c server_cookie=c2d849e50058e38c in_seq=0 out_seq=0 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd3bfff700 1 -- 172.21.15.49:0/46072147 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563961301db0 con 0x5639613dce50 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd3bfff700 10 client.26116 handle_command_reply: tid=0 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 client.26116 shutdown 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 msgr2=0x5639613d8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613d8410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbd44015480 tx=0x7fbd440047e0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 shutdown_connections 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639613dce50 0x5639613df270 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613d8410 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/46072147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 >> 172.21.15.49:0/46072147 conn(0x56396132a200 msgr2=0x5639613b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd3bfff700 0 client.26116 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 shutdown_connections 2023-10-08T09:21:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.996+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/46072147 wait complete. 2023-10-08T09:21:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:18.997+0000 7fbd6ee79b80 20 client.26116 trim_cache size 0 max 16384 2023-10-08T09:21:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.018+0000 7fbd6ee79b80 1 Processor -- start 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6ee79b80 1 -- start start 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613eca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639611898c0 con 0x5639613b72b0 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563961301db0 con 0x5639613b67a0 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639612d0e50 con 0x5639613d6020 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd5bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd5bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613eca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52050/0 (socket says 172.21.15.49:52050) 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd5bfff700 1 -- 172.21.15.49:0/3650991344 learned_addr learned my addr 172.21.15.49:0/3650991344 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6122d700 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6122d700 1 -- 172.21.15.49:0/3650991344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 msgr2=0x5639613b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6122d700 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6122d700 1 -- 172.21.15.49:0/3650991344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 msgr2=0x5639613eca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6122d700 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613eca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd6122d700 1 -- 172.21.15.49:0/3650991344 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396115a1f0 con 0x5639613d6020 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.019+0000 7fbd5bfff700 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613eca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:21:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.020+0000 7fbd6122d700 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613cf540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401f900 tx=0x7fbd44009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1ac6c01f740cb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.020+0000 7fbd537fe700 1 -- 172.21.15.49:0/3650991344 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4401fea0 con 0x5639613d6020 2023-10-08T09:21:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.020+0000 7fbd537fe700 1 -- 172.21.15.49:0/3650991344 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd44015db0 con 0x5639613d6020 2023-10-08T09:21:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.020+0000 7fbd537fe700 1 -- 172.21.15.49:0/3650991344 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44032ad0 con 0x5639613d6020 2023-10-08T09:21:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/3650991344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 msgr2=0x5639613cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613cf540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401f900 tx=0x7fbd44009eb0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/3650991344 shutdown_connections 2023-10-08T09:21:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x5639613cf540 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b67a0 0x5639613b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/3650991344 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b72b0 0x5639613eca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/3650991344 >> 172.21.15.49:0/3650991344 conn(0x56396132a200 msgr2=0x5639613e3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/3650991344 shutdown_connections 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.021+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/3650991344 wait complete. 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 -- start start 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 Processor -- start 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 0x5639613e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613ea1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x56396142f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639613ea840 con 0x5639613b67a0 2023-10-08T09:21:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396142f8c0 con 0x5639613b72b0 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396142fa00 con 0x5639613d6020 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.022+0000 7fbd6ee79b80 10 client.0 _ll_register_callbacks cb 0x5639611027a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613ea1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613ea1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41668/0 (socket says 172.21.15.49:41668) 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 -- 172.21.15.49:0/1556307684 learned_addr learned my addr 172.21.15.49:0/1556307684 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 0x5639613e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd6122d700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x56396142f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 -- 172.21.15.49:0/1556307684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 msgr2=0x56396142f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x56396142f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 -- 172.21.15.49:0/1556307684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 msgr2=0x5639613e9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 0x5639613e9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 -- 172.21.15.49:0/1556307684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c10da70 con 0x5639613b72b0 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd6122d700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x56396142f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd5bfff700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613ea1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd540093f0 tx=0x7fbd54005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcfd7de718b74e97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd3bfff700 1 -- 172.21.15.49:0/1556307684 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5401d030 con 0x5639613b72b0 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd4c099760 con 0x5639613b72b0 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.023+0000 7fbd3bfff700 1 -- 172.21.15.49:0/1556307684 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbd540050c0 con 0x5639613b72b0 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd3bfff700 1 -- 172.21.15.49:0/1556307684 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd54005280 con 0x5639613b72b0 2023-10-08T09:21:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd6ee79b80 10 client.26084 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56396111cc00 con 0x5639613b72b0 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd3bfff700 1 -- 172.21.15.49:0/1556307684 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4c099760 con 0x5639613b72b0 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd3bfff700 1 -- 172.21.15.49:0/1556307684 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd54023cc0 con 0x5639613b72b0 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd6ee79b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd6ee79b80 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd6ee79b80 20 client.26084 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639614314b0 0x5639614338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.024+0000 7fbd6ee79b80 4 client.26084 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.025+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563961301ad0 con 0x5639614314b0 2023-10-08T09:21:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.025+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639614314b0 0x5639614338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.025+0000 7fbd60a2c700 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639614314b0 0x5639614338d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=579cbb0b8706544a server_cookie=e47faf1a6386a808 in_seq=0 out_seq=0 2023-10-08T09:21:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.025+0000 7fbd3bfff700 10 client.26084 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd3bfff700 1 -- 172.21.15.49:0/1556307684 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563961301ad0 con 0x5639614314b0 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd3bfff700 10 client.26084 handle_command_reply: tid=0 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 client.26084 shutdown 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 msgr2=0x5639613ea1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613ea1f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbd540093f0 tx=0x7fbd54005ee0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 shutdown_connections 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5639614314b0 0x5639614338d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5639613d6020 0x56396142f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5639613b67a0 0x5639613e9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd3bfff700 0 client.26084 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 --2- 172.21.15.49:0/1556307684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5639613b72b0 0x5639613ea1f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 >> 172.21.15.49:0/1556307684 conn(0x56396132a200 msgr2=0x5639613cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 shutdown_connections 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 1 -- 172.21.15.49:0/1556307684 wait complete. 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.035+0000 7fbd6ee79b80 20 client.26084 trim_cache size 0 max 16384 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:21:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:21:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd480a1580 msgr2=0x7fbd480a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd480a1580 0x7fbd480a3a40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c1a6650 tx=0x7fbd44009190 comp rx=0 tx=0).stop 2023-10-08T09:21:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10cb30 msgr2=0x7fbd5c1a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10cb30 0x7fbd5c1a5890 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0719a0 tx=0x7fbd4c00d3b0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 shutdown_connections 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd480a1580 0x7fbd480a3a40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd5c10cb30 0x7fbd5c1a5890 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd5c1a6250 0x7fbd5c1ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 --2- 172.21.15.49:0/771717895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd5c1a5dd0 0x7fbd5c1aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 >> 172.21.15.49:0/771717895 conn(0x7fbd5c06d770 msgr2=0x7fbd5c108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 shutdown_connections 2023-10-08T09:21:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:19.036+0000 7fbd62c90700 1 -- 172.21.15.49:0/771717895 wait complete. 2023-10-08T09:21:19.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 355.0 s, 621.7 tps, lat 161.292 ms stddev 181.769 2023-10-08T09:21:20.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:19 smithi049 ceph-mon[99485]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 MiB/s wr, 527 op/s 2023-10-08T09:21:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:19 smithi184 ceph-mon[103675]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 MiB/s wr, 527 op/s 2023-10-08T09:21:20.302 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:19 smithi132 ceph-mon[103389]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.3 MiB/s wr, 527 op/s 2023-10-08T09:21:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:21 smithi049 ceph-mon[99485]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 554 op/s 2023-10-08T09:21:22.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:21 smithi184 ceph-mon[103675]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 554 op/s 2023-10-08T09:21:22.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:21 smithi132 ceph-mon[103389]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 554 op/s 2023-10-08T09:21:24.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:23 smithi049 ceph-mon[99485]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 580 op/s 2023-10-08T09:21:24.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:23 smithi184 ceph-mon[103675]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 580 op/s 2023-10-08T09:21:24.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:23 smithi132 ceph-mon[103389]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 580 op/s 2023-10-08T09:21:24.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 360.0 s, 660.7 tps, lat 152.811 ms stddev 182.152 2023-10-08T09:21:26.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:25 smithi049 ceph-mon[99485]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.9 MiB/s wr, 492 op/s 2023-10-08T09:21:26.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:25 smithi184 ceph-mon[103675]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.9 MiB/s wr, 492 op/s 2023-10-08T09:21:26.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:25 smithi132 ceph-mon[103389]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.9 MiB/s wr, 492 op/s 2023-10-08T09:21:27.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:26 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:21:27.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:26 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:21:27.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:26 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:21:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:27 smithi184 ceph-mon[103675]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 608 op/s 2023-10-08T09:21:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:21:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:21:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:21:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:27 smithi132 ceph-mon[103389]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 608 op/s 2023-10-08T09:21:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:21:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:21:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:21:28.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:27 smithi049 ceph-mon[99485]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.2 MiB/s wr, 608 op/s 2023-10-08T09:21:28.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:21:28.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:21:28.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:21:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 365.0 s, 586.0 tps, lat 171.309 ms stddev 189.710 2023-10-08T09:21:30.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:29 smithi184 ceph-mon[103675]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2023-10-08T09:21:30.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:29 smithi132 ceph-mon[103389]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2023-10-08T09:21:30.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:29 smithi049 ceph-mon[99485]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2023-10-08T09:21:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:31 smithi184 ceph-mon[103675]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-10-08T09:21:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.255+0000 7f1cfb33a700 1 -- 172.21.15.49:0/261517841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf4101200 msgr2=0x7f1cf4107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.255+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/261517841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf4101200 0x7f1cf4107ab0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40099c0 tx=0x7f1ce4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.255+0000 7f1cfb33a700 1 -- 172.21.15.49:0/261517841 shutdown_connections 2023-10-08T09:21:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.255+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/261517841 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf40ff670 0x7f1cf40ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.255+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/261517841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf4101200 0x7f1cf4107ab0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.255+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/261517841 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf4100860 0x7f1cf4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.255+0000 7f1cfb33a700 1 -- 172.21.15.49:0/261517841 >> 172.21.15.49:0/261517841 conn(0x7f1cf40fabe0 msgr2=0x7f1cf40fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.259+0000 7f1cfb33a700 1 -- 172.21.15.49:0/261517841 shutdown_connections 2023-10-08T09:21:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.270+0000 7f1cfb33a700 1 -- 172.21.15.49:0/261517841 wait complete. 2023-10-08T09:21:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.270+0000 7f1cfb33a700 1 Processor -- start 2023-10-08T09:21:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cfb33a700 1 -- start start 2023-10-08T09:21:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cfb33a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf40ff670 0x7f1cf41a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cfb33a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf4100860 0x7f1cf41a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cfb33a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf4101200 0x7f1cf41a8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cfb33a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41a90a0 con 0x7f1cf40ff670 2023-10-08T09:21:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cfb33a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41a9210 con 0x7f1cf4100860 2023-10-08T09:21:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cfb33a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf41a93d0 con 0x7f1cf4101200 2023-10-08T09:21:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf40ff670 0x7f1cf41a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf40ff670 0x7f1cf41a5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52068/0 (socket says 172.21.15.49:52068) 2023-10-08T09:21:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.271+0000 7f1cf90d6700 1 -- 172.21.15.49:0/3716280281 learned_addr learned my addr 172.21.15.49:0/3716280281 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.272+0000 7f1cf90d6700 1 -- 172.21.15.49:0/3716280281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf4101200 msgr2=0x7f1cf41a8ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:21:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.272+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf4101200 0x7f1cf41a8ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.272+0000 7f1cf90d6700 1 -- 172.21.15.49:0/3716280281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf4100860 msgr2=0x7f1cf41a5ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:21:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.272+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf4100860 0x7f1cf41a5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.272+0000 7f1cf90d6700 1 -- 172.21.15.49:0/3716280281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce40095f0 con 0x7f1cf40ff670 2023-10-08T09:21:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.272+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf40ff670 0x7f1cf41a5590 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000b550 tx=0x7f1cf000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.273+0000 7f1cea7fc700 1 -- 172.21.15.49:0/3716280281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0011830 con 0x7f1cf40ff670 2023-10-08T09:21:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.273+0000 7f1cea7fc700 1 -- 172.21.15.49:0/3716280281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf0011e50 con 0x7f1cf40ff670 2023-10-08T09:21:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.273+0000 7f1cea7fc700 1 -- 172.21.15.49:0/3716280281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000f2e0 con 0x7f1cf40ff670 2023-10-08T09:21:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.274+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf41a9650 con 0x7f1cf40ff670 2023-10-08T09:21:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.274+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf41a9b90 con 0x7f1cf40ff670 2023-10-08T09:21:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.278+0000 7f1d07523b80 1 Processor -- start 2023-10-08T09:21:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.281+0000 7f1d07523b80 1 -- start start 2023-10-08T09:21:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.281+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841ee40 0x56056841f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.281+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841f7a0 0x56056841fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.281+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5605684202b0 0x560568420730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.281+0000 7f1d07523b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560568387400 con 0x56056841ee40 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.281+0000 7f1d07523b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x560568378bc0 con 0x56056841f7a0 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.281+0000 7f1d07523b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605683764e0 con 0x5605684202b0 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841f7a0 0x56056841fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841f7a0 0x56056841fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41688/0 (socket says 172.21.15.49:41688) 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 -- 172.21.15.49:0/1660821717 learned_addr learned my addr 172.21.15.49:0/1660821717 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 -- 172.21.15.49:0/1660821717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5605684202b0 msgr2=0x560568420730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/1660821717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5605684202b0 0x560568420730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 -- 172.21.15.49:0/1660821717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841ee40 msgr2=0x56056841f260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/1660821717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841ee40 0x56056841f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.284+0000 7f1cf90d6700 1 -- 172.21.15.49:0/1660821717 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56056819cd80 con 0x56056841f7a0 2023-10-08T09:21:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.285+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/1660821717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841f7a0 0x56056841fcd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00d2b00 tx=0x7f1cf00d2e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982a03ea7c2888e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.286+0000 7f1cdf7fe700 1 -- 172.21.15.49:0/1660821717 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00d4dc0 con 0x56056841f7a0 2023-10-08T09:21:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.286+0000 7f1cdf7fe700 1 -- 172.21.15.49:0/1660821717 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf00de930 con 0x56056841f7a0 2023-10-08T09:21:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.286+0000 7f1cdf7fe700 1 -- 172.21.15.49:0/1660821717 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00d58b0 con 0x56056841f7a0 2023-10-08T09:21:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.287+0000 7f1cea7fc700 1 -- 172.21.15.49:0/3716280281 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1cf0011990 con 0x7f1cf40ff670 2023-10-08T09:21:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.287+0000 7f1d07523b80 1 -- 172.21.15.49:0/1660821717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841f7a0 msgr2=0x56056841fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.287+0000 7f1d07523b80 1 --2- 172.21.15.49:0/1660821717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841f7a0 0x56056841fcd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00d2b00 tx=0x7f1cf00d2e10 comp rx=0 tx=0).stop 2023-10-08T09:21:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.288+0000 7f1cea7fc700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ce00a1680 0x7f1ce00a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.288+0000 7f1cea7fc700 1 -- 172.21.15.49:0/3716280281 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1cf00cca20 con 0x7f1cf40ff670 2023-10-08T09:21:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 -- 172.21.15.49:0/1660821717 shutdown_connections 2023-10-08T09:21:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 --2- 172.21.15.49:0/1660821717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5605684202b0 0x560568420730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 --2- 172.21.15.49:0/1660821717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841ee40 0x56056841f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 --2- 172.21.15.49:0/1660821717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841f7a0 0x56056841fcd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 -- 172.21.15.49:0/1660821717 >> 172.21.15.49:0/1660821717 conn(0x560568393200 msgr2=0x5605684198a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1cf98d7700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ce00a1680 0x7f1ce00a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 -- 172.21.15.49:0/1660821717 shutdown_connections 2023-10-08T09:21:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 -- 172.21.15.49:0/1660821717 wait complete. 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1cf98d7700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ce00a1680 0x7f1ce00a3b40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4003790 tx=0x7f1ce4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.291+0000 7f1d07523b80 1 -- start start 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 1 Processor -- start 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841ee40 0x560568428870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841f7a0 0x560568428db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605684202b0 0x56056842fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x560568371dd0 con 0x5605684202b0 2023-10-08T09:21:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56056830e6b0 con 0x56056841ee40 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56056830e350 con 0x56056841f7a0 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1d07523b80 10 client.0 _ll_register_callbacks cb 0x56056816b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1cf98d7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605684202b0 0x56056842fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841ee40 0x560568428870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841ee40 0x560568428870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41692/0 (socket says 172.21.15.49:41692) 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.292+0000 7f1cf90d6700 1 -- 172.21.15.49:0/2824966061 learned_addr learned my addr 172.21.15.49:0/2824966061 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cf98d7700 1 -- 172.21.15.49:0/2824966061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841f7a0 msgr2=0x560568428db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cf98d7700 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841f7a0 0x560568428db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cf98d7700 1 -- 172.21.15.49:0/2824966061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841ee40 msgr2=0x560568428870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cf98d7700 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841ee40 0x560568428870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cf98d7700 1 -- 172.21.15.49:0/2824966061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf41a9b90 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cf98d7700 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605684202b0 0x56056842fdf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40038c0 tx=0x7f1ce4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf56e7c367bb6e2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cdcff9700 1 -- 172.21.15.49:0/2824966061 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4029070 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.293+0000 7f1cdcff9700 1 -- 172.21.15.49:0/2824966061 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1ce4004830 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560568339e50 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1cdcff9700 1 -- 172.21.15.49:0/2824966061 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ce401d070 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 10 client.16461 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x560568185c00 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1cdcff9700 1 -- 172.21.15.49:0/2824966061 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560568339e50 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1cdcff9700 1 -- 172.21.15.49:0/2824966061 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ce4003f30 con 0x5605684202b0 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 10 client.16461 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 20 client.16461 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5605684346f0 0x560568436b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 4 client.16461 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:21:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.294+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56056836adb0 con 0x5605684346f0 2023-10-08T09:21:32.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:31 smithi132 ceph-mon[103389]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-10-08T09:21:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.295+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5605684346f0 0x560568436b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.295+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5605684346f0 0x560568436b70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691114d69c9b7d85 server_cookie=2610c22694327cf9 in_seq=0 out_seq=0 2023-10-08T09:21:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.295+0000 7f1cdcff9700 10 client.16461 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1cdcff9700 1 -- 172.21.15.49:0/2824966061 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56056836adb0 con 0x5605684346f0 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1cdcff9700 10 client.16461 handle_command_reply: tid=0 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 client.16461 shutdown 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605684202b0 msgr2=0x56056842fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605684202b0 0x56056842fdf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40038c0 tx=0x7f1ce4009cd0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 shutdown_connections 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5605684346f0 0x560568436b70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5605684202b0 0x56056842fdf0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56056841ee40 0x560568428870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 --2- 172.21.15.49:0/2824966061 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841f7a0 0x560568428db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 >> 172.21.15.49:0/2824966061 conn(0x560568393200 msgr2=0x560568419010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1cdcff9700 0 client.16461 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 shutdown_connections 2023-10-08T09:21:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.296+0000 7f1d07523b80 1 -- 172.21.15.49:0/2824966061 wait complete. 2023-10-08T09:21:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.297+0000 7f1d07523b80 20 client.16461 trim_cache size 0 max 16384 2023-10-08T09:21:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 Processor -- start 2023-10-08T09:21:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 -- start start 2023-10-08T09:21:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x560568427d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x560568426780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605681f28c0 con 0x56056841f7a0 2023-10-08T09:21:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605684142b0 con 0x5605684202b0 2023-10-08T09:21:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.321+0000 7f1d07523b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605684273b0 con 0x56056841ee40 2023-10-08T09:21:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x560568427d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x560568427d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39122/0 (socket says 172.21.15.49:39122) 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 -- 172.21.15.49:0/286612586 learned_addr learned my addr 172.21.15.49:0/286612586 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 -- 172.21.15.49:0/286612586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 msgr2=0x560568426780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x560568426780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 -- 172.21.15.49:0/286612586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 msgr2=0x56056842fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf88d5700 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.322+0000 7f1cf90d6700 1 -- 172.21.15.49:0/286612586 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605681c31f0 con 0x56056841ee40 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.323+0000 7f1cf88d5700 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:21:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.323+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x560568427d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0096420 tx=0x7f1cf00cff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1307072818ee222 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.323+0000 7f1cdffff700 1 -- 172.21.15.49:0/286612586 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00ccc30 con 0x56056841ee40 2023-10-08T09:21:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.323+0000 7f1cdffff700 1 -- 172.21.15.49:0/286612586 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf0095cf0 con 0x56056841ee40 2023-10-08T09:21:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.323+0000 7f1cdffff700 1 -- 172.21.15.49:0/286612586 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf009db00 con 0x56056841ee40 2023-10-08T09:21:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 -- 172.21.15.49:0/286612586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 msgr2=0x560568427d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x560568427d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0096420 tx=0x7f1cf00cff90 comp rx=0 tx=0).stop 2023-10-08T09:21:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 -- 172.21.15.49:0/286612586 shutdown_connections 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x560568427d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x560568426780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 --2- 172.21.15.49:0/286612586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 -- 172.21.15.49:0/286612586 >> 172.21.15.49:0/286612586 conn(0x560568393200 msgr2=0x5605684275e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.324+0000 7f1d07523b80 1 -- 172.21.15.49:0/286612586 shutdown_connections 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.325+0000 7f1d07523b80 1 -- 172.21.15.49:0/286612586 wait complete. 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.325+0000 7f1d07523b80 1 -- start start 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.325+0000 7f1d07523b80 1 Processor -- start 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x56056842b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1d07523b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x56056842eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1d07523b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56056842f050 con 0x56056841f7a0 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1d07523b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56056842f1c0 con 0x5605684202b0 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1d07523b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56056842f380 con 0x56056841ee40 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1d07523b80 10 client.0 _ll_register_callbacks cb 0x7f1ce4005360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x56056842b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x56056842b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39126/0 (socket says 172.21.15.49:39126) 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 -- 172.21.15.49:0/119404037 learned_addr learned my addr 172.21.15.49:0/119404037 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf98d7700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x56056842eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf88d5700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 -- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 msgr2=0x56056842eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x56056842eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 -- 172.21.15.49:0/119404037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 msgr2=0x56056842bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 -- 172.21.15.49:0/119404037 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56056819cd80 con 0x56056841ee40 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf88d5700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf98d7700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x56056842eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cf90d6700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x56056842b6f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c00 tx=0x7f1cf00109c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=253d90de44ee23ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.326+0000 7f1cdcff9700 1 -- 172.21.15.49:0/119404037 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00994f0 con 0x56056841ee40 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1cdcff9700 1 -- 172.21.15.49:0/119404037 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1cf00b2230 con 0x56056841ee40 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1cf00de930 con 0x56056841ee40 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1cdcff9700 1 -- 172.21.15.49:0/119404037 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf0099310 con 0x56056841ee40 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 10 client.26128 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56056819b250 con 0x56056841ee40 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1cdcff9700 1 -- 172.21.15.49:0/119404037 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf00de930 con 0x56056841ee40 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1cdcff9700 1 -- 172.21.15.49:0/119404037 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1cf00960c0 con 0x56056841ee40 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 10 client.26128 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 20 client.26128 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560568488b20 0x56056848af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 4 client.26128 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56056836aad0 con 0x560568488b20 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.327+0000 7f1cf88d5700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560568488b20 0x56056848af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.328+0000 7f1cf88d5700 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560568488b20 0x56056848af40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1ff07b034b368d server_cookie=fda6ce6e2dde0dc3 in_seq=0 out_seq=0 2023-10-08T09:21:32.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.328+0000 7f1cdcff9700 10 client.26128 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.342+0000 7f1cdcff9700 1 -- 172.21.15.49:0/119404037 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56056836aad0 con 0x560568488b20 2023-10-08T09:21:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.342+0000 7f1cdcff9700 10 client.26128 handle_command_reply: tid=0 2023-10-08T09:21:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 client.26128 shutdown 2023-10-08T09:21:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 msgr2=0x56056842b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x56056842b6f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0000c00 tx=0x7f1cf00109c0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 shutdown_connections 2023-10-08T09:21:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560568488b20 0x56056848af40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56056841ee40 0x56056842b6f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5605684202b0 0x56056842eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 --2- 172.21.15.49:0/119404037 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56056841f7a0 0x56056842bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 >> 172.21.15.49:0/119404037 conn(0x560568393200 msgr2=0x560568427d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.343+0000 7f1cdcff9700 0 client.26128 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.344+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 shutdown_connections 2023-10-08T09:21:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.344+0000 7f1d07523b80 1 -- 172.21.15.49:0/119404037 wait complete. 2023-10-08T09:21:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.344+0000 7f1d07523b80 20 client.26128 trim_cache size 0 max 16384 2023-10-08T09:21:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:21:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:21:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:21:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ce00a1680 msgr2=0x7f1ce00a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ce00a1680 0x7f1ce00a3b40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4003790 tx=0x7f1ce4005fd0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf40ff670 msgr2=0x7f1cf41a5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf40ff670 0x7f1cf41a5590 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000b550 tx=0x7f1cf000b910 comp rx=0 tx=0).stop 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 shutdown_connections 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1ce00a1680 0x7f1ce00a3b40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1cf4101200 0x7f1cf41a8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1cf40ff670 0x7f1cf41a5590 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.345+0000 7f1cfb33a700 1 --2- 172.21.15.49:0/3716280281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cf4100860 0x7f1cf41a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.346+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 >> 172.21.15.49:0/3716280281 conn(0x7f1cf40fabe0 msgr2=0x7f1cf40fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.346+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 shutdown_connections 2023-10-08T09:21:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:32.346+0000 7f1cfb33a700 1 -- 172.21.15.49:0/3716280281 wait complete. 2023-10-08T09:21:32.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:31 smithi049 ceph-mon[99485]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-10-08T09:21:34.252 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:33 smithi184 ceph-mon[103675]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 MiB/s wr, 597 op/s 2023-10-08T09:21:34.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:33 smithi132 ceph-mon[103389]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 MiB/s wr, 597 op/s 2023-10-08T09:21:34.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 370.0 s, 518.0 tps, lat 192.127 ms stddev 213.093 2023-10-08T09:21:34.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:33 smithi049 ceph-mon[99485]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 MiB/s wr, 597 op/s 2023-10-08T09:21:36.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:35 smithi184 ceph-mon[103675]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 503 op/s 2023-10-08T09:21:36.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:35 smithi132 ceph-mon[103389]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 503 op/s 2023-10-08T09:21:36.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:35 smithi049 ceph-mon[99485]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.7 MiB/s wr, 503 op/s 2023-10-08T09:21:38.251 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:37 smithi184 ceph-mon[103675]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 598 op/s 2023-10-08T09:21:38.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:37 smithi132 ceph-mon[103389]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 598 op/s 2023-10-08T09:21:38.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:37 smithi049 ceph-mon[99485]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 598 op/s 2023-10-08T09:21:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 375.0 s, 568.7 tps, lat 177.977 ms stddev 180.144 2023-10-08T09:21:40.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:39 smithi184 ceph-mon[103675]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 MiB/s wr, 551 op/s 2023-10-08T09:21:40.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:39 smithi132 ceph-mon[103389]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 MiB/s wr, 551 op/s 2023-10-08T09:21:40.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:39 smithi049 ceph-mon[99485]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.7 MiB/s wr, 551 op/s 2023-10-08T09:21:42.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:42 smithi132 ceph-mon[103389]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 MiB/s wr, 585 op/s 2023-10-08T09:21:42.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:42 smithi049 ceph-mon[99485]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 MiB/s wr, 585 op/s 2023-10-08T09:21:42.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:42 smithi184 ceph-mon[103675]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.4 MiB/s wr, 585 op/s 2023-10-08T09:21:44.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:44 smithi132 ceph-mon[103389]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2023-10-08T09:21:44.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 380.0 s, 524.1 tps, lat 189.850 ms stddev 179.079 2023-10-08T09:21:44.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:44 smithi049 ceph-mon[99485]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2023-10-08T09:21:44.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:44 smithi184 ceph-mon[103675]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.9 MiB/s wr, 592 op/s 2023-10-08T09:21:44.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:21:45.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.052+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/2529602122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 msgr2=0x7f8e2810c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.052+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/2529602122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 0x7f8e2810c5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0098f0 tx=0x7f8e1c009c00 comp rx=0 tx=0).stop 2023-10-08T09:21:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.056+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/2529602122 shutdown_connections 2023-10-08T09:21:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.056+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/2529602122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e281077f0 0x7f8e28109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.056+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/2529602122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e2810cb30 0x7f8e2810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.056+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/2529602122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 0x7f8e2810c5f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.056+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/2529602122 >> 172.21.15.49:0/2529602122 conn(0x7f8e2806d790 msgr2=0x7f8e2806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.058+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/2529602122 shutdown_connections 2023-10-08T09:21:45.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.058+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/2529602122 wait complete. 2023-10-08T09:21:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.059+0000 7f8e2cfd1700 1 Processor -- start 2023-10-08T09:21:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2cfd1700 1 -- start start 2023-10-08T09:21:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2cfd1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e281077f0 0x7f8e281a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2cfd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 0x7f8e281a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.060 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2cfd1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e2810cb30 0x7f8e281acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2cfd1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e281ad190 con 0x7f8e281077f0 2023-10-08T09:21:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2cfd1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e281ad2d0 con 0x7f8e2810a160 2023-10-08T09:21:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2cfd1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e281ad460 con 0x7f8e2810cb30 2023-10-08T09:21:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e25d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 0x7f8e281a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.060+0000 7f8e2659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e281077f0 0x7f8e281a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.061 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e2659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e281077f0 0x7f8e281a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52120/0 (socket says 172.21.15.49:52120) 2023-10-08T09:21:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e2659c700 1 -- 172.21.15.49:0/1853270941 learned_addr learned my addr 172.21.15.49:0/1853270941 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e2810cb30 0x7f8e281acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e2659c700 1 -- 172.21.15.49:0/1853270941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e2810cb30 msgr2=0x7f8e281acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e2659c700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e2810cb30 0x7f8e281acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e2659c700 1 -- 172.21.15.49:0/1853270941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 msgr2=0x7f8e281a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.062 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e2659c700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 0x7f8e281a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.063 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.061+0000 7f8e2659c700 1 -- 172.21.15.49:0/1853270941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009520 con 0x7f8e281077f0 2023-10-08T09:21:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.062+0000 7f8e2659c700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e281077f0 0x7f8e281a5660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b840 tx=0x7f8e2000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.062+0000 7f8e177fe700 1 -- 172.21.15.49:0/1853270941 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000f840 con 0x7f8e281077f0 2023-10-08T09:21:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.062+0000 7f8e177fe700 1 -- 172.21.15.49:0/1853270941 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e2000fe60 con 0x7f8e281077f0 2023-10-08T09:21:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.063+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e281ad740 con 0x7f8e281077f0 2023-10-08T09:21:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.063+0000 7f8e177fe700 1 -- 172.21.15.49:0/1853270941 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000d3d0 con 0x7f8e281077f0 2023-10-08T09:21:45.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.063+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e281adc50 con 0x7f8e281077f0 2023-10-08T09:21:45.066 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.066+0000 7f8e177fe700 1 -- 172.21.15.49:0/1853270941 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8e2000d530 con 0x7f8e281077f0 2023-10-08T09:21:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.067+0000 7f8e177fe700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8e100a14d0 0x7f8e100a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.067+0000 7f8e177fe700 1 -- 172.21.15.49:0/1853270941 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8e200ccd40 con 0x7f8e281077f0 2023-10-08T09:21:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.067+0000 7f8e391bab80 1 Processor -- start 2023-10-08T09:21:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e391bab80 1 -- start start 2023-10-08T09:21:45.068 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 0x55e79fbad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad9c0 0x55e79fbae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbae800 0x55e79fbaec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e391bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fb15460 con 0x55e79fbae800 2023-10-08T09:21:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e391bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fb06c20 con 0x55e79fbad9c0 2023-10-08T09:21:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e391bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fb04540 con 0x55e79fbad060 2023-10-08T09:21:45.069 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e26d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbae800 0x55e79fbaec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e26d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbae800 0x55e79fbaec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52124/0 (socket says 172.21.15.49:52124) 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.068+0000 7f8e26d9d700 1 -- 172.21.15.49:0/771581633 learned_addr learned my addr 172.21.15.49:0/771581633 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad9c0 0x55e79fbae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e2659c700 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 0x55e79fbad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e26d9d700 1 -- 172.21.15.49:0/771581633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 msgr2=0x55e79fbad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 0x55e79fbad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e26d9d700 1 -- 172.21.15.49:0/771581633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad9c0 msgr2=0x55e79fbae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad9c0 0x55e79fbae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e26d9d700 1 -- 172.21.15.49:0/771581633 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e79f92ade0 con 0x55e79fbae800 2023-10-08T09:21:45.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.069+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbae800 0x55e79fbaec90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800bcb0 tx=0x7f8e1800f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4e030d1820505c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.070+0000 7f8e14ff9700 1 -- 172.21.15.49:0/771581633 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800fbe0 con 0x55e79fbae800 2023-10-08T09:21:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.070+0000 7f8e14ff9700 1 -- 172.21.15.49:0/771581633 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e1800fd40 con 0x55e79fbae800 2023-10-08T09:21:45.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.070+0000 7f8e14ff9700 1 -- 172.21.15.49:0/771581633 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e18011440 con 0x55e79fbae800 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.072+0000 7f8e391bab80 1 -- 172.21.15.49:0/771581633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbae800 msgr2=0x55e79fbaec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.072+0000 7f8e391bab80 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbae800 0x55e79fbaec90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800bcb0 tx=0x7f8e1800f360 comp rx=0 tx=0).stop 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.075+0000 7f8e391bab80 1 -- 172.21.15.49:0/771581633 shutdown_connections 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.075+0000 7f8e391bab80 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 0x55e79fbad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.075+0000 7f8e391bab80 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbae800 0x55e79fbaec90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.075+0000 7f8e391bab80 1 --2- 172.21.15.49:0/771581633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad9c0 0x55e79fbae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.075+0000 7f8e391bab80 1 -- 172.21.15.49:0/771581633 >> 172.21.15.49:0/771581633 conn(0x55e79fb21260 msgr2=0x55e79fba7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.079 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.076+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8e100a14d0 0x7f8e100a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.079+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8e100a14d0 0x7f8e100a3990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00fca0 tx=0x7f8e1c009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.079+0000 7f8e391bab80 1 -- 172.21.15.49:0/771581633 shutdown_connections 2023-10-08T09:21:45.081 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.079+0000 7f8e391bab80 1 -- 172.21.15.49:0/771581633 wait complete. 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 -- start start 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 Processor -- start 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbb7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79faffe30 con 0x55e79fbad9c0 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fa9c710 con 0x55e79fbad060 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fa9c3b0 con 0x55e79fbae800 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e391bab80 10 client.0 _ll_register_callbacks cb 0x55e79f8f9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e26d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e2659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.082 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e26d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39168/0 (socket says 172.21.15.49:39168) 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.080+0000 7f8e26d9d700 1 -- 172.21.15.49:0/147435609 learned_addr learned my addr 172.21.15.49:0/147435609 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbb7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e26d9d700 1 -- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 msgr2=0x55e79fbb6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e26d9d700 1 -- 172.21.15.49:0/147435609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 msgr2=0x55e79fbb7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbb7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e26d9d700 1 -- 172.21.15.49:0/147435609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1800b8e0 con 0x55e79fbae800 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbb7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbe380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e18005270 tx=0x7f8e1800fc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306508ce30b6aa7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.083 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.081+0000 7f8e0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:21:45.084 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.082+0000 7f8e0dffb700 1 -- 172.21.15.49:0/147435609 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180048f0 con 0x55e79fbae800 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.082+0000 7f8e0dffb700 1 -- 172.21.15.49:0/147435609 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8e18004a50 con 0x55e79fbae800 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.083+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e79fac7eb0 con 0x55e79fbae800 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.083+0000 7f8e0dffb700 1 -- 172.21.15.49:0/147435609 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e18004c10 con 0x55e79fbae800 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.083+0000 7f8e391bab80 10 client.26137 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.083+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e79f913c60 con 0x55e79fbae800 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e0dffb700 1 -- 172.21.15.49:0/147435609 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e79fac7eb0 con 0x55e79fbae800 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e0dffb700 1 -- 172.21.15.49:0/147435609 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e1802c070 con 0x55e79fbae800 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e391bab80 10 client.26137 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e391bab80 10 client.26137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e391bab80 20 client.26137 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:45.085 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e391bab80 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fbc2a80 0x55e79fbc4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e391bab80 4 client.26137 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:21:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.084+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e79faf8e10 con 0x55e79fbc2a80 2023-10-08T09:21:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.086+0000 7f8e2659c700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:21:45.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.086+0000 7f8e2659c700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fbc2a80 0x55e79fbc4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.087+0000 7f8e0dffb700 10 client.26137 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.087 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.087+0000 7f8e2659c700 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fbc2a80 0x55e79fbc4ea0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5004682d64335b server_cookie=29b0062ca00b3c6c in_seq=0 out_seq=0 2023-10-08T09:21:45.088 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.087+0000 7f8e0dffb700 1 -- 172.21.15.49:0/147435609 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e79faf8e10 con 0x55e79fbc2a80 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e0dffb700 10 client.26137 handle_command_reply: tid=0 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 client.26137 shutdown 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 msgr2=0x55e79fbbe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbe380 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8e18005270 tx=0x7f8e1800fc10 comp rx=0 tx=0).stop 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 shutdown_connections 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fbc2a80 0x55e79fbc4ea0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbe380 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e0dffb700 0 client.26137 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 --2- 172.21.15.49:0/147435609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbb7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 >> 172.21.15.49:0/147435609 conn(0x55e79fb21260 msgr2=0x55e79fba74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 shutdown_connections 2023-10-08T09:21:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 1 -- 172.21.15.49:0/147435609 wait complete. 2023-10-08T09:21:45.090 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.088+0000 7f8e391bab80 20 client.26137 trim_cache size 0 max 16384 2023-10-08T09:21:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 Processor -- start 2023-10-08T09:21:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 -- start start 2023-10-08T09:21:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 0x55e79fbc4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbbe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbae800 0x55e79fbb5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79f980920 con 0x55e79fbad9c0 2023-10-08T09:21:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fbbf500 con 0x55e79fbae800 2023-10-08T09:21:45.122 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.116+0000 7f8e391bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fba25a0 con 0x55e79fbad060 2023-10-08T09:21:45.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.125+0000 7f8e26d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbae800 0x55e79fbb5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.125+0000 7f8e26d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbae800 0x55e79fbb5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41752/0 (socket says 172.21.15.49:41752) 2023-10-08T09:21:45.125 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.125+0000 7f8e26d9d700 1 -- 172.21.15.49:0/564769809 learned_addr learned my addr 172.21.15.49:0/564769809 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.127+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbbe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.131+0000 7f8e26d9d700 1 -- 172.21.15.49:0/564769809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 msgr2=0x55e79fbc4e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:21:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.131+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 0x55e79fbc4e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.131+0000 7f8e26d9d700 1 -- 172.21.15.49:0/564769809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 msgr2=0x55e79fbbe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.135 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.131+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbbe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.131+0000 7f8e26d9d700 1 -- 172.21.15.49:0/564769809 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e79f951250 con 0x55e79fbae800 2023-10-08T09:21:45.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.143+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbae800 0x55e79fbb5fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e18019490 tx=0x7f8e1800f360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be1d915e160c9ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.156+0000 7f8e157fa700 1 -- 172.21.15.49:0/564769809 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180199a0 con 0x55e79fbae800 2023-10-08T09:21:45.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.156+0000 7f8e157fa700 1 -- 172.21.15.49:0/564769809 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e1801e070 con 0x55e79fbae800 2023-10-08T09:21:45.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.156+0000 7f8e157fa700 1 -- 172.21.15.49:0/564769809 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180118a0 con 0x55e79fbae800 2023-10-08T09:21:45.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.157+0000 7f8e391bab80 1 -- 172.21.15.49:0/564769809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbae800 msgr2=0x55e79fbb5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.157+0000 7f8e391bab80 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbae800 0x55e79fbb5fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e18019490 tx=0x7f8e1800f360 comp rx=0 tx=0).stop 2023-10-08T09:21:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.159+0000 7f8e391bab80 1 -- 172.21.15.49:0/564769809 shutdown_connections 2023-10-08T09:21:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.159+0000 7f8e391bab80 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbad060 0x55e79fbc4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.159+0000 7f8e391bab80 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbae800 0x55e79fbb5fc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.159+0000 7f8e391bab80 1 --2- 172.21.15.49:0/564769809 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbbe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.159+0000 7f8e391bab80 1 -- 172.21.15.49:0/564769809 >> 172.21.15.49:0/564769809 conn(0x55e79fb21260 msgr2=0x55e79fba3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.166+0000 7f8e391bab80 1 -- 172.21.15.49:0/564769809 shutdown_connections 2023-10-08T09:21:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.178+0000 7f8e391bab80 1 -- 172.21.15.49:0/564769809 wait complete. 2023-10-08T09:21:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.178+0000 7f8e391bab80 1 -- start start 2023-10-08T09:21:45.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.178+0000 7f8e391bab80 1 Processor -- start 2023-10-08T09:21:45.190 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.190+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.190+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.190+0000 7f8e391bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.190+0000 7f8e391bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fbbd850 con 0x55e79fbad9c0 2023-10-08T09:21:45.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.190+0000 7f8e391bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fbbd9c0 con 0x55e79fbad060 2023-10-08T09:21:45.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.190+0000 7f8e391bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79fbbdb80 con 0x55e79fbae800 2023-10-08T09:21:45.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.190+0000 7f8e391bab80 10 client.0 _ll_register_callbacks cb 0x7f8e20096070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:45.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.195+0000 7f8e2659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.195+0000 7f8e2659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41760/0 (socket says 172.21.15.49:41760) 2023-10-08T09:21:45.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.195+0000 7f8e2659c700 1 -- 172.21.15.49:0/4116721572 learned_addr learned my addr 172.21.15.49:0/4116721572 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.201 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.201+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.202+0000 7f8e26d9d700 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.203+0000 7f8e2659c700 1 -- 172.21.15.49:0/4116721572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 msgr2=0x55e79fbbd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.203+0000 7f8e2659c700 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.203+0000 7f8e2659c700 1 -- 172.21.15.49:0/4116721572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 msgr2=0x55e79fbba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.203+0000 7f8e2659c700 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.212 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.203+0000 7f8e2659c700 1 -- 172.21.15.49:0/4116721572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e79f913c60 con 0x55e79fbad060 2023-10-08T09:21:45.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.208+0000 7f8e2659c700 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb9db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000bf50 tx=0x7f8e2009df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c56a1b7e5fb24 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.208+0000 7f8e0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:21:45.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.215+0000 7f8e0dffb700 1 -- 172.21.15.49:0/4116721572 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20099220 con 0x55e79fbad060 2023-10-08T09:21:45.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.216+0000 7f8e0dffb700 1 -- 172.21.15.49:0/4116721572 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8e20065520 con 0x55e79fbad060 2023-10-08T09:21:45.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.219+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e1800fd40 con 0x55e79fbad060 2023-10-08T09:21:45.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.219+0000 7f8e0dffb700 1 -- 172.21.15.49:0/4116721572 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e200652b0 con 0x55e79fbad060 2023-10-08T09:21:45.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.223+0000 7f8e391bab80 10 client.26114 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:45.223 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.223+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e79f9292b0 con 0x55e79fbad060 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.226+0000 7f8e0dffb700 1 -- 172.21.15.49:0/4116721572 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e1800fd40 con 0x55e79fbad060 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.226+0000 7f8e0dffb700 1 -- 172.21.15.49:0/4116721572 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8e2000cbb0 con 0x55e79fbad060 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.227+0000 7f8e391bab80 10 client.26114 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.227+0000 7f8e391bab80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.227+0000 7f8e391bab80 20 client.26114 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.227+0000 7f8e391bab80 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fc28500 0x55e79fc2a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.228+0000 7f8e391bab80 4 client.26114 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:21:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.228+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e79faf8b30 con 0x55e79fc28500 2023-10-08T09:21:45.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.235+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fc28500 0x55e79fc2a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.236+0000 7f8e0dffb700 10 client.26114 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.241 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.236+0000 7f8e25d9b700 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fc28500 0x55e79fc2a920 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dadb6cd78c25fff server_cookie=f46225205d6229b5 in_seq=0 out_seq=0 2023-10-08T09:21:45.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.236+0000 7f8e0dffb700 1 -- 172.21.15.49:0/4116721572 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e79faf8b30 con 0x55e79fc28500 2023-10-08T09:21:45.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.236+0000 7f8e0dffb700 10 client.26114 handle_command_reply: tid=0 2023-10-08T09:21:45.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.236+0000 7f8e391bab80 1 client.26114 shutdown 2023-10-08T09:21:45.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.240+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 msgr2=0x55e79fbb9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.242 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.240+0000 7f8e391bab80 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb9db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000bf50 tx=0x7f8e2009df50 comp rx=0 tx=0).stop 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.241+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 shutdown_connections 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.241+0000 7f8e391bab80 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e79fc28500 0x55e79fc2a920 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.241+0000 7f8e391bab80 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e79fbae800 0x55e79fbbd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.241+0000 7f8e391bab80 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e79fbad060 0x55e79fbb9db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.241+0000 7f8e0dffb700 0 client.26114 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.241+0000 7f8e391bab80 1 --2- 172.21.15.49:0/4116721572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e79fbad9c0 0x55e79fbba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.241+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 >> 172.21.15.49:0/4116721572 conn(0x55e79fb21260 msgr2=0x55e79fbb5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.243 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.242+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 shutdown_connections 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.242+0000 7f8e391bab80 1 -- 172.21.15.49:0/4116721572 wait complete. 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.242+0000 7f8e391bab80 20 client.26114 trim_cache size 0 max 16384 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:21:45.244 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:21:45.245 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:21:45.245 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:21:45.245 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:21:45.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8e100a14d0 msgr2=0x7f8e100a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8e100a14d0 0x7f8e100a3990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00fca0 tx=0x7f8e1c009c00 comp rx=0 tx=0).stop 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e281077f0 msgr2=0x7f8e281a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e281077f0 0x7f8e281a5660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b840 tx=0x7f8e2000bb50 comp rx=0 tx=0).stop 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 shutdown_connections 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8e100a14d0 0x7f8e100a3990 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00fca0 tx=0x7f8e1c009c00 comp rx=0 tx=0).stop 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8e2810cb30 0x7f8e281acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e281077f0 0x7f8e281a5660 secure :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000b840 tx=0x7f8e2000bb50 comp rx=0 tx=0).stop 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 --2- 172.21.15.49:0/1853270941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8e2810a160 0x7f8e281a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.243+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 >> 172.21.15.49:0/1853270941 conn(0x7f8e2806d790 msgr2=0x7f8e28107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.244+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 shutdown_connections 2023-10-08T09:21:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:21:45.244+0000 7f8e2cfd1700 1 -- 172.21.15.49:0/1853270941 wait complete. 2023-10-08T09:21:45.275 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:21:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.553+0000 7f1577587700 1 -- 172.21.15.49:0/4253240215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570103b60 msgr2=0x7f1570103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.553+0000 7f1577587700 1 --2- 172.21.15.49:0/4253240215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570103b60 0x7f1570103fc0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f156c0099c0 tx=0x7f156c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.554+0000 7f1577587700 1 -- 172.21.15.49:0/4253240215 shutdown_connections 2023-10-08T09:21:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.554+0000 7f1577587700 1 --2- 172.21.15.49:0/4253240215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1570102970 0x7f1570102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.554+0000 7f1577587700 1 --2- 172.21.15.49:0/4253240215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1570104500 0x7f157010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.554+0000 7f1577587700 1 --2- 172.21.15.49:0/4253240215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570103b60 0x7f1570103fc0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.555+0000 7f1577587700 1 -- 172.21.15.49:0/4253240215 >> 172.21.15.49:0/4253240215 conn(0x7f15700fdf00 msgr2=0x7f1570100320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.569+0000 7f1577587700 1 -- 172.21.15.49:0/4253240215 shutdown_connections 2023-10-08T09:21:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.580+0000 7f1577587700 1 -- 172.21.15.49:0/4253240215 wait complete. 2023-10-08T09:21:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.580+0000 7f1577587700 1 Processor -- start 2023-10-08T09:21:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.580+0000 7f1577587700 1 -- start start 2023-10-08T09:21:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.581+0000 7f1577587700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1570102970 0x7f157010f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.581+0000 7f1577587700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1570103b60 0x7f157010fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.581+0000 7f1577587700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570104500 0x7f1570116db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.581+0000 7f1577587700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15701173b0 con 0x7f1570104500 2023-10-08T09:21:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.581+0000 7f1577587700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570117520 con 0x7f1570102970 2023-10-08T09:21:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.581+0000 7f1577587700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570117690 con 0x7f1570103b60 2023-10-08T09:21:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.586+0000 7f1575b24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570104500 0x7f1570116db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.586+0000 7f1575b24700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570104500 0x7f1570116db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52156/0 (socket says 172.21.15.49:52156) 2023-10-08T09:21:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.586+0000 7f1575b24700 1 -- 172.21.15.49:0/1634517302 learned_addr learned my addr 172.21.15.49:0/1634517302 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.587+0000 7f1575b24700 1 -- 172.21.15.49:0/1634517302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1570103b60 msgr2=0x7f157010fd50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:21:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.587+0000 7f1575b24700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1570103b60 0x7f157010fd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.587+0000 7f1575b24700 1 -- 172.21.15.49:0/1634517302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1570102970 msgr2=0x7f157010f810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.587+0000 7f1575b24700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1570102970 0x7f157010f810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.587+0000 7f1575b24700 1 -- 172.21.15.49:0/1634517302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0095f0 con 0x7f1570104500 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.587+0000 7f1575b24700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570104500 0x7f1570116db0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1568007ee0 tx=0x7f156800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.587+0000 7f15667fc700 1 -- 172.21.15.49:0/1634517302 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568017070 con 0x7f1570104500 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.588+0000 7f15667fc700 1 -- 172.21.15.49:0/1634517302 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156800f040 con 0x7f1570104500 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.588+0000 7f15667fc700 1 -- 172.21.15.49:0/1634517302 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15680057d0 con 0x7f1570104500 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.588+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1570117970 con 0x7f1570104500 2023-10-08T09:21:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.588+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1570117e30 con 0x7f1570104500 2023-10-08T09:21:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.592+0000 7f15667fc700 1 -- 172.21.15.49:0/1634517302 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1568005930 con 0x7f1570104500 2023-10-08T09:21:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.593+0000 7f15667fc700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f155c0a1680 0x7f155c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.593+0000 7f15667fc700 1 -- 172.21.15.49:0/1634517302 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f15680cd740 con 0x7f1570104500 2023-10-08T09:21:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.602+0000 7f1583770b80 1 Processor -- start 2023-10-08T09:21:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.605+0000 7f1575323700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f155c0a1680 0x7f155c0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.609+0000 7f1583770b80 1 -- start start 2023-10-08T09:21:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.609+0000 7f1583770b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.609+0000 7f1583770b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedb0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.609+0000 7f1583770b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb12b0 0x55c8cedb1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.609+0000 7f1583770b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ced18400 con 0x55c8cedafe40 2023-10-08T09:21:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.609+0000 7f1583770b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ced09bc0 con 0x55c8cedb12b0 2023-10-08T09:21:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.609+0000 7f1583770b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ced074e0 con 0x55c8cedb07a0 2023-10-08T09:21:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb12b0 0x55c8cedb1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb12b0 0x55c8cedb1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41776/0 (socket says 172.21.15.49:41776) 2023-10-08T09:21:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 -- 172.21.15.49:0/1050826455 learned_addr learned my addr 172.21.15.49:0/1050826455 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 -- 172.21.15.49:0/1050826455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 msgr2=0x55c8cedb0cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:21:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 --2- 172.21.15.49:0/1050826455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedb0cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 -- 172.21.15.49:0/1050826455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 msgr2=0x55c8cedb0260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 --2- 172.21.15.49:0/1050826455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb0260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.615+0000 7f1575b24700 1 -- 172.21.15.49:0/1050826455 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ceb2dd80 con 0x55c8cedb12b0 2023-10-08T09:21:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.617+0000 7f1575b24700 1 --2- 172.21.15.49:0/1050826455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb12b0 0x55c8cedb1730 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f156800d7b0 tx=0x7f156809a4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c63ef9bf8b38e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.629+0000 7f155b7fe700 1 -- 172.21.15.49:0/1050826455 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156809a6d0 con 0x55c8cedb12b0 2023-10-08T09:21:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.629+0000 7f155b7fe700 1 -- 172.21.15.49:0/1050826455 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156809e820 con 0x55c8cedb12b0 2023-10-08T09:21:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.629+0000 7f155b7fe700 1 -- 172.21.15.49:0/1050826455 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15680192c0 con 0x55c8cedb12b0 2023-10-08T09:21:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.630+0000 7f1583770b80 1 -- 172.21.15.49:0/1050826455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb12b0 msgr2=0x55c8cedb1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.630+0000 7f1583770b80 1 --2- 172.21.15.49:0/1050826455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb12b0 0x55c8cedb1730 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f156800d7b0 tx=0x7f156809a4a0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.635+0000 7f1583770b80 1 -- 172.21.15.49:0/1050826455 shutdown_connections 2023-10-08T09:21:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.635+0000 7f1583770b80 1 --2- 172.21.15.49:0/1050826455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb12b0 0x55c8cedb1730 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.635+0000 7f1583770b80 1 --2- 172.21.15.49:0/1050826455 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.635+0000 7f1583770b80 1 --2- 172.21.15.49:0/1050826455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedb0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.635+0000 7f1583770b80 1 -- 172.21.15.49:0/1050826455 >> 172.21.15.49:0/1050826455 conn(0x55c8ced24200 msgr2=0x55c8cedaa8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.636+0000 7f1575323700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f155c0a1680 0x7f155c0a3b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1570110d10 tx=0x7f1560006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.648+0000 7f1583770b80 1 -- 172.21.15.49:0/1050826455 shutdown_connections 2023-10-08T09:21:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.651+0000 7f1583770b80 1 -- 172.21.15.49:0/1050826455 wait complete. 2023-10-08T09:21:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.651+0000 7f1583770b80 1 -- start start 2023-10-08T09:21:45.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.651+0000 7f1583770b80 1 Processor -- start 2023-10-08T09:21:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.659+0000 7f1583770b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedafe40 0x55c8cedca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.659+0000 7f1583770b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedcae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.659+0000 7f1583770b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.659+0000 7f1583770b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ced02dd0 con 0x55c8cedb12b0 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.659+0000 7f1583770b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8cec9f6b0 con 0x55c8cedafe40 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.659+0000 7f1583770b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8cec9f350 con 0x55c8cedb07a0 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.659+0000 7f1583770b80 10 client.0 _ll_register_callbacks cb 0x55c8ceafc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedcae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedcae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39200/0 (socket says 172.21.15.49:39200) 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 -- 172.21.15.49:0/3270327857 learned_addr learned my addr 172.21.15.49:0/3270327857 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 -- 172.21.15.49:0/3270327857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedafe40 msgr2=0x55c8cedca8d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedafe40 0x55c8cedca8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 -- 172.21.15.49:0/3270327857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 msgr2=0x55c8cedd1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.660+0000 7f1574b22700 1 -- 172.21.15.49:0/3270327857 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1570117e30 con 0x55c8cedb07a0 2023-10-08T09:21:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.669+0000 7f1574b22700 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedcae10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f156c003720 tx=0x7f156c00bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567e6c8d321928b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.669+0000 7f1558ff9700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:21:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.680+0000 7f1558ff9700 1 -- 172.21.15.49:0/3270327857 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c01d070 con 0x55c8cedb07a0 2023-10-08T09:21:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.680+0000 7f1558ff9700 1 -- 172.21.15.49:0/3270327857 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f156c0038d0 con 0x55c8cedb07a0 2023-10-08T09:21:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.680+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8ceccae50 con 0x55c8cedb07a0 2023-10-08T09:21:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.681+0000 7f1558ff9700 1 -- 172.21.15.49:0/3270327857 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f156c00be90 con 0x55c8cedb07a0 2023-10-08T09:21:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.683+0000 7f1583770b80 10 client.26143 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:45.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.683+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c8ceb16c00 con 0x55c8cedb07a0 2023-10-08T09:21:45.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.683+0000 7f1558ff9700 1 -- 172.21.15.49:0/3270327857 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8ceccae50 con 0x55c8cedb07a0 2023-10-08T09:21:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.687+0000 7f1558ff9700 1 -- 172.21.15.49:0/3270327857 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f156c003a90 con 0x55c8cedb07a0 2023-10-08T09:21:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.689+0000 7f1583770b80 10 client.26143 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.689+0000 7f1583770b80 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.689+0000 7f1583770b80 20 client.26143 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:45.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.689+0000 7f1583770b80 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cedd6760 0x55c8cedd8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.689+0000 7f1583770b80 4 client.26143 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:21:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.689+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8cecfbdb0 con 0x55c8cedd6760 2023-10-08T09:21:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.689+0000 7f1575323700 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cedd6760 0x55c8cedd8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.690+0000 7f1575323700 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cedd6760 0x55c8cedd8b80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce78d1618241af7 server_cookie=9ef94f6207b2c896 in_seq=0 out_seq=0 2023-10-08T09:21:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.690+0000 7f1558ff9700 10 client.26143 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.691+0000 7f1558ff9700 1 -- 172.21.15.49:0/3270327857 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8cecfbdb0 con 0x55c8cedd6760 2023-10-08T09:21:45.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.691+0000 7f1558ff9700 10 client.26143 handle_command_reply: tid=0 2023-10-08T09:21:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 client.26143 shutdown 2023-10-08T09:21:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 msgr2=0x55c8cedcae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedcae10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f156c003720 tx=0x7f156c00bae0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 shutdown_connections 2023-10-08T09:21:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cedd6760 0x55c8cedd8b80 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedafe40 0x55c8cedca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 --2- 172.21.15.49:0/3270327857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb07a0 0x55c8cedcae10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 >> 172.21.15.49:0/3270327857 conn(0x55c8ced24200 msgr2=0x55c8cedaa010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1558ff9700 0 client.26143 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.692+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 shutdown_connections 2023-10-08T09:21:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.693+0000 7f1583770b80 1 -- 172.21.15.49:0/3270327857 wait complete. 2023-10-08T09:21:45.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.693+0000 7f1583770b80 20 client.26143 trim_cache size 0 max 16384 2023-10-08T09:21:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.718+0000 7f1583770b80 1 Processor -- start 2023-10-08T09:21:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.733+0000 7f1583770b80 1 -- start start 2023-10-08T09:21:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.733+0000 7f1583770b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.733+0000 7f1583770b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.733+0000 7f1583770b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 0x55c8cedb4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.733+0000 7f1583770b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ceb838c0 con 0x55c8cedafe40 2023-10-08T09:21:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.733+0000 7f1583770b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8cedc93c0 con 0x55c8cedb41e0 2023-10-08T09:21:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.733+0000 7f1583770b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8cedd2fd0 con 0x55c8cedb12b0 2023-10-08T09:21:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.736+0000 7f1575b24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 0x55c8cedb4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.740+0000 7f1574b22700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.741+0000 7f1575323700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.741+0000 7f1575323700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52172/0 (socket says 172.21.15.49:52172) 2023-10-08T09:21:45.747 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.741+0000 7f1575323700 1 -- 172.21.15.49:0/3207058620 learned_addr learned my addr 172.21.15.49:0/3207058620 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.745+0000 7f1574b22700 1 -- 172.21.15.49:0/3207058620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 msgr2=0x55c8cedb4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.745+0000 7f1574b22700 1 --2- 172.21.15.49:0/3207058620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 0x55c8cedb4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.745+0000 7f1574b22700 1 -- 172.21.15.49:0/3207058620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 msgr2=0x55c8cedb16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.745+0000 7f1574b22700 1 --2- 172.21.15.49:0/3207058620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.745+0000 7f1574b22700 1 -- 172.21.15.49:0/3207058620 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ceb541f0 con 0x55c8cedb12b0 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.745+0000 7f1574b22700 1 --2- 172.21.15.49:0/3207058620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098e0 tx=0x7f156c0037f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b844c9a87738ed0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.746+0000 7f155bfff700 1 -- 172.21.15.49:0/3207058620 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c035070 con 0x55c8cedb12b0 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.746+0000 7f155bfff700 1 -- 172.21.15.49:0/3207058620 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c00be90 con 0x55c8cedb12b0 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.746+0000 7f155bfff700 1 -- 172.21.15.49:0/3207058620 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c01d070 con 0x55c8cedb12b0 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.747+0000 7f1583770b80 1 -- 172.21.15.49:0/3207058620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb12b0 msgr2=0x55c8cedd1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.747+0000 7f1583770b80 1 --2- 172.21.15.49:0/3207058620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f156c0098e0 tx=0x7f156c0037f0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.747+0000 7f1583770b80 1 -- 172.21.15.49:0/3207058620 shutdown_connections 2023-10-08T09:21:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.747+0000 7f1583770b80 1 --2- 172.21.15.49:0/3207058620 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 0x55c8cedb4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.747+0000 7f1583770b80 1 --2- 172.21.15.49:0/3207058620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedafe40 0x55c8cedb16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.747+0000 7f1583770b80 1 --2- 172.21.15.49:0/3207058620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedb12b0 0x55c8cedd1e10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.747+0000 7f1583770b80 1 -- 172.21.15.49:0/3207058620 >> 172.21.15.49:0/3207058620 conn(0x55c8ced24200 msgr2=0x55c8cedb7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.756+0000 7f1583770b80 1 -- 172.21.15.49:0/3207058620 shutdown_connections 2023-10-08T09:21:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.761+0000 7f1583770b80 1 -- 172.21.15.49:0/3207058620 wait complete. 2023-10-08T09:21:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.761+0000 7f1583770b80 1 -- start start 2023-10-08T09:21:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.761+0000 7f1583770b80 1 Processor -- start 2023-10-08T09:21:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.765+0000 7f1583770b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedcdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.765+0000 7f1583770b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 0x55c8cedce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.765+0000 7f1583770b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedd0f50 0x55c8cedd13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.765+0000 7f1583770b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8cedd1910 con 0x55c8cedb12b0 2023-10-08T09:21:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.765+0000 7f1583770b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8cedd1a80 con 0x55c8cedb41e0 2023-10-08T09:21:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.765+0000 7f1583770b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8cedd1c40 con 0x55c8cedd0f50 2023-10-08T09:21:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.765+0000 7f1583770b80 10 client.0 _ll_register_callbacks cb 0x7f156c009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedcdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.767 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedcdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52178/0 (socket says 172.21.15.49:52178) 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 -- 172.21.15.49:0/3284315973 learned_addr learned my addr 172.21.15.49:0/3284315973 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575b24700 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedd0f50 0x55c8cedd13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 -- 172.21.15.49:0/3284315973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedd0f50 msgr2=0x55c8cedd13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedd0f50 0x55c8cedd13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 -- 172.21.15.49:0/3284315973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 msgr2=0x55c8cedce020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 0x55c8cedce020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.766+0000 7f1575323700 1 -- 172.21.15.49:0/3284315973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8ceb16c00 con 0x55c8cedb12b0 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.767+0000 7f1575323700 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedcdae0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1560009fd0 tx=0x7f1560004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10a924975ec567a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.767+0000 7f1558ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:21:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.767+0000 7f1558ff9700 1 -- 172.21.15.49:0/3284315973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156001a040 con 0x55c8cedb12b0 2023-10-08T09:21:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.767+0000 7f1558ff9700 1 -- 172.21.15.49:0/3284315973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1560021910 con 0x55c8cedb12b0 2023-10-08T09:21:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.767+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f156809e820 con 0x55c8cedb12b0 2023-10-08T09:21:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.767+0000 7f1558ff9700 1 -- 172.21.15.49:0/3284315973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1560021580 con 0x55c8cedb12b0 2023-10-08T09:21:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.771+0000 7f1583770b80 10 client.16488 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.771+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c8ceb2c250 con 0x55c8cedb12b0 2023-10-08T09:21:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.773+0000 7f1558ff9700 1 -- 172.21.15.49:0/3284315973 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156809e820 con 0x55c8cedb12b0 2023-10-08T09:21:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.773+0000 7f1558ff9700 1 -- 172.21.15.49:0/3284315973 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1560012ca0 con 0x55c8cedb12b0 2023-10-08T09:21:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.774+0000 7f1583770b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.774+0000 7f1583770b80 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.774+0000 7f1583770b80 20 client.16488 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.774+0000 7f1583770b80 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cee2b860 0x55c8cee2dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.774+0000 7f1583770b80 4 client.16488 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:21:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.774+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c8cecfbad0 con 0x55c8cee2b860 2023-10-08T09:21:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.785+0000 7f1574b22700 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cee2b860 0x55c8cee2dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.786+0000 7f1574b22700 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cee2b860 0x55c8cee2dc80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fc86ee4a0c6a37 server_cookie=11946d5488da2b9c in_seq=0 out_seq=0 2023-10-08T09:21:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.786+0000 7f1558ff9700 10 client.16488 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.836+0000 7f1558ff9700 1 -- 172.21.15.49:0/3284315973 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c8cecfbad0 con 0x55c8cee2b860 2023-10-08T09:21:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.836+0000 7f1558ff9700 10 client.16488 handle_command_reply: tid=0 2023-10-08T09:21:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.836+0000 7f1583770b80 1 client.16488 shutdown 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.837+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 msgr2=0x55c8cedcdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.837+0000 7f1583770b80 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedcdae0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1560009fd0 tx=0x7f1560004800 comp rx=0 tx=0).stop 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 shutdown_connections 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c8cee2b860 0x55c8cee2dc80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c8cedd0f50 0x55c8cedd13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c8cedb12b0 0x55c8cedcdae0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 --2- 172.21.15.49:0/3284315973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c8cedb41e0 0x55c8cedce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 >> 172.21.15.49:0/3284315973 conn(0x55c8ced24200 msgr2=0x55c8cedb7790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1558ff9700 0 client.16488 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 shutdown_connections 2023-10-08T09:21:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 1 -- 172.21.15.49:0/3284315973 wait complete. 2023-10-08T09:21:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.838+0000 7f1583770b80 20 client.16488 trim_cache size 0 max 16384 2023-10-08T09:21:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:21:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:21:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:21:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:21:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f155c0a1680 msgr2=0x7f155c0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f155c0a1680 0x7f155c0a3b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1570110d10 tx=0x7f1560006c60 comp rx=0 tx=0).stop 2023-10-08T09:21:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570104500 msgr2=0x7f1570116db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570104500 0x7f1570116db0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1568007ee0 tx=0x7f156800d3b0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 shutdown_connections 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f155c0a1680 0x7f155c0a3b40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1570104500 0x7f1570116db0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1570102970 0x7f157010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 --2- 172.21.15.49:0/1634517302 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1570103b60 0x7f157010fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 >> 172.21.15.49:0/1634517302 conn(0x7f15700fdf00 msgr2=0x7f1570104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 shutdown_connections 2023-10-08T09:21:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:45.839+0000 7f1577587700 1 -- 172.21.15.49:0/1634517302 wait complete. 2023-10-08T09:21:46.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:46 smithi049 ceph-mon[99485]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 MiB/s wr, 469 op/s 2023-10-08T09:21:46.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:46 smithi184 ceph-mon[103675]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 MiB/s wr, 469 op/s 2023-10-08T09:21:46.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:46 smithi132 ceph-mon[103389]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.7 MiB/s wr, 469 op/s 2023-10-08T09:21:48.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:48 smithi049 ceph-mon[99485]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.7 MiB/s wr, 564 op/s 2023-10-08T09:21:48.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:48 smithi184 ceph-mon[103675]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.7 MiB/s wr, 564 op/s 2023-10-08T09:21:48.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:48 smithi132 ceph-mon[103389]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.7 MiB/s wr, 564 op/s 2023-10-08T09:21:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 385.0 s, 578.8 tps, lat 171.827 ms stddev 181.005 2023-10-08T09:21:50.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:50 smithi049 ceph-mon[99485]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.6 MiB/s wr, 536 op/s 2023-10-08T09:21:50.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:50 smithi184 ceph-mon[103675]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.6 MiB/s wr, 536 op/s 2023-10-08T09:21:50.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:50 smithi132 ceph-mon[103389]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.6 MiB/s wr, 536 op/s 2023-10-08T09:21:52.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:52 smithi049 ceph-mon[99485]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2023-10-08T09:21:52.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:52 smithi184 ceph-mon[103675]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2023-10-08T09:21:52.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:52 smithi132 ceph-mon[103389]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2023-10-08T09:21:53.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:53 smithi049 ceph-mon[99485]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2023-10-08T09:21:53.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:53 smithi184 ceph-mon[103675]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2023-10-08T09:21:53.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:53 smithi132 ceph-mon[103389]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.5 MiB/s wr, 560 op/s 2023-10-08T09:21:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 390.0 s, 679.6 tps, lat 149.730 ms stddev 180.300 2023-10-08T09:21:54.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:54 smithi132 ceph-mon[103389]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 MiB/s wr, 454 op/s 2023-10-08T09:21:54.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:54 smithi049 ceph-mon[99485]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 MiB/s wr, 454 op/s 2023-10-08T09:21:54.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:54 smithi184 ceph-mon[103675]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 MiB/s wr, 454 op/s 2023-10-08T09:21:56.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:56 smithi049 ceph-mon[99485]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 598 op/s 2023-10-08T09:21:56.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:56 smithi184 ceph-mon[103675]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 598 op/s 2023-10-08T09:21:56.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:56 smithi132 ceph-mon[103389]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.6 MiB/s wr, 598 op/s 2023-10-08T09:21:58.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:21:58 smithi049 ceph-mon[99485]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 560 op/s 2023-10-08T09:21:58.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:21:58 smithi184 ceph-mon[103675]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 560 op/s 2023-10-08T09:21:58.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:21:58 smithi132 ceph-mon[103389]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 560 op/s 2023-10-08T09:21:59.313 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 395.0 s, 573.9 tps, lat 171.586 ms stddev 213.408 2023-10-08T09:21:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.354+0000 7fe7a6980700 1 -- 172.21.15.49:0/2142036831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a0100830 msgr2=0x7fe7a0100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.354+0000 7fe7a6980700 1 --2- 172.21.15.49:0/2142036831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a0100830 0x7fe7a0100c90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe7940099c0 tx=0x7fe794009cd0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 -- 172.21.15.49:0/2142036831 shutdown_connections 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 --2- 172.21.15.49:0/2142036831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a00ff640 0x7fe7a00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 --2- 172.21.15.49:0/2142036831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a01011d0 0x7fe7a0107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 --2- 172.21.15.49:0/2142036831 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a0100830 0x7fe7a0100c90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 -- 172.21.15.49:0/2142036831 >> 172.21.15.49:0/2142036831 conn(0x7fe7a00fabb0 msgr2=0x7fe7a00fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 -- 172.21.15.49:0/2142036831 shutdown_connections 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 -- 172.21.15.49:0/2142036831 wait complete. 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 Processor -- start 2023-10-08T09:21:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.355+0000 7fe7a6980700 1 -- start start 2023-10-08T09:21:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe7a6980700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a00ff640 0x7fe7a0196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe7a6980700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a0100830 0x7fe7a0196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe7a6980700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a01011d0 0x7fe7a019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe7a6980700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019e200 con 0x7fe7a00ff640 2023-10-08T09:21:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe7a6980700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019e340 con 0x7fe7a01011d0 2023-10-08T09:21:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe7a6980700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019e4d0 con 0x7fe7a0100830 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a00ff640 0x7fe7a0196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a00ff640 0x7fe7a0196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52190/0 (socket says 172.21.15.49:52190) 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 -- 172.21.15.49:0/564591554 learned_addr learned my addr 172.21.15.49:0/564591554 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a0100830 0x7fe7a0196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe7a4f1d700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a01011d0 0x7fe7a019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 -- 172.21.15.49:0/564591554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a0100830 msgr2=0x7fe7a0196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a0100830 0x7fe7a0196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 -- 172.21.15.49:0/564591554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a01011d0 msgr2=0x7fe7a019dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a01011d0 0x7fe7a019dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 -- 172.21.15.49:0/564591554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940095f0 con 0x7fe7a00ff640 2023-10-08T09:21:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a0100830 0x7fe7a0196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:21:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.356+0000 7fe79ffff700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a00ff640 0x7fe7a0196720 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe79800efc0 tx=0x7fe79800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.357+0000 7fe79d7fa700 1 -- 172.21.15.49:0/564591554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798003c00 con 0x7fe7a00ff640 2023-10-08T09:21:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.357+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a019e7b0 con 0x7fe7a00ff640 2023-10-08T09:21:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.357+0000 7fe79d7fa700 1 -- 172.21.15.49:0/564591554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79800f040 con 0x7fe7a00ff640 2023-10-08T09:21:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.357+0000 7fe79d7fa700 1 -- 172.21.15.49:0/564591554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798004590 con 0x7fe7a00ff640 2023-10-08T09:21:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.357+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a019ec70 con 0x7fe7a00ff640 2023-10-08T09:21:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.360+0000 7fe79d7fa700 1 -- 172.21.15.49:0/564591554 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe798003d60 con 0x7fe7a00ff640 2023-10-08T09:21:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.361+0000 7fe79d7fa700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7880a1500 0x7fe7880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.361+0000 7fe79d7fa700 1 -- 172.21.15.49:0/564591554 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe7980cd020 con 0x7fe7a00ff640 2023-10-08T09:21:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.362+0000 7fe7b2b69b80 1 Processor -- start 2023-10-08T09:21:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.362+0000 7fe7b2b69b80 1 -- start start 2023-10-08T09:21:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.362+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7880a1500 0x7fe7880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3c85260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3c85cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3c86730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3bed400 con 0x558be3c857a0 2023-10-08T09:21:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3bdebc0 con 0x558be3c84e40 2023-10-08T09:21:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3bdc4e0 con 0x558be3c862b0 2023-10-08T09:21:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3c85cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7a4f1d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3c86730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3c85cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52198/0 (socket says 172.21.15.49:52198) 2023-10-08T09:21:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe79f7fe700 1 -- 172.21.15.49:0/3782431493 learned_addr learned my addr 172.21.15.49:0/3782431493 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7880a1500 0x7fe7880a39c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe794004800 tx=0x7fe79400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7a4f1d700 1 -- 172.21.15.49:0/3782431493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 msgr2=0x558be3c85260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:21:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7a4f1d700 1 --2- 172.21.15.49:0/3782431493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3c85260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7a4f1d700 1 -- 172.21.15.49:0/3782431493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 msgr2=0x558be3c85cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7a4f1d700 1 --2- 172.21.15.49:0/3782431493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3c85cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7a4f1d700 1 -- 172.21.15.49:0/3782431493 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be3a02d80 con 0x558be3c862b0 2023-10-08T09:21:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.363+0000 7fe7a4f1d700 1 --2- 172.21.15.49:0/3782431493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3c86730 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe790009c10 tx=0x7fe790009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e008fc6adbc72e39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7867fc700 1 -- 172.21.15.49:0/3782431493 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790004310 con 0x558be3c862b0 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7867fc700 1 -- 172.21.15.49:0/3782431493 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe790011e30 con 0x558be3c862b0 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7867fc700 1 -- 172.21.15.49:0/3782431493 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790005020 con 0x558be3c862b0 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3782431493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 msgr2=0x558be3c86730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3782431493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3c86730 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe790009c10 tx=0x7fe790009f20 comp rx=0 tx=0).stop 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3782431493 shutdown_connections 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3782431493 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3c86730 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3782431493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3c85260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3782431493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3c85cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.364+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3782431493 >> 172.21.15.49:0/3782431493 conn(0x558be3bf9200 msgr2=0x558be3c7f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3782431493 shutdown_connections 2023-10-08T09:21:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3782431493 wait complete. 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 -- start start 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 Processor -- start 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3c9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c857a0 0x558be3c9fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.365+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3bd7dd0 con 0x558be3c84e40 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3b746b0 con 0x558be3c857a0 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3b74350 con 0x558be3c862b0 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe7b2b69b80 10 client.0 _ll_register_callbacks cb 0x558be39d17a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3c9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3c9f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52204/0 (socket says 172.21.15.49:52204) 2023-10-08T09:21:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 -- 172.21.15.49:0/3564110574 learned_addr learned my addr 172.21.15.49:0/3564110574 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe7a4f1d700 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 -- 172.21.15.49:0/3564110574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 msgr2=0x558be3ca6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 -- 172.21.15.49:0/3564110574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c857a0 msgr2=0x558be3c9fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c857a0 0x558be3c9fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 -- 172.21.15.49:0/3564110574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7900098c0 con 0x558be3c84e40 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe79ffff700 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3c9f8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe7980964b0 tx=0x7fe798099af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25bd1ad0ad8e726 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe77f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3564110574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79809a410 con 0x558be3c84e40 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3564110574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe79809a6b0 con 0x558be3c84e40 2023-10-08T09:21:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.366+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558be3b9fe50 con 0x558be3c84e40 2023-10-08T09:21:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.367+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3564110574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe798099fb0 con 0x558be3c84e40 2023-10-08T09:21:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.367+0000 7fe7b2b69b80 10 client.16500 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.367+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558be39ebc00 con 0x558be3c84e40 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3564110574 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe798063080 con 0x558be3c84e40 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3564110574 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe798099530 con 0x558be3c84e40 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe7b2b69b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe7b2b69b80 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe7b2b69b80 20 client.16500 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3cab6b0 0x558be3cadad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe7b2b69b80 4 client.16500 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558be3bd0db0 con 0x558be3cab6b0 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.370+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3cab6b0 0x558be3cadad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.371+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3cab6b0 0x558be3cadad0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47789a505fd4db08 server_cookie=b55154f90800c227 in_seq=0 out_seq=0 2023-10-08T09:21:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.371+0000 7fe77f7fe700 10 client.16500 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.371+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3564110574 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558be3bd0db0 con 0x558be3cab6b0 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe77f7fe700 10 client.16500 handle_command_reply: tid=0 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 client.16500 shutdown 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 msgr2=0x558be3c9f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3c9f8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe7980964b0 tx=0x7fe798099af0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 shutdown_connections 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3cab6b0 0x558be3cadad0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3c9f8f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3564110574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c857a0 0x558be3c9fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 >> 172.21.15.49:0/3564110574 conn(0x558be3bf9200 msgr2=0x558be3c7f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe77f7fe700 0 client.16500 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 shutdown_connections 2023-10-08T09:21:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3564110574 wait complete. 2023-10-08T09:21:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.372+0000 7fe7b2b69b80 20 client.16500 trim_cache size 0 max 16384 2023-10-08T09:21:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 Processor -- start 2023-10-08T09:21:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 -- start start 2023-10-08T09:21:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3ca7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c857a0 0x558be3ca6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c862b0 0x558be3c86690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.398 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3a588c0 con 0x558be3c84e40 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3bd0db0 con 0x558be3c862b0 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.395+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3c7a2b0 con 0x558be3c857a0 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3ca7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c857a0 0x558be3ca6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3ca7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52210/0 (socket says 172.21.15.49:52210) 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c857a0 0x558be3ca6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39246/0 (socket says 172.21.15.49:39246) 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79ffff700 1 -- 172.21.15.49:0/3256250215 learned_addr learned my addr 172.21.15.49:0/3256250215 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe7a4f1d700 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c862b0 0x558be3c86690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 -- 172.21.15.49:0/3256250215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c862b0 msgr2=0x558be3c86690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c862b0 0x558be3c86690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 -- 172.21.15.49:0/3256250215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 msgr2=0x558be3ca7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3ca7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 -- 172.21.15.49:0/3256250215 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558be3a291f0 con 0x558be3c857a0 2023-10-08T09:21:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c857a0 0x558be3ca6d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe79400a010 tx=0x7fe79402a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aef4ea5e66bcdb78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe786ffd700 1 -- 172.21.15.49:0/3256250215 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794034130 con 0x558be3c857a0 2023-10-08T09:21:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe786ffd700 1 -- 172.21.15.49:0/3256250215 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe794004cf0 con 0x558be3c857a0 2023-10-08T09:21:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.396+0000 7fe786ffd700 1 -- 172.21.15.49:0/3256250215 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794005380 con 0x558be3c857a0 2023-10-08T09:21:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.397+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3256250215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c857a0 msgr2=0x558be3ca6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.397+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c857a0 0x558be3ca6d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe79400a010 tx=0x7fe79402a000 comp rx=0 tx=0).stop 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.397+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3256250215 shutdown_connections 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.397+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c862b0 0x558be3c86690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.397+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c84e40 0x558be3ca7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.397+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3256250215 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c857a0 0x558be3ca6d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.397+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3256250215 >> 172.21.15.49:0/3256250215 conn(0x558be3bf9200 msgr2=0x558be3cb2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.398+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3256250215 shutdown_connections 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.398+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3256250215 wait complete. 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.398+0000 7fe7b2b69b80 1 -- start start 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 1 Processor -- start 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3ca4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3ca4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3ca5ef0 con 0x558be3c857a0 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3ca6060 con 0x558be3c84e40 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558be3ca6220 con 0x558be3c862b0 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe7b2b69b80 10 client.0 _ll_register_callbacks cb 0x7fe798095c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.399+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3ca4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe7a4f1d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3ca4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41830/0 (socket says 172.21.15.49:41830) 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe7a4f1d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39254/0 (socket says 172.21.15.49:39254) 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79ffff700 1 -- 172.21.15.49:0/3786091129 learned_addr learned my addr 172.21.15.49:0/3786091129 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:21:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3ca4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79ffff700 1 -- 172.21.15.49:0/3786091129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 msgr2=0x558be3ca59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79ffff700 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79ffff700 1 -- 172.21.15.49:0/3786091129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 msgr2=0x558be3ca4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79ffff700 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3ca4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.400+0000 7fe79ffff700 1 -- 172.21.15.49:0/3786091129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe794020040 con 0x558be3c84e40 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe79ffff700 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3ca4500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c6f0 tx=0x7fe7980a2270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a23bc4457c32bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe77f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3786091129 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980996b0 con 0x558be3c84e40 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3786091129 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe798099fb0 con 0x558be3c84e40 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe790011e30 con 0x558be3c84e40 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3786091129 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7980a2340 con 0x558be3c84e40 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe7b2b69b80 10 client.26132 fetch_fsmap learned FSMap version 40 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558be3a01250 con 0x558be3c84e40 2023-10-08T09:21:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3786091129 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe790011e30 con 0x558be3c84e40 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3786091129 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe798004ad0 con 0x558be3c84e40 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe7b2b69b80 10 client.26132 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe7b2b69b80 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.401+0000 7fe7b2b69b80 20 client.26132 populate_metadata read hostname 'smithi049' 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.402+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3d082e0 0x558be3d0a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.402+0000 7fe7b2b69b80 4 client.26132 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.402+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558be3bd0ad0 con 0x558be3d082e0 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.402+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3d082e0 0x558be3d0a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.402+0000 7fe79f7fe700 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3d082e0 0x558be3d0a700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb119ad21579fe6 server_cookie=40e73551931f673b in_seq=0 out_seq=0 2023-10-08T09:21:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.402+0000 7fe77f7fe700 10 client.26132 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.413+0000 7fe77f7fe700 1 -- 172.21.15.49:0/3786091129 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558be3bd0ad0 con 0x558be3d082e0 2023-10-08T09:21:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.413+0000 7fe77f7fe700 10 client.26132 handle_command_reply: tid=0 2023-10-08T09:21:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.413+0000 7fe7b2b69b80 1 client.26132 shutdown 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 msgr2=0x558be3ca4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3ca4500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c6f0 tx=0x7fe7980a2270 comp rx=0 tx=0).stop 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 shutdown_connections 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558be3d082e0 0x558be3d0a700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558be3c862b0 0x558be3ca59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe77f7fe700 0 client.26132 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558be3c84e40 0x558be3ca4500 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 --2- 172.21.15.49:0/3786091129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558be3c857a0 0x558be3ca4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 >> 172.21.15.49:0/3786091129 conn(0x558be3bf9200 msgr2=0x558be3c89220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 shutdown_connections 2023-10-08T09:21:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 1 -- 172.21.15.49:0/3786091129 wait complete. 2023-10-08T09:21:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.415+0000 7fe7b2b69b80 20 client.26132 trim_cache size 0 max 16384 2023-10-08T09:21:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:21:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:21:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:21:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:21:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7880a1500 msgr2=0x7fe7880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7880a1500 0x7fe7880a39c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe794004800 tx=0x7fe79400b540 comp rx=0 tx=0).stop 2023-10-08T09:21:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a00ff640 msgr2=0x7fe7a0196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:21:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a00ff640 0x7fe7a0196720 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe79800efc0 tx=0x7fe79800c3a0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 shutdown_connections 2023-10-08T09:21:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7880a1500 0x7fe7880a39c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a01011d0 0x7fe7a019dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a00ff640 0x7fe7a0196720 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 --2- 172.21.15.49:0/564591554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a0100830 0x7fe7a0196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:21:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 >> 172.21.15.49:0/564591554 conn(0x7fe7a00fabb0 msgr2=0x7fe7a00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:21:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 shutdown_connections 2023-10-08T09:21:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:21:59.416+0000 7fe7a6980700 1 -- 172.21.15.49:0/564591554 wait complete. 2023-10-08T09:22:00.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:00 smithi132 ceph-mon[103389]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 563 op/s 2023-10-08T09:22:00.688 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:00 smithi049 ceph-mon[99485]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 563 op/s 2023-10-08T09:22:00.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:00 smithi184 ceph-mon[103675]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 563 op/s 2023-10-08T09:22:03.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:02 smithi132 ceph-mon[103389]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 602 op/s 2023-10-08T09:22:03.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:02 smithi049 ceph-mon[99485]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 602 op/s 2023-10-08T09:22:03.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:02 smithi184 ceph-mon[103675]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.9 MiB/s wr, 602 op/s 2023-10-08T09:22:04.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 400.0 s, 523.2 tps, lat 178.933 ms stddev 182.804 2023-10-08T09:22:04.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:04 smithi132 ceph-mon[103389]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 495 op/s 2023-10-08T09:22:04.721 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:04 smithi049 ceph-mon[99485]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 495 op/s 2023-10-08T09:22:04.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:04 smithi184 ceph-mon[103675]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 495 op/s 2023-10-08T09:22:06.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:06 smithi132 ceph-mon[103389]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 723 op/s 2023-10-08T09:22:06.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:06 smithi049 ceph-mon[99485]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 723 op/s 2023-10-08T09:22:06.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:06 smithi184 ceph-mon[103675]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 723 op/s 2023-10-08T09:22:08.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:08 smithi132 ceph-mon[103389]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2023-10-08T09:22:08.661 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:08 smithi049 ceph-mon[99485]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2023-10-08T09:22:08.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:08 smithi184 ceph-mon[103675]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.0 MiB/s wr, 745 op/s 2023-10-08T09:22:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 405.0 s, 652.1 tps, lat 163.768 ms stddev 206.864 2023-10-08T09:22:10.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:10 smithi132 ceph-mon[103389]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 753 op/s 2023-10-08T09:22:10.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:10 smithi049 ceph-mon[99485]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 753 op/s 2023-10-08T09:22:10.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:10 smithi184 ceph-mon[103675]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.1 MiB/s wr, 753 op/s 2023-10-08T09:22:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.878+0000 7f61a1fb5700 1 -- 172.21.15.49:0/1267355412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c10cb30 msgr2=0x7f619c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.878+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/1267355412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c10cb30 0x7f619c10cfb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.883+0000 7f61a1fb5700 1 -- 172.21.15.49:0/1267355412 shutdown_connections 2023-10-08T09:22:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.883+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/1267355412 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619c1077f0 0x7f619c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.883+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/1267355412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c10cb30 0x7f619c10cfb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.883+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/1267355412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10a160 0x7f619c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.883+0000 7f61a1fb5700 1 -- 172.21.15.49:0/1267355412 >> 172.21.15.49:0/1267355412 conn(0x7f619c06d770 msgr2=0x7f619c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.884+0000 7f61a1fb5700 1 -- 172.21.15.49:0/1267355412 shutdown_connections 2023-10-08T09:22:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.884+0000 7f61a1fb5700 1 -- 172.21.15.49:0/1267355412 wait complete. 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 Processor -- start 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 -- start start 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c1077f0 0x7f619c1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619c10a160 0x7f619c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10cb30 0x7f619c1aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1ad010 con 0x7f619c10cb30 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1ad150 con 0x7f619c1077f0 2023-10-08T09:22:12.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f61a1fb5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1ad2e0 con 0x7f619c10a160 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c1077f0 0x7f619c1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f619bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10cb30 0x7f619c1aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c1077f0 0x7f619c1a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41842/0 (socket says 172.21.15.49:41842) 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f619bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10cb30 0x7f619c1aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52232/0 (socket says 172.21.15.49:52232) 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.885+0000 7f619b7fe700 1 -- 172.21.15.49:0/3525851330 learned_addr learned my addr 172.21.15.49:0/3525851330 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f619bfff700 1 -- 172.21.15.49:0/3525851330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619c10a160 msgr2=0x7f619c1a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f619bfff700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619c10a160 0x7f619c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f619bfff700 1 -- 172.21.15.49:0/3525851330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c1077f0 msgr2=0x7f619c1a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f619bfff700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c1077f0 0x7f619c1a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f619bfff700 1 -- 172.21.15.49:0/3525851330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194009d00 con 0x7f619c10cb30 2023-10-08T09:22:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f619b7fe700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c1077f0 0x7f619c1a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f619bfff700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10cb30 0x7f619c1aca40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f619400b070 tx=0x7f619400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f6198ff9700 1 -- 172.21.15.49:0/3525851330 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619400e040 con 0x7f619c10cb30 2023-10-08T09:22:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f6198ff9700 1 -- 172.21.15.49:0/3525851330 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61940094c0 con 0x7f619c10cb30 2023-10-08T09:22:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c1ad560 con 0x7f619c10cb30 2023-10-08T09:22:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f6198ff9700 1 -- 172.21.15.49:0/3525851330 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61940052c0 con 0x7f619c10cb30 2023-10-08T09:22:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.886+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c1ad9e0 con 0x7f619c10cb30 2023-10-08T09:22:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.888+0000 7f6198ff9700 1 -- 172.21.15.49:0/3525851330 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f619401d470 con 0x7f619c10cb30 2023-10-08T09:22:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.889+0000 7f6198ff9700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61840a1480 0x7f61840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.889+0000 7f619b7fe700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61840a1480 0x7f61840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.889+0000 7f6198ff9700 1 -- 172.21.15.49:0/3525851330 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f61940ce6c0 con 0x7f619c10cb30 2023-10-08T09:22:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 Processor -- start 2023-10-08T09:22:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f619b7fe700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61840a1480 0x7f61840a3940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6190009ea0 tx=0x7f6190009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 -- start start 2023-10-08T09:22:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e569b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e569bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e569c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e5603400 con 0x5635e569ae40 2023-10-08T09:22:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e55f4bc0 con 0x5635e569b7a0 2023-10-08T09:22:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.890+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e55f24e0 con 0x5635e569c2b0 2023-10-08T09:22:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e569c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e569c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39270/0 (socket says 172.21.15.49:39270) 2023-10-08T09:22:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 -- 172.21.15.49:0/3905868780 learned_addr learned my addr 172.21.15.49:0/3905868780 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619b7fe700 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e569b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 -- 172.21.15.49:0/3905868780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 msgr2=0x5635e569bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e569bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 -- 172.21.15.49:0/3905868780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 msgr2=0x5635e569b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e569b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 -- 172.21.15.49:0/3905868780 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e5418d80 con 0x5635e569c2b0 2023-10-08T09:22:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.891+0000 7f619bfff700 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e569c730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6194007c00 tx=0x7f6194004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9ec200192d755a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f6181ffb700 1 -- 172.21.15.49:0/3905868780 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409b740 con 0x5635e569c2b0 2023-10-08T09:22:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f6181ffb700 1 -- 172.21.15.49:0/3905868780 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619409f890 con 0x5635e569c2b0 2023-10-08T09:22:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f6181ffb700 1 -- 172.21.15.49:0/3905868780 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409b9a0 con 0x5635e569c2b0 2023-10-08T09:22:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f61ae19eb80 1 -- 172.21.15.49:0/3905868780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 msgr2=0x5635e569c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e569c730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6194007c00 tx=0x7f6194004f60 comp rx=0 tx=0).stop 2023-10-08T09:22:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f61ae19eb80 1 -- 172.21.15.49:0/3905868780 shutdown_connections 2023-10-08T09:22:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e569c730 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e569b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/3905868780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e569bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.892+0000 7f61ae19eb80 1 -- 172.21.15.49:0/3905868780 >> 172.21.15.49:0/3905868780 conn(0x5635e560f200 msgr2=0x5635e56958a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 -- 172.21.15.49:0/3905868780 shutdown_connections 2023-10-08T09:22:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 -- 172.21.15.49:0/3905868780 wait complete. 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 -- start start 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 Processor -- start 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569ae40 0x5635e56a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569b7a0 0x5635e56a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569c2b0 0x5635e56abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e55eddd0 con 0x5635e569b7a0 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e558a6b0 con 0x5635e569c2b0 2023-10-08T09:22:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e558a350 con 0x5635e569ae40 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f61ae19eb80 10 client.0 _ll_register_callbacks cb 0x5635e53e77a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569ae40 0x5635e56a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f619affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569b7a0 0x5635e56a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f619affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569b7a0 0x5635e56a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52244/0 (socket says 172.21.15.49:52244) 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569ae40 0x5635e56a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39276/0 (socket says 172.21.15.49:39276) 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f619affd700 1 -- 172.21.15.49:0/1399649980 learned_addr learned my addr 172.21.15.49:0/1399649980 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.893+0000 7f619bfff700 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569c2b0 0x5635e56abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f619b7fe700 1 -- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569c2b0 msgr2=0x5635e56abbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f619b7fe700 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569c2b0 0x5635e56abbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f619b7fe700 1 -- 172.21.15.49:0/1399649980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569b7a0 msgr2=0x5635e56a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f619b7fe700 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569b7a0 0x5635e56a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f619b7fe700 1 -- 172.21.15.49:0/1399649980 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c1ad9e0 con 0x5635e569ae40 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f619affd700 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569b7a0 0x5635e56a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f619b7fe700 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569ae40 0x5635e56a46e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6190009c80 tx=0x7f6190004800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c819ff98255207ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f617affd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f617affd700 1 -- 172.21.15.49:0/1399649980 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190015e60 con 0x5635e569ae40 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f617affd700 1 -- 172.21.15.49:0/1399649980 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6190021470 con 0x5635e569ae40 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.894+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e55b5e50 con 0x5635e569ae40 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f617affd700 1 -- 172.21.15.49:0/1399649980 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6190021630 con 0x5635e569ae40 2023-10-08T09:22:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 10 client.26173 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5635e5401c00 con 0x5635e569ae40 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f617affd700 1 -- 172.21.15.49:0/1399649980 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635e55b5e50 con 0x5635e569ae40 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f617affd700 1 -- 172.21.15.49:0/1399649980 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f61900138c0 con 0x5635e569ae40 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 10 client.26173 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 10 client.26173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 20 client.26173 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e56b0520 0x5635e56b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 4 client.26173 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e55e6db0 con 0x5635e56b0520 2023-10-08T09:22:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.895+0000 7f619affd700 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e56b0520 0x5635e56b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.896+0000 7f617affd700 10 client.26173 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.896+0000 7f619affd700 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e56b0520 0x5635e56b2940 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a02be1c6c15f342 server_cookie=7b8b57f6044c1614 in_seq=0 out_seq=0 2023-10-08T09:22:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f617affd700 1 -- 172.21.15.49:0/1399649980 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635e55e6db0 con 0x5635e56b0520 2023-10-08T09:22:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f617affd700 10 client.26173 handle_command_reply: tid=0 2023-10-08T09:22:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 client.26173 shutdown 2023-10-08T09:22:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569ae40 msgr2=0x5635e56a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569ae40 0x5635e56a46e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6190009c80 tx=0x7f6190004800 comp rx=0 tx=0).stop 2023-10-08T09:22:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 shutdown_connections 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e56b0520 0x5635e56b2940 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569ae40 0x5635e56a46e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f617affd700 0 client.26173 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569c2b0 0x5635e56abbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/1399649980 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569b7a0 0x5635e56a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 >> 172.21.15.49:0/1399649980 conn(0x5635e560f200 msgr2=0x5635e5695010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 shutdown_connections 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 1 -- 172.21.15.49:0/1399649980 wait complete. 2023-10-08T09:22:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.897+0000 7f61ae19eb80 20 client.26173 trim_cache size 0 max 16384 2023-10-08T09:22:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 Processor -- start 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 -- start start 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e56b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e546e8c0 con 0x5635e569ae40 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e569bb80 con 0x5635e569b7a0 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e5692850 con 0x5635e569c2b0 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f619bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f619bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39286/0 (socket says 172.21.15.49:39286) 2023-10-08T09:22:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.918+0000 7f619bfff700 1 -- 172.21.15.49:0/2130291927 learned_addr learned my addr 172.21.15.49:0/2130291927 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f619bfff700 1 -- 172.21.15.49:0/2130291927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 msgr2=0x5635e56abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f619bfff700 1 --2- 172.21.15.49:0/2130291927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f619bfff700 1 -- 172.21.15.49:0/2130291927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 msgr2=0x5635e56b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f619bfff700 1 --2- 172.21.15.49:0/2130291927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e56b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f619bfff700 1 -- 172.21.15.49:0/2130291927 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e543f1f0 con 0x5635e569c2b0 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f619bfff700 1 --2- 172.21.15.49:0/2130291927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56a28c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933c3739dfad0756 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f61827fc700 1 -- 172.21.15.49:0/2130291927 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194011df0 con 0x5635e569c2b0 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f61827fc700 1 -- 172.21.15.49:0/2130291927 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619409b740 con 0x5635e569c2b0 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.919+0000 7f61827fc700 1 -- 172.21.15.49:0/2130291927 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409b9a0 con 0x5635e569c2b0 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 -- 172.21.15.49:0/2130291927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 msgr2=0x5635e56a28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/2130291927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56a28c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619400b3b0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 -- 172.21.15.49:0/2130291927 shutdown_connections 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/2130291927 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56a28c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/2130291927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e56b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/2130291927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 -- 172.21.15.49:0/2130291927 >> 172.21.15.49:0/2130291927 conn(0x5635e560f200 msgr2=0x5635e56b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 -- 172.21.15.49:0/2130291927 shutdown_connections 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.920+0000 7f61ae19eb80 1 -- 172.21.15.49:0/2130291927 wait complete. 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 -- start start 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 Processor -- start 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e56a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e56aad40 con 0x5635e569ae40 2023-10-08T09:22:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e56aaeb0 con 0x5635e569b7a0 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e56ab070 con 0x5635e569c2b0 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f61ae19eb80 10 client.0 _ll_register_callbacks cb 0x7f619000eed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e56a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41872/0 (socket says 172.21.15.49:41872) 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619affd700 1 -- 172.21.15.49:0/674868174 learned_addr learned my addr 172.21.15.49:0/674868174 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619bfff700 1 -- 172.21.15.49:0/674868174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 msgr2=0x5635e56a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619bfff700 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56a9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619bfff700 1 -- 172.21.15.49:0/674868174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 msgr2=0x5635e56a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619bfff700 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e56a93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.921+0000 7f619bfff700 1 -- 172.21.15.49:0/674868174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635e5418d80 con 0x5635e569c2b0 2023-10-08T09:22:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f619bfff700 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56aa800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619409b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c3959b8c1889d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f617affd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:22:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f617affd700 1 -- 172.21.15.49:0/674868174 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409bba0 con 0x5635e569c2b0 2023-10-08T09:22:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f617affd700 1 -- 172.21.15.49:0/674868174 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f61940a35b0 con 0x5635e569c2b0 2023-10-08T09:22:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f619409f890 con 0x5635e569c2b0 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f617affd700 1 -- 172.21.15.49:0/674868174 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61940a3770 con 0x5635e569c2b0 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 10 client.26179 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5635e5417250 con 0x5635e569c2b0 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f617affd700 1 -- 172.21.15.49:0/674868174 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6194065080 con 0x5635e569c2b0 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f617affd700 1 -- 172.21.15.49:0/674868174 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f619409a6a0 con 0x5635e569c2b0 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 10 client.26179 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 20 client.26179 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e5704760 0x5635e5706b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 4 client.26179 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:22:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.922+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635e55e6ad0 con 0x5635e5704760 2023-10-08T09:22:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.923+0000 7f619b7fe700 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e5704760 0x5635e5706b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.923+0000 7f617affd700 10 client.26179 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.923+0000 7f619b7fe700 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e5704760 0x5635e5706b50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ce9e6672ce094 server_cookie=835d1ab69d9a826 in_seq=0 out_seq=0 2023-10-08T09:22:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f617affd700 1 -- 172.21.15.49:0/674868174 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635e55e6ad0 con 0x5635e5704760 2023-10-08T09:22:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f617affd700 10 client.26179 handle_command_reply: tid=0 2023-10-08T09:22:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 client.26179 shutdown 2023-10-08T09:22:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 msgr2=0x5635e56aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56aa800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619409b3b0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 shutdown_connections 2023-10-08T09:22:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5635e5704760 0x5635e5706b50 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5635e569c2b0 0x5635e56aa800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5635e569ae40 0x5635e56a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 --2- 172.21.15.49:0/674868174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5635e569b7a0 0x5635e56a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 >> 172.21.15.49:0/674868174 conn(0x5635e560f200 msgr2=0x5635e56a24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.935+0000 7f617affd700 0 client.26179 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 shutdown_connections 2023-10-08T09:22:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61ae19eb80 1 -- 172.21.15.49:0/674868174 wait complete. 2023-10-08T09:22:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61ae19eb80 20 client.26179 trim_cache size 0 max 16384 2023-10-08T09:22:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:22:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:22:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:22:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61840a1480 msgr2=0x7f61840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61840a1480 0x7f61840a3940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6190009ea0 tx=0x7f6190009210 comp rx=0 tx=0).stop 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10cb30 msgr2=0x7f619c1aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10cb30 0x7f619c1aca40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f619400b070 tx=0x7f619400bce0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 shutdown_connections 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61840a1480 0x7f61840a3940 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f619c10cb30 0x7f619c1aca40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f619c1077f0 0x7f619c1a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 --2- 172.21.15.49:0/3525851330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f619c10a160 0x7f619c1a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.937+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 >> 172.21.15.49:0/3525851330 conn(0x7f619c06d770 msgr2=0x7f619c108d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.938+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 shutdown_connections 2023-10-08T09:22:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:12.938+0000 7f61a1fb5700 1 -- 172.21.15.49:0/3525851330 wait complete. 2023-10-08T09:22:13.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:12 smithi132 ceph-mon[103389]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.8 MiB/s wr, 798 op/s 2023-10-08T09:22:13.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:12 smithi049 ceph-mon[99485]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.8 MiB/s wr, 798 op/s 2023-10-08T09:22:13.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:12 smithi184 ceph-mon[103675]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.8 MiB/s wr, 798 op/s 2023-10-08T09:22:14.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 410.0 s, 528.2 tps, lat 191.046 ms stddev 204.491 2023-10-08T09:22:14.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:14 smithi132 ceph-mon[103389]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 652 op/s 2023-10-08T09:22:14.644 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:14 smithi049 ceph-mon[99485]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 652 op/s 2023-10-08T09:22:14.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:14 smithi184 ceph-mon[103675]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.4 MiB/s wr, 652 op/s 2023-10-08T09:22:15.276 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.671+0000 7f9050f9b700 1 -- 172.21.15.49:0/2341160353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c108080 msgr2=0x7f904c10a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.671+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2341160353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c108080 0x7f904c10a470 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f903c009990 tx=0x7f903c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.672+0000 7f9050f9b700 1 -- 172.21.15.49:0/2341160353 shutdown_connections 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.672+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2341160353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f904c10d2e0 0x7f904c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.672+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2341160353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c108080 0x7f904c10a470 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.672+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2341160353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c10a9b0 0x7f904c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.672+0000 7f9050f9b700 1 -- 172.21.15.49:0/2341160353 >> 172.21.15.49:0/2341160353 conn(0x7f904c06d790 msgr2=0x7f904c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.672+0000 7f9050f9b700 1 -- 172.21.15.49:0/2341160353 shutdown_connections 2023-10-08T09:22:15.674 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.672+0000 7f9050f9b700 1 -- 172.21.15.49:0/2341160353 wait complete. 2023-10-08T09:22:15.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 Processor -- start 2023-10-08T09:22:15.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 -- start start 2023-10-08T09:22:15.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f904c108080 0x7f904c072d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c10a9b0 0x7f904c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c10d2e0 0x7f904c07a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c07a820 con 0x7f904c10a9b0 2023-10-08T09:22:15.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c07a990 con 0x7f904c108080 2023-10-08T09:22:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f9050f9b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c07ab50 con 0x7f904c10d2e0 2023-10-08T09:22:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f904ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c10d2e0 0x7f904c07a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f904ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c10d2e0 0x7f904c07a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39300/0 (socket says 172.21.15.49:39300) 2023-10-08T09:22:15.676 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.673+0000 7f904ad9d700 1 -- 172.21.15.49:0/2802832939 learned_addr learned my addr 172.21.15.49:0/2802832939 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:15.677 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.676+0000 7f904a59c700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f904c108080 0x7f904c072d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.676+0000 7f9043fff700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c10a9b0 0x7f904c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.679+0000 7f9043fff700 1 -- 172.21.15.49:0/2802832939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c10d2e0 msgr2=0x7f904c07a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.679+0000 7f9043fff700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c10d2e0 0x7f904c07a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.679+0000 7f9043fff700 1 -- 172.21.15.49:0/2802832939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f904c108080 msgr2=0x7f904c072d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.679+0000 7f9043fff700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f904c108080 0x7f904c072d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.679+0000 7f9043fff700 1 -- 172.21.15.49:0/2802832939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c0095f0 con 0x7f904c10a9b0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.679+0000 7f9043fff700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c10a9b0 0x7f904c0732d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9034006fd0 tx=0x7f903400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.680+0000 7f90437fe700 1 -- 172.21.15.49:0/2802832939 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034011830 con 0x7f904c10a9b0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.680+0000 7f90437fe700 1 -- 172.21.15.49:0/2802832939 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9034011e50 con 0x7f904c10a9b0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.680+0000 7f90437fe700 1 -- 172.21.15.49:0/2802832939 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034010df0 con 0x7f904c10a9b0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.680+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c07ae30 con 0x7f904c10a9b0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.680+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c07b2f0 con 0x7f904c10a9b0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.684+0000 7f90437fe700 1 -- 172.21.15.49:0/2802832939 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f903401a400 con 0x7f904c10a9b0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.684+0000 7f90437fe700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90380a15d0 0x7f90380a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f904a59c700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90380a15d0 0x7f90380a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 Processor -- start 2023-10-08T09:22:15.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f90437fe700 1 -- 172.21.15.49:0/2802832939 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f903409a2e0 con 0x7f904c10a9b0 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 -- start start 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df95093480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df95094270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95094c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94ffb460 con 0x55df95094800 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94fecc20 con 0x55df95093060 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.685+0000 7f905d184b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94fea540 con 0x55df950939c0 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f9043fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df95094270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95094c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95094c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52276/0 (socket says 172.21.15.49:52276) 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f9043fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df95094270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39312/0 (socket says 172.21.15.49:39312) 2023-10-08T09:22:15.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 -- 172.21.15.49:0/4216922142 learned_addr learned my addr 172.21.15.49:0/4216922142 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 -- 172.21.15.49:0/4216922142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 msgr2=0x55df95094270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 --2- 172.21.15.49:0/4216922142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df95094270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 -- 172.21.15.49:0/4216922142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 msgr2=0x55df95093480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 --2- 172.21.15.49:0/4216922142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df95093480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 -- 172.21.15.49:0/4216922142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df94e10de0 con 0x55df95094800 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f904ad9d700 1 --2- 172.21.15.49:0/4216922142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95094c90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f904400e270 tx=0x7f90440096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de31f5b3cb4fe63 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.686+0000 7f9040ff9700 1 -- 172.21.15.49:0/4216922142 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044014480 con 0x55df95094800 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f9040ff9700 1 -- 172.21.15.49:0/4216922142 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904400f040 con 0x55df95094800 2023-10-08T09:22:15.691 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f9040ff9700 1 -- 172.21.15.49:0/4216922142 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90440139a0 con 0x55df95094800 2023-10-08T09:22:15.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f905d184b80 1 -- 172.21.15.49:0/4216922142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 msgr2=0x55df95094c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.694 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f904a59c700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90380a15d0 0x7f90380a3a90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f903c000c00 tx=0x7f903c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f905d184b80 1 --2- 172.21.15.49:0/4216922142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95094c90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f904400e270 tx=0x7f90440096f0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f905d184b80 1 -- 172.21.15.49:0/4216922142 shutdown_connections 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f905d184b80 1 --2- 172.21.15.49:0/4216922142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95094c90 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.687+0000 7f905d184b80 1 --2- 172.21.15.49:0/4216922142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df95093480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 --2- 172.21.15.49:0/4216922142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df95094270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 -- 172.21.15.49:0/4216922142 >> 172.21.15.49:0/4216922142 conn(0x55df95007260 msgr2=0x55df9508de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 -- 172.21.15.49:0/4216922142 shutdown_connections 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 -- 172.21.15.49:0/4216922142 wait complete. 2023-10-08T09:22:15.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 -- start start 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 Processor -- start 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df9509cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df9509d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df950a4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94fe5e30 con 0x55df95094800 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94f82710 con 0x55df95093060 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94f823b0 con 0x55df950939c0 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.688+0000 7f905d184b80 10 client.0 _ll_register_callbacks cb 0x55df94ddf800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f904a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df9509cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f904a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df9509cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41894/0 (socket says 172.21.15.49:41894) 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f904a59c700 1 -- 172.21.15.49:0/3679518385 learned_addr learned my addr 172.21.15.49:0/3679518385 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:15.696 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9043fff700 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df9509d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9043fff700 1 -- 172.21.15.49:0/3679518385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 msgr2=0x55df9509cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9043fff700 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df9509cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9043fff700 1 -- 172.21.15.49:0/3679518385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 msgr2=0x55df950a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9043fff700 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df950a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9043fff700 1 -- 172.21.15.49:0/3679518385 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9044009da0 con 0x55df950939c0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9043fff700 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df9509d320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f904c07b480 tx=0x7f90340995c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b42fd6529088d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.689+0000 7f9031ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f9031ffb700 1 -- 172.21.15.49:0/3679518385 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90340b1070 con 0x55df950939c0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df94fadeb0 con 0x55df950939c0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f9031ffb700 1 -- 172.21.15.49:0/3679518385 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f903409dd30 con 0x55df950939c0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f9031ffb700 1 -- 172.21.15.49:0/3679518385 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90340a1f70 con 0x55df950939c0 2023-10-08T09:22:15.697 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 10 client.26188 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55df94df9c60 con 0x55df950939c0 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f9031ffb700 1 -- 172.21.15.49:0/3679518385 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df94fadeb0 con 0x55df950939c0 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f9031ffb700 1 -- 172.21.15.49:0/3679518385 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f90340a1ae0 con 0x55df950939c0 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 10 client.26188 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 10 client.26188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 20 client.26188 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950a8be0 0x55df950ab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 4 client.26188 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df94fdee10 con 0x55df950a8be0 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.690+0000 7f904a59c700 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950a8be0 0x55df950ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.691+0000 7f9031ffb700 10 client.26188 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:15.698 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.691+0000 7f904a59c700 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950a8be0 0x55df950ab000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b6756586d3a458 server_cookie=c6a6700e97772bf2 in_seq=0 out_seq=0 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f9031ffb700 1 -- 172.21.15.49:0/3679518385 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55df94fdee10 con 0x55df950a8be0 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f9031ffb700 10 client.26188 handle_command_reply: tid=0 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 client.26188 shutdown 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 msgr2=0x55df9509d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df9509d320 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f904c07b480 tx=0x7f90340995c0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 shutdown_connections 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950a8be0 0x55df950ab000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df950a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df9509cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 --2- 172.21.15.49:0/3679518385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df9509d320 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 >> 172.21.15.49:0/3679518385 conn(0x55df95007260 msgr2=0x55df9508d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:15.699 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.692+0000 7f9031ffb700 0 client.26188 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.693+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 shutdown_connections 2023-10-08T09:22:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.693+0000 7f905d184b80 1 -- 172.21.15.49:0/3679518385 wait complete. 2023-10-08T09:22:15.700 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.693+0000 7f905d184b80 20 client.26188 trim_cache size 0 max 16384 2023-10-08T09:22:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 Processor -- start 2023-10-08T09:22:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 -- start start 2023-10-08T09:22:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df95097720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df950a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95097c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df94e66920 con 0x55df95094800 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df95088570 con 0x55df95093060 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f905d184b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9509dc80 con 0x55df950939c0 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95097c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95097c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52292/0 (socket says 172.21.15.49:52292) 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 -- 172.21.15.49:0/3336453765 learned_addr learned my addr 172.21.15.49:0/3336453765 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904a59c700 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df95097720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f9043fff700 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df950a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 -- 172.21.15.49:0/3336453765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 msgr2=0x55df950a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:15.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df950a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 -- 172.21.15.49:0/3336453765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 msgr2=0x55df95097720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:15.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df95097720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.722+0000 7f904ad9d700 1 -- 172.21.15.49:0/3336453765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df94e37250 con 0x55df95094800 2023-10-08T09:22:15.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.723+0000 7f904ad9d700 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95097c60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9044004b00 tx=0x7f9044004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724e6ebdad58b8c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.723+0000 7f90417fa700 1 -- 172.21.15.49:0/3336453765 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044014480 con 0x55df95094800 2023-10-08T09:22:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.723+0000 7f90417fa700 1 -- 172.21.15.49:0/3336453765 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9044014aa0 con 0x55df95094800 2023-10-08T09:22:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.724+0000 7f90417fa700 1 -- 172.21.15.49:0/3336453765 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044026a80 con 0x55df95094800 2023-10-08T09:22:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 -- 172.21.15.49:0/3336453765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 msgr2=0x55df95097c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95097c60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9044004b00 tx=0x7f9044004b30 comp rx=0 tx=0).stop 2023-10-08T09:22:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 -- 172.21.15.49:0/3336453765 shutdown_connections 2023-10-08T09:22:15.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df95094800 0x55df95097c60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95093060 0x55df95097720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 --2- 172.21.15.49:0/3336453765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df950939c0 0x55df950a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 -- 172.21.15.49:0/3336453765 >> 172.21.15.49:0/3336453765 conn(0x55df95007260 msgr2=0x55df9509bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:15.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 -- 172.21.15.49:0/3336453765 shutdown_connections 2023-10-08T09:22:15.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.725+0000 7f905d184b80 1 -- 172.21.15.49:0/3336453765 wait complete. 2023-10-08T09:22:15.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 -- start start 2023-10-08T09:22:15.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 Processor -- start 2023-10-08T09:22:15.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df95093060 0x55df950a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df950939c0 0x55df950a4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95094800 0x55df950a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df950a0ce0 con 0x55df950939c0 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df950a0e50 con 0x55df95094800 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df950a1010 con 0x55df95093060 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f905d184b80 10 client.0 _ll_register_callbacks cb 0x55df94ddf800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f9043fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df950939c0 0x55df950a4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f9043fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df950939c0 0x55df950a4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52296/0 (socket says 172.21.15.49:52296) 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f904a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df95093060 0x55df950a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f9043fff700 1 -- 172.21.15.49:0/3453594070 learned_addr learned my addr 172.21.15.49:0/3453594070 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.726+0000 7f904ad9d700 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95094800 0x55df950a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9043fff700 1 -- 172.21.15.49:0/3453594070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df95093060 msgr2=0x55df950a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9043fff700 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df95093060 0x55df950a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9043fff700 1 -- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95094800 msgr2=0x55df950a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9043fff700 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95094800 0x55df950a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9043fff700 1 -- 172.21.15.49:0/3453594070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df94e10de0 con 0x55df950939c0 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f904a59c700 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df95093060 0x55df950a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9043fff700 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df950939c0 0x55df950a4130 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f90340994d0 tx=0x7f90340992e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd64d929fa4aa750 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9031ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9031ffb700 1 -- 172.21.15.49:0/3453594070 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90340b1070 con 0x55df950939c0 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9031ffb700 1 -- 172.21.15.49:0/3453594070 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f903409dd30 con 0x55df950939c0 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f904400f040 con 0x55df950939c0 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f9031ffb700 1 -- 172.21.15.49:0/3453594070 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90340a9080 con 0x55df950939c0 2023-10-08T09:22:15.733 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f905d184b80 10 client.16536 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.727+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55df94e0f2b0 con 0x55df950939c0 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f9031ffb700 1 -- 172.21.15.49:0/3453594070 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f904400f040 con 0x55df950939c0 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f9031ffb700 1 -- 172.21.15.49:0/3453594070 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9034078b70 con 0x55df950939c0 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f905d184b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f905d184b80 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f905d184b80 20 client.16536 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f905d184b80 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950fde20 0x55df95100240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f905d184b80 4 client.16536 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df94fdeb30 con 0x55df950fde20 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.728+0000 7f904a59c700 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950fde20 0x55df95100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f9031ffb700 10 client.16536 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:15.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f904a59c700 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950fde20 0x55df95100240 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d9b8587c484de9 server_cookie=e633ee6d9e08b26b in_seq=0 out_seq=0 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f9031ffb700 1 -- 172.21.15.49:0/3453594070 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55df94fdeb30 con 0x55df950fde20 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f9031ffb700 10 client.16536 handle_command_reply: tid=0 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 client.16536 shutdown 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df950939c0 msgr2=0x55df950a4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df950939c0 0x55df950a4130 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f90340994d0 tx=0x7f90340992e0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 shutdown_connections 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55df950fde20 0x55df95100240 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55df95093060 0x55df950a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f9031ffb700 0 client.16536 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55df95094800 0x55df950a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 --2- 172.21.15.49:0/3453594070 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55df950939c0 0x55df950a4130 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 >> 172.21.15.49:0/3453594070 conn(0x55df95007260 msgr2=0x55df9509bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 shutdown_connections 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 1 -- 172.21.15.49:0/3453594070 wait complete. 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.729+0000 7f905d184b80 20 client.16536 trim_cache size 0 max 16384 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:22:15.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:22:15.737 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:22:15.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:22:15.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:22:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90380a15d0 msgr2=0x7f90380a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90380a15d0 0x7f90380a3a90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f903c000c00 tx=0x7f903c003860 comp rx=0 tx=0).stop 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c10a9b0 msgr2=0x7f904c0732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c10a9b0 0x7f904c0732d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9034006fd0 tx=0x7f903400b730 comp rx=0 tx=0).stop 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 shutdown_connections 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f90380a15d0 0x7f90380a3a90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f904c10d2e0 0x7f904c07a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f904c108080 0x7f904c072d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 --2- 172.21.15.49:0/2802832939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f904c10a9b0 0x7f904c0732d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 >> 172.21.15.49:0/2802832939 conn(0x7f904c06d790 msgr2=0x7f904c108c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:15.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 shutdown_connections 2023-10-08T09:22:15.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:15.730+0000 7f9050f9b700 1 -- 172.21.15.49:0/2802832939 wait complete. 2023-10-08T09:22:15.747 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:22:16.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:16 smithi132 ceph-mon[103389]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.2 MiB/s wr, 762 op/s 2023-10-08T09:22:16.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:16 smithi049 ceph-mon[99485]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.2 MiB/s wr, 762 op/s 2023-10-08T09:22:16.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:16 smithi184 ceph-mon[103675]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.2 MiB/s wr, 762 op/s 2023-10-08T09:22:18.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:18 smithi132 ceph-mon[103389]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 627 op/s 2023-10-08T09:22:18.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:18 smithi049 ceph-mon[99485]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 627 op/s 2023-10-08T09:22:18.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:18 smithi184 ceph-mon[103675]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 627 op/s 2023-10-08T09:22:19.317 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 415.0 s, 499.4 tps, lat 194.463 ms stddev 192.686 2023-10-08T09:22:20.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:20 smithi049 ceph-mon[99485]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2023-10-08T09:22:20.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:20 smithi184 ceph-mon[103675]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2023-10-08T09:22:20.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:20 smithi132 ceph-mon[103389]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2023-10-08T09:22:23.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:22 smithi132 ceph-mon[103389]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 5.8 MiB/s wr, 577 op/s 2023-10-08T09:22:23.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:22 smithi049 ceph-mon[99485]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 5.8 MiB/s wr, 577 op/s 2023-10-08T09:22:23.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:22 smithi184 ceph-mon[103675]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 5.8 MiB/s wr, 577 op/s 2023-10-08T09:22:24.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 420.0 s, 537.2 tps, lat 187.999 ms stddev 215.913 2023-10-08T09:22:24.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:24 smithi049 ceph-mon[99485]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 4.2 MiB/s wr, 441 op/s 2023-10-08T09:22:24.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:24 smithi184 ceph-mon[103675]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 4.2 MiB/s wr, 441 op/s 2023-10-08T09:22:24.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:24 smithi132 ceph-mon[103389]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 4.2 MiB/s wr, 441 op/s 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 -- 172.21.15.49:0/3869453900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 msgr2=0x7f7c30107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 --2- 172.21.15.49:0/3869453900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c30107200 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800b0d0 tx=0x7f7c2800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 -- 172.21.15.49:0/3869453900 shutdown_connections 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 --2- 172.21.15.49:0/3869453900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c300feed0 0x7f7c300ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 --2- 172.21.15.49:0/3869453900 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c30107200 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 --2- 172.21.15.49:0/3869453900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c300fffb0 0x7f7c30100410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 -- 172.21.15.49:0/3869453900 >> 172.21.15.49:0/3869453900 conn(0x7f7c300f9ef0 msgr2=0x7f7c300fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 -- 172.21.15.49:0/3869453900 shutdown_connections 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.225+0000 7f7c37208700 1 -- 172.21.15.49:0/3869453900 wait complete. 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 Processor -- start 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 -- start start 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c300feed0 0x7f7c30195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c300fffb0 0x7f7c30195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c3019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019d230 con 0x7f7c30100950 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019d370 con 0x7f7c300fffb0 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c37208700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019d500 con 0x7f7c300feed0 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c3019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c3019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52312/0 (socket says 172.21.15.49:52312) 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 -- 172.21.15.49:0/3878154883 learned_addr learned my addr 172.21.15.49:0/3878154883 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c300fffb0 0x7f7c30195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c300feed0 0x7f7c30195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 -- 172.21.15.49:0/3878154883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c300feed0 msgr2=0x7f7c30195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c300feed0 0x7f7c30195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 -- 172.21.15.49:0/3878154883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c300fffb0 msgr2=0x7f7c30195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c300fffb0 0x7f7c30195c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.226+0000 7f7c357a5700 1 -- 172.21.15.49:0/3878154883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24009520 con 0x7f7c30100950 2023-10-08T09:22:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.227+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c300feed0 0x7f7c30195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.227+0000 7f7c357a5700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c3019cc60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007b60 tx=0x7f7c28008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.228+0000 7f7c2dffb700 1 -- 172.21.15.49:0/3878154883 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800e040 con 0x7f7c30100950 2023-10-08T09:22:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.228+0000 7f7c2dffb700 1 -- 172.21.15.49:0/3878154883 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2801e070 con 0x7f7c30100950 2023-10-08T09:22:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.228+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c28009d00 con 0x7f7c30100950 2023-10-08T09:22:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.228+0000 7f7c2dffb700 1 -- 172.21.15.49:0/3878154883 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c280259e0 con 0x7f7c30100950 2023-10-08T09:22:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.228+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c300fb840 con 0x7f7c30100950 2023-10-08T09:22:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.233+0000 7f7c433f1b80 1 Processor -- start 2023-10-08T09:22:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c2dffb700 1 -- 172.21.15.49:0/3878154883 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7c2801aba0 con 0x7f7c30100950 2023-10-08T09:22:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c433f1b80 1 -- start start 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea82e40 0x556f2ea83260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea83cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea84730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e9eb400 con 0x556f2ea82e40 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e9dcbc0 con 0x556f2ea837a0 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e9da4e0 con 0x556f2ea842b0 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.234+0000 7f7c34fa4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea83cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea83cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41928/0 (socket says 172.21.15.49:41928) 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea84730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 -- 172.21.15.49:0/3431499554 learned_addr learned my addr 172.21.15.49:0/3431499554 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea82e40 0x556f2ea83260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c2dffb700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7c200a1500 0x7f7c200a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 -- 172.21.15.49:0/3431499554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 msgr2=0x556f2ea84730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea84730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 -- 172.21.15.49:0/3431499554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea82e40 msgr2=0x556f2ea83260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea82e40 0x556f2ea83260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 -- 172.21.15.49:0/3431499554 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2e800d80 con 0x556f2ea837a0 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea84730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c2dffb700 1 -- 172.21.15.49:0/3878154883 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7c28020070 con 0x7f7c30100950 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c357a5700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7c200a1500 0x7f7c200a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea83cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009ee0 tx=0x7f7c24011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd795b9f5efaf0be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c1affd700 1 -- 172.21.15.49:0/3431499554 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24016710 con 0x556f2ea837a0 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c1affd700 1 -- 172.21.15.49:0/3431499554 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c24016d30 con 0x556f2ea837a0 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.235+0000 7f7c1affd700 1 -- 172.21.15.49:0/3431499554 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2401ecf0 con 0x556f2ea837a0 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 -- 172.21.15.49:0/3431499554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 msgr2=0x556f2ea83cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea83cd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009ee0 tx=0x7f7c24011af0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c357a5700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7c200a1500 0x7f7c200a39c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7c28006090 tx=0x7f7c280a3040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 -- 172.21.15.49:0/3431499554 shutdown_connections 2023-10-08T09:22:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea84730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea82e40 0x556f2ea83260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/3431499554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea83cd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 -- 172.21.15.49:0/3431499554 >> 172.21.15.49:0/3431499554 conn(0x556f2e9f7200 msgr2=0x556f2ea7d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 -- 172.21.15.49:0/3431499554 shutdown_connections 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.236+0000 7f7c433f1b80 1 -- 172.21.15.49:0/3431499554 wait complete. 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 -- start start 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 Processor -- start 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea93dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e9d5dd0 con 0x556f2ea842b0 2023-10-08T09:22:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e9726b0 con 0x556f2ea837a0 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e972350 con 0x556f2ea82e40 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c433f1b80 10 client.0 _ll_register_callbacks cb 0x556f2e7cf7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c34fa4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c34fa4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41936/0 (socket says 172.21.15.49:41936) 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39354/0 (socket says 172.21.15.49:39354) 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c34fa4700 1 -- 172.21.15.49:0/188257329 learned_addr learned my addr 172.21.15.49:0/188257329 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea93dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c34fa4700 1 -- 172.21.15.49:0/188257329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 msgr2=0x556f2ea8c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.237+0000 7f7c34fa4700 1 -- 172.21.15.49:0/188257329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 msgr2=0x556f2ea93dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea93dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c34fa4700 1 -- 172.21.15.49:0/188257329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c019040 con 0x556f2ea837a0 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ce20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009840 tx=0x7f7c2401fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883839df25e02ebb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c0ffff700 1 -- 172.21.15.49:0/188257329 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24016720 con 0x556f2ea837a0 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c0ffff700 1 -- 172.21.15.49:0/188257329 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7c24016880 con 0x556f2ea837a0 2023-10-08T09:22:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.238+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f2e99de50 con 0x556f2ea837a0 2023-10-08T09:22:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.243+0000 7f7c0ffff700 1 -- 172.21.15.49:0/188257329 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c24016a40 con 0x556f2ea837a0 2023-10-08T09:22:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.243+0000 7f7c433f1b80 10 client.26168 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.243+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f7c24011440 con 0x556f2ea837a0 2023-10-08T09:22:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c0ffff700 1 -- 172.21.15.49:0/188257329 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f2e99de50 con 0x556f2ea837a0 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c0ffff700 1 -- 172.21.15.49:0/188257329 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c24004a30 con 0x556f2ea837a0 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c433f1b80 10 client.26168 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c433f1b80 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c433f1b80 20 client.26168 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2ea98660 0x556f2ea9aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c433f1b80 4 client.26168 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f2e9cedb0 con 0x556f2ea98660 2023-10-08T09:22:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.244+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2ea98660 0x556f2ea9aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.245+0000 7f7c0ffff700 10 client.26168 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.245+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2ea98660 0x556f2ea9aa80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a9e247e5e6eccc server_cookie=8860ab68e3aa9ec9 in_seq=0 out_seq=0 2023-10-08T09:22:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.246+0000 7f7c0ffff700 1 -- 172.21.15.49:0/188257329 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f2e9cedb0 con 0x556f2ea98660 2023-10-08T09:22:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.246+0000 7f7c0ffff700 10 client.26168 handle_command_reply: tid=0 2023-10-08T09:22:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.246+0000 7f7c433f1b80 1 client.26168 shutdown 2023-10-08T09:22:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.246+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 msgr2=0x556f2ea8ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.246+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ce20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009840 tx=0x7f7c2401fdf0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 shutdown_connections 2023-10-08T09:22:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2ea98660 0x556f2ea9aa80 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c0ffff700 0 client.26168 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea93dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/188257329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ce20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 >> 172.21.15.49:0/188257329 conn(0x556f2e9f7200 msgr2=0x556f2ea7d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 shutdown_connections 2023-10-08T09:22:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 1 -- 172.21.15.49:0/188257329 wait complete. 2023-10-08T09:22:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.248+0000 7f7c433f1b80 20 client.26168 trim_cache size 0 max 16384 2023-10-08T09:22:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.266+0000 7f7c433f1b80 1 Processor -- start 2023-10-08T09:22:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.266+0000 7f7c433f1b80 1 -- start start 2023-10-08T09:22:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea82e40 0x556f2ea84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea837a0 0x556f2ea93d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea8a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e8568c0 con 0x556f2ea837a0 2023-10-08T09:22:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2ea83b80 con 0x556f2ea82e40 2023-10-08T09:22:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2e99de50 con 0x556f2ea842b0 2023-10-08T09:22:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea82e40 0x556f2ea84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea82e40 0x556f2ea84690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:41942/0 (socket says 172.21.15.49:41942) 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2579105192 learned_addr learned my addr 172.21.15.49:0/2579105192 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea8a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.267+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea837a0 0x556f2ea93d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2579105192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 msgr2=0x556f2ea8a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea8a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2579105192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea837a0 msgr2=0x556f2ea93d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea837a0 0x556f2ea93d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2579105192 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2e7e9c00 con 0x556f2ea82e40 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea82e40 0x556f2ea84690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009e70 tx=0x7f7c1c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3e1f4747f97180 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c1b7fe700 1 -- 172.21.15.49:0/2579105192 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c028e50 con 0x556f2ea82e40 2023-10-08T09:22:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c1b7fe700 1 -- 172.21.15.49:0/2579105192 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c1c004030 con 0x556f2ea82e40 2023-10-08T09:22:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.268+0000 7f7c1b7fe700 1 -- 172.21.15.49:0/2579105192 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c033bc0 con 0x556f2ea82e40 2023-10-08T09:22:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2579105192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea82e40 msgr2=0x556f2ea84690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea82e40 0x556f2ea84690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009e70 tx=0x7f7c1c009260 comp rx=0 tx=0).stop 2023-10-08T09:22:26.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2579105192 shutdown_connections 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea842b0 0x556f2ea8a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea82e40 0x556f2ea84690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2579105192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea837a0 0x556f2ea93d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2579105192 >> 172.21.15.49:0/2579105192 conn(0x556f2e9f7200 msgr2=0x556f2ea9f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2579105192 shutdown_connections 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.269+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2579105192 wait complete. 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 -- start start 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 Processor -- start 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea92f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.273 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2ea93470 con 0x556f2ea842b0 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2ea935e0 con 0x556f2ea837a0 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2ea937a0 con 0x556f2ea82e40 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c433f1b80 10 client.0 _ll_register_callbacks cb 0x7f7c1c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea92f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c2ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea92f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52336/0 (socket says 172.21.15.49:52336) 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2715245015 learned_addr learned my addr 172.21.15.49:0/2715245015 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.270+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2715245015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 msgr2=0x556f2ea8fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 msgr2=0x556f2ea8ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c2ffff700 1 -- 172.21.15.49:0/2715245015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c028b50 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c2ffff700 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea92f30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c027330 tx=0x7f7c1c02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469c648670a6efd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c0ffff700 1 -- 172.21.15.49:0/2715245015 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c027450 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c0ffff700 1 -- 172.21.15.49:0/2715245015 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7c1c0041a0 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c24016d30 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c0ffff700 1 -- 172.21.15.49:0/2715245015 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c1c004360 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c433f1b80 10 client.16554 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.271+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556f2e8271f0 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c0ffff700 1 -- 172.21.15.49:0/2715245015 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c1c046020 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c0ffff700 1 -- 172.21.15.49:0/2715245015 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7c1c0047c0 con 0x556f2ea842b0 2023-10-08T09:22:26.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c433f1b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c433f1b80 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c433f1b80 20 client.16554 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2eafe270 0x556f2eb00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c433f1b80 4 client.16554 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:22:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f2e9cead0 con 0x556f2eafe270 2023-10-08T09:22:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.272+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2eafe270 0x556f2eb00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.292+0000 7f7c34fa4700 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2eafe270 0x556f2eb00690 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9edfcfb173fc4c51 server_cookie=eeb9e7e1a36932b1 in_seq=0 out_seq=0 2023-10-08T09:22:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.302+0000 7f7c0ffff700 10 client.16554 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.434+0000 7f7c0ffff700 1 -- 172.21.15.49:0/2715245015 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f2e9cead0 con 0x556f2eafe270 2023-10-08T09:22:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.434+0000 7f7c0ffff700 10 client.16554 handle_command_reply: tid=0 2023-10-08T09:22:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.434+0000 7f7c433f1b80 1 client.16554 shutdown 2023-10-08T09:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.436+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 msgr2=0x556f2ea92f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.436+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea92f30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c027330 tx=0x7f7c1c02a040 comp rx=0 tx=0).stop 2023-10-08T09:22:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 shutdown_connections 2023-10-08T09:22:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556f2eafe270 0x556f2eb00690 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556f2ea82e40 0x556f2ea8fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556f2ea842b0 0x556f2ea92f30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 --2- 172.21.15.49:0/2715245015 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556f2ea837a0 0x556f2ea8ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 >> 172.21.15.49:0/2715245015 conn(0x556f2e9f7200 msgr2=0x556f2ea8bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c0ffff700 0 client.16554 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 shutdown_connections 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 1 -- 172.21.15.49:0/2715245015 wait complete. 2023-10-08T09:22:26.438 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.437+0000 7f7c433f1b80 20 client.16554 trim_cache size 0 max 16384 2023-10-08T09:22:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:22:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:22:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:22:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:22:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.438+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7c200a1500 msgr2=0x7f7c200a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.439 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.438+0000 7f7c37208700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7c200a1500 0x7f7c200a39c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7c28006090 tx=0x7f7c280a3040 comp rx=0 tx=0).stop 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.438+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 msgr2=0x7f7c3019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.438+0000 7f7c37208700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c3019cc60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007b60 tx=0x7f7c28008e40 comp rx=0 tx=0).stop 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.439+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 shutdown_connections 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.439+0000 7f7c37208700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7c200a1500 0x7f7c200a39c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.439+0000 7f7c37208700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c300feed0 0x7f7c30195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.439+0000 7f7c37208700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c30100950 0x7f7c3019cc60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.439+0000 7f7c37208700 1 --2- 172.21.15.49:0/3878154883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c300fffb0 0x7f7c30195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.439+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 >> 172.21.15.49:0/3878154883 conn(0x7f7c300f9ef0 msgr2=0x7f7c301036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.440+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 shutdown_connections 2023-10-08T09:22:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:26.440+0000 7f7c37208700 1 -- 172.21.15.49:0/3878154883 wait complete. 2023-10-08T09:22:26.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:26 smithi049 ceph-mon[99485]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2023-10-08T09:22:26.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:26 smithi184 ceph-mon[103675]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2023-10-08T09:22:26.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:26 smithi132 ceph-mon[103389]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.4 MiB/s wr, 570 op/s 2023-10-08T09:22:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:27 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:22:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:27 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:22:28.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:27 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:22:29.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2023-10-08T09:22:29.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:29 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 425.0 s, 717.1 tps, lat 139.464 ms stddev 159.477 2023-10-08T09:22:29.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2023-10-08T09:22:29.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:29 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 5.2 MiB/s wr, 542 op/s 2023-10-08T09:22:29.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:29.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:29 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:30.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:30 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi184 to 196.2M 2023-10-08T09:22:30.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:30 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi132 to 196.2M 2023-10-08T09:22:30.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:30 smithi132 ceph-mon[103389]: Unable to set osd_memory_target on smithi184 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-10-08T09:22:30.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:30 smithi132 ceph-mon[103389]: Unable to set osd_memory_target on smithi132 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-08T09:22:30.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:30 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi184 to 196.2M 2023-10-08T09:22:30.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:30 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi132 to 196.2M 2023-10-08T09:22:30.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:30 smithi049 ceph-mon[99485]: Unable to set osd_memory_target on smithi184 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-10-08T09:22:30.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:30 smithi049 ceph-mon[99485]: Unable to set osd_memory_target on smithi132 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-08T09:22:30.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:30 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi184 to 196.2M 2023-10-08T09:22:30.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:30 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi132 to 196.2M 2023-10-08T09:22:30.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:30 smithi184 ceph-mon[103675]: Unable to set osd_memory_target on smithi184 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-10-08T09:22:30.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:30 smithi184 ceph-mon[103675]: Unable to set osd_memory_target on smithi132 to 205773926: error parsing value: Value '205773926' is below minimum 939524096 2023-10-08T09:22:31.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:31 smithi049 ceph-mon[99485]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2023-10-08T09:22:31.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:31 smithi184 ceph-mon[103675]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2023-10-08T09:22:31.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:31 smithi132 ceph-mon[103389]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2023-10-08T09:22:32.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:22:32.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:22:32.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:22:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:22:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:22:32.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:22:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-08T09:22:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:22:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:22:32.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:22:33.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:33 smithi049 ceph-mon[99485]: Adjusting osd_memory_target on smithi049 to 196.2M 2023-10-08T09:22:33.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:33 smithi049 ceph-mon[99485]: Unable to set osd_memory_target on smithi049 to 205744537: error parsing value: Value '205744537' is below minimum 939524096 2023-10-08T09:22:33.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:33 smithi049 ceph-mon[99485]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2023-10-08T09:22:33.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:33 smithi184 ceph-mon[103675]: Adjusting osd_memory_target on smithi049 to 196.2M 2023-10-08T09:22:33.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:33 smithi184 ceph-mon[103675]: Unable to set osd_memory_target on smithi049 to 205744537: error parsing value: Value '205744537' is below minimum 939524096 2023-10-08T09:22:33.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:33 smithi184 ceph-mon[103675]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2023-10-08T09:22:33.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:33 smithi132 ceph-mon[103389]: Adjusting osd_memory_target on smithi049 to 196.2M 2023-10-08T09:22:33.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:33 smithi132 ceph-mon[103389]: Unable to set osd_memory_target on smithi049 to 205744537: error parsing value: Value '205744537' is below minimum 939524096 2023-10-08T09:22:33.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:33 smithi132 ceph-mon[103389]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 5.8 MiB/s wr, 590 op/s 2023-10-08T09:22:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 430.0 s, 578.3 tps, lat 171.791 ms stddev 187.012 2023-10-08T09:22:34.642 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:34 smithi049 ceph-mon[99485]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2023-10-08T09:22:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:34 smithi184 ceph-mon[103675]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2023-10-08T09:22:34.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:34 smithi132 ceph-mon[103389]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 4.4 MiB/s wr, 467 op/s 2023-10-08T09:22:36.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:36 smithi049 ceph-mon[99485]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 5.8 MiB/s wr, 594 op/s 2023-10-08T09:22:36.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:36 smithi184 ceph-mon[103675]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 5.8 MiB/s wr, 594 op/s 2023-10-08T09:22:36.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:36 smithi132 ceph-mon[103389]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 5.8 MiB/s wr, 594 op/s 2023-10-08T09:22:38.636 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:38 smithi049 ceph-mon[99485]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2023-10-08T09:22:38.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:38 smithi184 ceph-mon[103675]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2023-10-08T09:22:38.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:38 smithi132 ceph-mon[103389]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 5.3 MiB/s wr, 555 op/s 2023-10-08T09:22:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 435.0 s, 534.2 tps, lat 188.785 ms stddev 205.905 2023-10-08T09:22:39.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.903+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/882944540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 msgr2=0x7fbcd410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.903+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/882944540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 0x7fbcd410cfb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fbccc01c0d0 tx=0x7fbccc01c3e0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.908+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/882944540 shutdown_connections 2023-10-08T09:22:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.908+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/882944540 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 0x7fbcd410cfb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.908+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/882944540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd41077f0 0x7fbcd4109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.908+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/882944540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd410a160 0x7fbcd410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.908+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/882944540 >> 172.21.15.49:0/882944540 conn(0x7fbcd406d770 msgr2=0x7fbcd406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.920+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/882944540 shutdown_connections 2023-10-08T09:22:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.931+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/882944540 wait complete. 2023-10-08T09:22:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.931+0000 7fbcdbdaf700 1 Processor -- start 2023-10-08T09:22:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.936+0000 7fbcdbdaf700 1 -- start start 2023-10-08T09:22:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.936+0000 7fbcdbdaf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd41077f0 0x7fbcd419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.936+0000 7fbcdbdaf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd410a160 0x7fbcd419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.936+0000 7fbcdbdaf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 0x7fbcd41c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.936+0000 7fbcdbdaf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd41c6810 con 0x7fbcd410cb30 2023-10-08T09:22:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.936+0000 7fbcdbdaf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd41c6980 con 0x7fbcd410a160 2023-10-08T09:22:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.936+0000 7fbcdbdaf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd41c6b40 con 0x7fbcd41077f0 2023-10-08T09:22:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd41077f0 0x7fbcd419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd41077f0 0x7fbcd419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39382/0 (socket says 172.21.15.49:39382) 2023-10-08T09:22:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/1760906565 learned_addr learned my addr 172.21.15.49:0/1760906565 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcda34c700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 0x7fbcd41c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/1760906565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd410a160 msgr2=0x7fbcd419d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd410a160 0x7fbcd419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 msgr2=0x7fbcd41c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 0x7fbcd41c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.937+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/1760906565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc01ad10 con 0x7fbcd41077f0 2023-10-08T09:22:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.938+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd41077f0 0x7fbcd419cca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400b550 tx=0x7fbcc400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.945+0000 7fbccaffd700 1 -- 172.21.15.49:0/1760906565 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4011830 con 0x7fbcd41077f0 2023-10-08T09:22:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.945+0000 7fbccaffd700 1 -- 172.21.15.49:0/1760906565 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc4011e50 con 0x7fbcd41077f0 2023-10-08T09:22:40.070 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.945+0000 7fbccaffd700 1 -- 172.21.15.49:0/1760906565 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400f2e0 con 0x7fbcd41077f0 2023-10-08T09:22:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.951+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd41c6e20 con 0x7fbcd41077f0 2023-10-08T09:22:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.951+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd41c7250 con 0x7fbcd41077f0 2023-10-08T09:22:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.969+0000 7fbccaffd700 1 -- 172.21.15.49:0/1760906565 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbcc4011990 con 0x7fbcd41077f0 2023-10-08T09:22:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.970+0000 7fbccaffd700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbcc00a16c0 0x7fbcc00a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.970+0000 7fbccaffd700 1 -- 172.21.15.49:0/1760906565 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbcc40cc9e0 con 0x7fbcd41077f0 2023-10-08T09:22:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.972+0000 7fbcd934a700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbcc00a16c0 0x7fbcc00a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:39.974+0000 7fbcd934a700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbcc00a16c0 0x7fbcc00a3b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005950 tx=0x7fbcd00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 Processor -- start 2023-10-08T09:22:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 -- start start 2023-10-08T09:22:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cb7a0 0x5583ba7cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cc2b0 0x5583ba7cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba733400 con 0x5583ba7cc2b0 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba724bc0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.089+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba7224e0 con 0x5583ba7cae40 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39390/0 (socket says 172.21.15.49:39390) 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/26867749 learned_addr learned my addr 172.21.15.49:0/26867749 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcd934a700 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cb7a0 0x5583ba7cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcda34c700 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cc2b0 0x5583ba7cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcda34c700 1 -- 172.21.15.49:0/26867749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 msgr2=0x5583ba7cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcda34c700 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcda34c700 1 -- 172.21.15.49:0/26867749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cb7a0 msgr2=0x5583ba7cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcda34c700 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cb7a0 0x5583ba7cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcda34c700 1 -- 172.21.15.49:0/26867749 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583ba548d80 con 0x5583ba7cc2b0 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcda34c700 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cc2b0 0x5583ba7cc730 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbccc01e010 tx=0x7fbccc01a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f894cdd801cd5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.090+0000 7fbcbffff700 1 -- 172.21.15.49:0/26867749 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc01f040 con 0x5583ba7cc2b0 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbcbffff700 1 -- 172.21.15.49:0/26867749 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbccc0050d0 con 0x5583ba7cc2b0 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbcbffff700 1 -- 172.21.15.49:0/26867749 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc007700 con 0x5583ba7cc2b0 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 -- 172.21.15.49:0/26867749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cc2b0 msgr2=0x5583ba7cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cc2b0 0x5583ba7cc730 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbccc01e010 tx=0x7fbccc01a0d0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 -- 172.21.15.49:0/26867749 shutdown_connections 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cc2b0 0x5583ba7cc730 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/26867749 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cb7a0 0x5583ba7cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 -- 172.21.15.49:0/26867749 >> 172.21.15.49:0/26867749 conn(0x5583ba73f200 msgr2=0x5583ba7c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 -- 172.21.15.49:0/26867749 shutdown_connections 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.091+0000 7fbce7f98b80 1 -- 172.21.15.49:0/26867749 wait complete. 2023-10-08T09:22:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 -- start start 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 Processor -- start 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba71ddd0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba6ba6b0 con 0x5583ba7cc2b0 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba6ba350 con 0x5583ba7cae40 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbce7f98b80 10 client.0 _ll_register_callbacks cb 0x5583ba5177a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39396/0 (socket says 172.21.15.49:39396) 2023-10-08T09:22:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/3449220274 learned_addr learned my addr 172.21.15.49:0/3449220274 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbcd934a700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.092+0000 7fbcda34c700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 msgr2=0x5583ba7dbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/3449220274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 msgr2=0x5583ba7d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/3449220274 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd41c7250 con 0x5583ba7cae40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcd934a700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7d47f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40963e0 tx=0x7fbcc4010f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc17f688c2187ecb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/3449220274 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc409a0f0 con 0x5583ba7cae40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/3449220274 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbcc4010380 con 0x5583ba7cae40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583ba6e5e50 con 0x5583ba7cae40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/3449220274 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcc40b2220 con 0x5583ba7cae40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbce7f98b80 10 client.26224 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.093+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fbcd410da70 con 0x5583ba7cae40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/3449220274 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc40a5020 con 0x5583ba7cae40 2023-10-08T09:22:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/3449220274 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbcc4099bd0 con 0x5583ba7cae40 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbce7f98b80 10 client.26224 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbce7f98b80 10 client.26224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbce7f98b80 20 client.26224 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba7e0600 0x5583ba7e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbce7f98b80 4 client.26224 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583ba716db0 con 0x5583ba7e0600 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.094+0000 7fbcd934a700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba7e0600 0x5583ba7e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.095+0000 7fbcd934a700 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba7e0600 0x5583ba7e2a20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4662c78f64ad2b21 server_cookie=1de7c4e4bec07986 in_seq=0 out_seq=0 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.095+0000 7fbcbd7fa700 10 client.26224 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.095+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/3449220274 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583ba716db0 con 0x5583ba7e0600 2023-10-08T09:22:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.095+0000 7fbcbd7fa700 10 client.26224 handle_command_reply: tid=0 2023-10-08T09:22:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.095+0000 7fbce7f98b80 1 client.26224 shutdown 2023-10-08T09:22:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 msgr2=0x5583ba7d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7d47f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40963e0 tx=0x7fbcc4010f30 comp rx=0 tx=0).stop 2023-10-08T09:22:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 shutdown_connections 2023-10-08T09:22:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba7e0600 0x5583ba7e2a20 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cae40 0x5583ba7d47f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbcbd7fa700 0 client.26224 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/3449220274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 >> 172.21.15.49:0/3449220274 conn(0x5583ba73f200 msgr2=0x5583ba7c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 shutdown_connections 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 1 -- 172.21.15.49:0/3449220274 wait complete. 2023-10-08T09:22:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.096+0000 7fbce7f98b80 20 client.26224 trim_cache size 0 max 16384 2023-10-08T09:22:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 Processor -- start 2023-10-08T09:22:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 -- start start 2023-10-08T09:22:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cae40 0x5583ba7cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cb7a0 0x5583ba7dbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba59e8c0 con 0x5583ba7cae40 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba7ddab0 con 0x5583ba7cc2b0 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba7c02b0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cae40 0x5583ba7cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbcd934a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cb7a0 0x5583ba7dbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbcda34c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbcd9b4b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cae40 0x5583ba7cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52370/0 (socket says 172.21.15.49:52370) 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbcd934a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cb7a0 0x5583ba7dbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39404/0 (socket says 172.21.15.49:39404) 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.115+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/2460484213 learned_addr learned my addr 172.21.15.49:0/2460484213 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/2460484213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cb7a0 msgr2=0x5583ba7dbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/2460484213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cb7a0 0x5583ba7dbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/2460484213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 msgr2=0x5583ba7d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/2460484213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7d3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcd9b4b700 1 -- 172.21.15.49:0/2460484213 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583ba531c00 con 0x5583ba7cae40 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/2460484213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cae40 0x5583ba7cc690 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4099670 tx=0x7fbcc4098eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79266537ab21eb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcc8ff9700 1 -- 172.21.15.49:0/2460484213 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40b2220 con 0x5583ba7cae40 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcc8ff9700 1 -- 172.21.15.49:0/2460484213 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc40a5070 con 0x5583ba7cae40 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.116+0000 7fbcc8ff9700 1 -- 172.21.15.49:0/2460484213 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40a1d00 con 0x5583ba7cae40 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2460484213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cae40 msgr2=0x5583ba7cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2460484213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cae40 0x5583ba7cc690 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4099670 tx=0x7fbcc4098eb0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2460484213 shutdown_connections 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2460484213 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cc2b0 0x5583ba7d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2460484213 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cae40 0x5583ba7cc690 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2460484213 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cb7a0 0x5583ba7dbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2460484213 >> 172.21.15.49:0/2460484213 conn(0x5583ba73f200 msgr2=0x5583ba7c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2460484213 shutdown_connections 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2460484213 wait complete. 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 -- start start 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 Processor -- start 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cae40 0x5583ba7d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.117+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbce7f98b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba7db1d0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba7db340 con 0x5583ba7cae40 2023-10-08T09:22:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbce7f98b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583ba7db500 con 0x5583ba7cc2b0 2023-10-08T09:22:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbce7f98b80 10 client.0 _ll_register_callbacks cb 0x5583ba5177a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52378/0 (socket says 172.21.15.49:52378) 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 -- 172.21.15.49:0/2690457283 learned_addr learned my addr 172.21.15.49:0/2690457283 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcda34c700 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cae40 0x5583ba7d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 -- 172.21.15.49:0/2690457283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cc2b0 msgr2=0x5583ba7dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 -- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cae40 msgr2=0x5583ba7d9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cae40 0x5583ba7d9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 -- 172.21.15.49:0/2690457283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd410da70 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcd934a700 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d9cd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0012040 tx=0x7fbcd0005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dfee9aa7d6601a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.118+0000 7fbcbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.119+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/2690457283 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0024040 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.119+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/2690457283 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbcd0017040 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.119+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbccc0050d0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.119+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/2690457283 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcd0010660 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.119+0000 7fbce7f98b80 10 client.16572 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.119+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5583ba56f1f0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/2690457283 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbccc0050d0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/2690457283 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbcd002e8f0 con 0x5583ba7cb7a0 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbce7f98b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbce7f98b80 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbce7f98b80 20 client.16572 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba8350f0 0x5583ba8374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbce7f98b80 4 client.16572 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:22:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5583ba716ad0 con 0x5583ba8350f0 2023-10-08T09:22:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.120+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba8350f0 0x5583ba8374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.121+0000 7fbcd9b4b700 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba8350f0 0x5583ba8374e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980d83cd5e5ca96a server_cookie=b86755edaeda6b24 in_seq=0 out_seq=0 2023-10-08T09:22:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.121+0000 7fbcbd7fa700 10 client.16572 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.146+0000 7fbcbd7fa700 1 -- 172.21.15.49:0/2690457283 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5583ba716ad0 con 0x5583ba8350f0 2023-10-08T09:22:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.146+0000 7fbcbd7fa700 10 client.16572 handle_command_reply: tid=0 2023-10-08T09:22:40.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.147+0000 7fbce7f98b80 1 client.16572 shutdown 2023-10-08T09:22:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 msgr2=0x5583ba7d9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d9cd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0012040 tx=0x7fbcd0005d20 comp rx=0 tx=0).stop 2023-10-08T09:22:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 shutdown_connections 2023-10-08T09:22:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5583ba8350f0 0x5583ba8374e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5583ba7cc2b0 0x5583ba7dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5583ba7cae40 0x5583ba7d9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 --2- 172.21.15.49:0/2690457283 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5583ba7cb7a0 0x5583ba7d9cd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 >> 172.21.15.49:0/2690457283 conn(0x5583ba73f200 msgr2=0x5583ba7d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbcbd7fa700 0 client.16572 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 shutdown_connections 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 1 -- 172.21.15.49:0/2690457283 wait complete. 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.148+0000 7fbce7f98b80 20 client.16572 trim_cache size 0 max 16384 2023-10-08T09:22:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:22:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:22:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:22:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:22:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbcc00a16c0 msgr2=0x7fbcc00a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbcc00a16c0 0x7fbcc00a3b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005950 tx=0x7fbcd00058e0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd41077f0 msgr2=0x7fbcd419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd41077f0 0x7fbcd419cca0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400b550 tx=0x7fbcc400b860 comp rx=0 tx=0).stop 2023-10-08T09:22:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 shutdown_connections 2023-10-08T09:22:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbcc00a16c0 0x7fbcc00a3b80 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005950 tx=0x7fbcd00058e0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcd41077f0 0x7fbcd419cca0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbcd410cb30 0x7fbcd41c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 --2- 172.21.15.49:0/1760906565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcd410a160 0x7fbcd419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.149+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 >> 172.21.15.49:0/1760906565 conn(0x7fbcd406d770 msgr2=0x7fbcd410a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.152+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 shutdown_connections 2023-10-08T09:22:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:40.154+0000 7fbcdbdaf700 1 -- 172.21.15.49:0/1760906565 wait complete. 2023-10-08T09:22:40.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:40 smithi049 ceph-mon[99485]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 7.1 MiB/s wr, 617 op/s 2023-10-08T09:22:40.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:40 smithi184 ceph-mon[103675]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 7.1 MiB/s wr, 617 op/s 2023-10-08T09:22:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:40 smithi132 ceph-mon[103389]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 7.1 MiB/s wr, 617 op/s 2023-10-08T09:22:43.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:42 smithi132 ceph-mon[103389]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.7 MiB/s wr, 705 op/s 2023-10-08T09:22:43.145 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:42 smithi049 ceph-mon[99485]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.7 MiB/s wr, 705 op/s 2023-10-08T09:22:43.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:42 smithi184 ceph-mon[103675]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 7.7 MiB/s wr, 705 op/s 2023-10-08T09:22:44.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 440.0 s, 492.2 tps, lat 204.286 ms stddev 226.495 2023-10-08T09:22:45.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:45 smithi132 ceph-mon[103389]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 589 op/s 2023-10-08T09:22:45.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:45 smithi049 ceph-mon[99485]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 589 op/s 2023-10-08T09:22:45.749 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:22:45.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:45 smithi184 ceph-mon[103675]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.6 MiB/s wr, 589 op/s 2023-10-08T09:22:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.096+0000 7fd50f36e700 1 -- 172.21.15.49:0/3263950799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810cb30 msgr2=0x7fd50810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.096+0000 7fd50f36e700 1 --2- 172.21.15.49:0/3263950799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810cb30 0x7fd50810cfb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b0a0 tx=0x7fd50000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.097+0000 7fd50f36e700 1 -- 172.21.15.49:0/3263950799 shutdown_connections 2023-10-08T09:22:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.097+0000 7fd50f36e700 1 --2- 172.21.15.49:0/3263950799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810cb30 0x7fd50810cfb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.097+0000 7fd50f36e700 1 --2- 172.21.15.49:0/3263950799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5081077f0 0x7fd508109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.097+0000 7fd50f36e700 1 --2- 172.21.15.49:0/3263950799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd50810a160 0x7fd50810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.097+0000 7fd50f36e700 1 -- 172.21.15.49:0/3263950799 >> 172.21.15.49:0/3263950799 conn(0x7fd50806d790 msgr2=0x7fd50806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.097+0000 7fd50f36e700 1 -- 172.21.15.49:0/3263950799 shutdown_connections 2023-10-08T09:22:46.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.097+0000 7fd50f36e700 1 -- 172.21.15.49:0/3263950799 wait complete. 2023-10-08T09:22:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 Processor -- start 2023-10-08T09:22:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 -- start start 2023-10-08T09:22:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5081077f0 0x7fd5081a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810a160 0x7fd5081a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd50810cb30 0x7fd5081acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081ad120 con 0x7fd50810a160 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081ad260 con 0x7fd50810cb30 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50f36e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081ad3f0 con 0x7fd5081077f0 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50d10a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5081077f0 0x7fd5081a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50d90b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd50810cb30 0x7fd5081acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50c909700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810a160 0x7fd5081a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50d10a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5081077f0 0x7fd5081a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39424/0 (socket says 172.21.15.49:39424) 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50d10a700 1 -- 172.21.15.49:0/1660115045 learned_addr learned my addr 172.21.15.49:0/1660115045 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.098+0000 7fd50d90b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd50810cb30 0x7fd5081acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42006/0 (socket says 172.21.15.49:42006) 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.099+0000 7fd50d10a700 1 -- 172.21.15.49:0/1660115045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd50810cb30 msgr2=0x7fd5081acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.099+0000 7fd50d10a700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd50810cb30 0x7fd5081acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.099+0000 7fd50d10a700 1 -- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810a160 msgr2=0x7fd5081a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.099+0000 7fd50d10a700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810a160 0x7fd5081a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.099+0000 7fd50d10a700 1 -- 172.21.15.49:0/1660115045 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500009d00 con 0x7fd5081077f0 2023-10-08T09:22:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.099+0000 7fd50d10a700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5081077f0 0x7fd5081a55f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b550 tx=0x7fd4f800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.100+0000 7fd4fe7fc700 1 -- 172.21.15.49:0/1660115045 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8011830 con 0x7fd5081077f0 2023-10-08T09:22:46.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.100+0000 7fd4fe7fc700 1 -- 172.21.15.49:0/1660115045 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f8011e50 con 0x7fd5081077f0 2023-10-08T09:22:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.100+0000 7fd4fe7fc700 1 -- 172.21.15.49:0/1660115045 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8010dc0 con 0x7fd5081077f0 2023-10-08T09:22:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.100+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5081ad6d0 con 0x7fd5081077f0 2023-10-08T09:22:46.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.100+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5081adb90 con 0x7fd5081077f0 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 Processor -- start 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 -- start start 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d5211480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5212270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d5212800 0x5649d5212c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5179460 con 0x5649d5212800 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d516ac20 con 0x5649d5211060 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd51b557b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5168540 con 0x5649d52119c0 2023-10-08T09:22:46.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.104+0000 7fd50d90b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d5211480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d5211480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42010/0 (socket says 172.21.15.49:42010) 2023-10-08T09:22:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 -- 172.21.15.49:0/1637976487 learned_addr learned my addr 172.21.15.49:0/1637976487 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:46.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d10a700 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5212270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 -- 172.21.15.49:0/1637976487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 msgr2=0x5649d5212270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5212270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 -- 172.21.15.49:0/1637976487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d5212800 msgr2=0x5649d5212c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d5212800 0x5649d5212c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 -- 172.21.15.49:0/1637976487 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d4f8ede0 con 0x5649d5211060 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd50d90b700 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d5211480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd500015000 tx=0x7fd500009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80feaa8b1ae4bd0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd4f37fe700 1 -- 172.21.15.49:0/1637976487 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50000e040 con 0x5649d5211060 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd4f37fe700 1 -- 172.21.15.49:0/1637976487 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd500009480 con 0x5649d5211060 2023-10-08T09:22:46.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.105+0000 7fd4f37fe700 1 -- 172.21.15.49:0/1637976487 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5000055a0 con 0x5649d5211060 2023-10-08T09:22:46.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd4fe7fc700 1 -- 172.21.15.49:0/1660115045 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd4f8010380 con 0x7fd5081077f0 2023-10-08T09:22:46.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd51b557b80 1 -- 172.21.15.49:0/1637976487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 msgr2=0x5649d5211480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.109 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d5211480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd500015000 tx=0x7fd500009260 comp rx=0 tx=0).stop 2023-10-08T09:22:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd51b557b80 1 -- 172.21.15.49:0/1637976487 shutdown_connections 2023-10-08T09:22:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd4fe7fc700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4f40a1580 0x7fd4f40a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d5212800 0x5649d5212c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d5211480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1637976487 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5212270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.106+0000 7fd51b557b80 1 -- 172.21.15.49:0/1637976487 >> 172.21.15.49:0/1637976487 conn(0x5649d5185260 msgr2=0x5649d520be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:46.110 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.107+0000 7fd4fe7fc700 1 -- 172.21.15.49:0/1660115045 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd4f80cca30 con 0x7fd5081077f0 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.107+0000 7fd50d90b700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4f40a1580 0x7fd4f40a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.107+0000 7fd50d90b700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4f40a1580 0x7fd4f40a3a40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b0a0 tx=0x7fd50000b3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.108+0000 7fd51b557b80 1 -- 172.21.15.49:0/1637976487 shutdown_connections 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.108+0000 7fd51b557b80 1 -- 172.21.15.49:0/1637976487 wait complete. 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.108+0000 7fd51b557b80 1 -- start start 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.108+0000 7fd51b557b80 1 Processor -- start 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.108+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d521ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.108+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d521b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52203c0 0x5649d52227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd51b557b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5163e30 con 0x5649d52119c0 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd51b557b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5100710 con 0x5649d5211060 2023-10-08T09:22:46.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd51b557b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d51003b0 con 0x5649d52203c0 2023-10-08T09:22:46.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd51b557b80 10 client.0 _ll_register_callbacks cb 0x5649d4f5d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:46.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50d90b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52203c0 0x5649d52227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50d90b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52203c0 0x5649d52227b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39436/0 (socket says 172.21.15.49:39436) 2023-10-08T09:22:46.112 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50d90b700 1 -- 172.21.15.49:0/874188527 learned_addr learned my addr 172.21.15.49:0/874188527 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50c909700 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d521b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50d10a700 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d521ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50c909700 1 -- 172.21.15.49:0/874188527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52203c0 msgr2=0x5649d52227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50c909700 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52203c0 0x5649d52227b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50c909700 1 -- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 msgr2=0x5649d521ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.109+0000 7fd50c909700 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d521ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.110+0000 7fd50c909700 1 -- 172.21.15.49:0/874188527 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50810dae0 con 0x5649d52119c0 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.110+0000 7fd50c909700 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d521b390 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x5649d5212be0 tx=0x7fd50400e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b11841c09ac4a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:46.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.110+0000 7fd4f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:22:46.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.110+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/874188527 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400ee40 con 0x5649d52119c0 2023-10-08T09:22:46.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.110+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/874188527 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd504013470 con 0x5649d52119c0 2023-10-08T09:22:46.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.110+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649d512beb0 con 0x5649d52119c0 2023-10-08T09:22:46.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.110+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/874188527 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd504013630 con 0x5649d52119c0 2023-10-08T09:22:46.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.111+0000 7fd51b557b80 10 client.16581 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:46.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.111+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5649d4f77c60 con 0x5649d52119c0 2023-10-08T09:22:46.114 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.111+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/874188527 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649d512beb0 con 0x5649d52119c0 2023-10-08T09:22:46.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/874188527 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd504011070 con 0x5649d52119c0 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd51b557b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd51b557b80 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd51b557b80 20 client.16581 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd51b557b80 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d5227090 0x5649d52294b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd51b557b80 4 client.16581 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649d515ce10 con 0x5649d5227090 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.112+0000 7fd50d10a700 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d5227090 0x5649d52294b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.113+0000 7fd50d10a700 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d5227090 0x5649d52294b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8c0c5954ce1064 server_cookie=78aa8f5528a7f33c in_seq=0 out_seq=0 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.113+0000 7fd4f0ff9700 10 client.16581 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/874188527 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649d515ce10 con 0x5649d5227090 2023-10-08T09:22:46.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd4f0ff9700 10 client.16581 handle_command_reply: tid=0 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 client.16581 shutdown 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 msgr2=0x5649d521b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d521b390 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x5649d5212be0 tx=0x7fd50400e730 comp rx=0 tx=0).stop 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 shutdown_connections 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d5227090 0x5649d52294b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52203c0 0x5649d52227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d521ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 --2- 172.21.15.49:0/874188527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d521b390 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd4f0ff9700 0 client.16581 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 >> 172.21.15.49:0/874188527 conn(0x5649d5185260 msgr2=0x5649d520b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 shutdown_connections 2023-10-08T09:22:46.117 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 1 -- 172.21.15.49:0/874188527 wait complete. 2023-10-08T09:22:46.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.114+0000 7fd51b557b80 20 client.16581 trim_cache size 0 max 16384 2023-10-08T09:22:46.136 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.135+0000 7fd51b557b80 1 Processor -- start 2023-10-08T09:22:46.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.135+0000 7fd51b557b80 1 -- start start 2023-10-08T09:22:46.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.136+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d52227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.136+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5236c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.136+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52203c0 0x5649d5207f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.136+0000 7fd51b557b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d4fe4920 con 0x5649d52203c0 2023-10-08T09:22:46.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.136+0000 7fd51b557b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d521bd50 con 0x5649d5211060 2023-10-08T09:22:46.138 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.136+0000 7fd51b557b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5203b40 con 0x5649d52119c0 2023-10-08T09:22:46.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5236c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5236c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39446/0 (socket says 172.21.15.49:39446) 2023-10-08T09:22:46.139 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 -- 172.21.15.49:0/1819844614 learned_addr learned my addr 172.21.15.49:0/1819844614 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50d10a700 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d52227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 -- 172.21.15.49:0/1819844614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 msgr2=0x5649d52227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d52227b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 -- 172.21.15.49:0/1819844614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52203c0 msgr2=0x5649d5207f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52203c0 0x5649d5207f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 -- 172.21.15.49:0/1819844614 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d4fb5250 con 0x5649d52119c0 2023-10-08T09:22:46.140 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.138+0000 7fd50c909700 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5236c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd50400a010 tx=0x7fd504006af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=badaa474655df408 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:46.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.139+0000 7fd4f3fff700 1 -- 172.21.15.49:0/1819844614 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504013670 con 0x5649d52119c0 2023-10-08T09:22:46.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.139+0000 7fd4f3fff700 1 -- 172.21.15.49:0/1819844614 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd504006ce0 con 0x5649d52119c0 2023-10-08T09:22:46.141 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.139+0000 7fd4f3fff700 1 -- 172.21.15.49:0/1819844614 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504012ec0 con 0x5649d52119c0 2023-10-08T09:22:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 -- 172.21.15.49:0/1819844614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 msgr2=0x5649d5236c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5236c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd50400a010 tx=0x7fd504006af0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 -- 172.21.15.49:0/1819844614 shutdown_connections 2023-10-08T09:22:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52203c0 0x5649d5207f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d5211060 0x5649d52227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 --2- 172.21.15.49:0/1819844614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d52119c0 0x5649d5236c90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 -- 172.21.15.49:0/1819844614 >> 172.21.15.49:0/1819844614 conn(0x5649d5185260 msgr2=0x5649d5218f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 -- 172.21.15.49:0/1819844614 shutdown_connections 2023-10-08T09:22:46.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.140+0000 7fd51b557b80 1 -- 172.21.15.49:0/1819844614 wait complete. 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 -- start start 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 Processor -- start 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d5211060 0x5649d52789c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d5278f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d52203c0 0x5649d5236c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d52770c0 con 0x5649d52119c0 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5277230 con 0x5649d52203c0 2023-10-08T09:22:46.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d52773f0 con 0x5649d5211060 2023-10-08T09:22:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.141+0000 7fd51b557b80 10 client.0 _ll_register_callbacks cb 0x7fd504009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d5211060 0x5649d52789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d5211060 0x5649d52789c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39450/0 (socket says 172.21.15.49:39450) 2023-10-08T09:22:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 -- 172.21.15.49:0/752290705 learned_addr learned my addr 172.21.15.49:0/752290705 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d90b700 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d52203c0 0x5649d5236c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50c909700 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d5278f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 -- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d52203c0 msgr2=0x5649d5236c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d52203c0 0x5649d5236c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 -- 172.21.15.49:0/752290705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 msgr2=0x5649d5278f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d5278f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 -- 172.21.15.49:0/752290705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50400e040 con 0x5649d5211060 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd50d10a700 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d5211060 0x5649d52789c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800bbb0 tx=0x7fd4f80d24d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2eb70ed730041c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd4f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.142+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/752290705 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80d2160 con 0x5649d5211060 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/752290705 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd4f809a0c0 con 0x5649d5211060 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd500009480 con 0x5649d5211060 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/752290705 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f80092a0 con 0x5649d5211060 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 10 client.26245 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:46.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5649d4f8d2b0 con 0x5649d5211060 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/752290705 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f80b6070 con 0x5649d5211060 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/752290705 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd4f8098eb0 con 0x5649d5211060 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 10 client.26245 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 10 client.26245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 20 client.26245 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d528d170 0x5649d528f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 4 client.26245 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.143+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5649d515cb30 con 0x5649d528d170 2023-10-08T09:22:46.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.144+0000 7fd50c909700 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d528d170 0x5649d528f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:46.149 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.147+0000 7fd50c909700 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d528d170 0x5649d528f590 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5929fc66c24d0cd5 server_cookie=b3f7b3dfc6a8648b in_seq=0 out_seq=0 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.147+0000 7fd4f0ff9700 10 client.26245 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd4f0ff9700 1 -- 172.21.15.49:0/752290705 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5649d515cb30 con 0x5649d528d170 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd4f0ff9700 10 client.26245 handle_command_reply: tid=0 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 client.26245 shutdown 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d5211060 msgr2=0x5649d52789c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d5211060 0x5649d52789c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800bbb0 tx=0x7fd4f80d24d0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 shutdown_connections 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5649d528d170 0x5649d528f590 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5649d5211060 0x5649d52789c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd4f0ff9700 0 client.26245 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:46.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5649d52203c0 0x5649d5236c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 --2- 172.21.15.49:0/752290705 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5649d52119c0 0x5649d5278f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 >> 172.21.15.49:0/752290705 conn(0x5649d5185260 msgr2=0x5649d5218ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 shutdown_connections 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 1 -- 172.21.15.49:0/752290705 wait complete. 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.148+0000 7fd51b557b80 20 client.26245 trim_cache size 0 max 16384 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:22:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:22:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:22:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:22:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:22:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:22:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:22:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:22:46.152 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4f40a1580 msgr2=0x7fd4f40a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4f40a1580 0x7fd4f40a3a40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd50000b0a0 tx=0x7fd50000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5081077f0 msgr2=0x7fd5081a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5081077f0 0x7fd5081a55f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b550 tx=0x7fd4f800b860 comp rx=0 tx=0).stop 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 shutdown_connections 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd4f40a1580 0x7fd4f40a3a40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5081077f0 0x7fd5081a55f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd50810cb30 0x7fd5081acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 --2- 172.21.15.49:0/1660115045 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd50810a160 0x7fd5081a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:46.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 >> 172.21.15.49:0/1660115045 conn(0x7fd50806d790 msgr2=0x7fd5081099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:46.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 shutdown_connections 2023-10-08T09:22:46.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:22:46.149+0000 7fd50f36e700 1 -- 172.21.15.49:0/1660115045 wait complete. 2023-10-08T09:22:46.180 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:22:46.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:46 smithi049 ceph-mon[99485]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 746 op/s 2023-10-08T09:22:46.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:46 smithi184 ceph-mon[103675]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 746 op/s 2023-10-08T09:22:46.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:46 smithi132 ceph-mon[103389]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 10 MiB/s wr, 746 op/s 2023-10-08T09:22:49.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 445.0 s, 571.4 tps, lat 173.376 ms stddev 191.465 2023-10-08T09:22:49.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:49 smithi049 ceph-mon[99485]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.3 MiB/s wr, 701 op/s 2023-10-08T09:22:49.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:49 smithi184 ceph-mon[103675]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.3 MiB/s wr, 701 op/s 2023-10-08T09:22:49.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:49 smithi132 ceph-mon[103389]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 9.3 MiB/s wr, 701 op/s 2023-10-08T09:22:50.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:50 smithi049 ceph-mon[99485]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 709 op/s 2023-10-08T09:22:50.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:50 smithi184 ceph-mon[103675]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 709 op/s 2023-10-08T09:22:50.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:50 smithi132 ceph-mon[103389]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 10 MiB/s wr, 709 op/s 2023-10-08T09:22:53.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:52 smithi132 ceph-mon[103389]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 642 op/s 2023-10-08T09:22:53.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:52 smithi049 ceph-mon[99485]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 642 op/s 2023-10-08T09:22:53.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:52 smithi184 ceph-mon[103675]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 642 op/s 2023-10-08T09:22:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.361+0000 7fd57cd5e700 1 -- 172.21.15.49:0/303486500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 msgr2=0x7fd57810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.361+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/303486500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd57810f2e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd568009990 tx=0x7fd568009ca0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.361+0000 7fd57cd5e700 1 -- 172.21.15.49:0/303486500 shutdown_connections 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.361+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/303486500 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd57810f2e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.361+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/303486500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd5781079d0 0x7fd578109dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.361+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/303486500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd57810a300 0x7fd57810c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.361+0000 7fd57cd5e700 1 -- 172.21.15.49:0/303486500 >> 172.21.15.49:0/303486500 conn(0x7fd57806d930 msgr2=0x7fd57806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.362+0000 7fd57cd5e700 1 -- 172.21.15.49:0/303486500 shutdown_connections 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.362+0000 7fd57cd5e700 1 -- 172.21.15.49:0/303486500 wait complete. 2023-10-08T09:22:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 Processor -- start 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 -- start start 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd57810a300 0x7fd578118020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd578116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd578116c40 0x7fd5781170c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd578118770 con 0x7fd57810ce20 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5781201d0 con 0x7fd578116c40 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57cd5e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd578120340 con 0x7fd57810a300 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd575d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd578116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd575d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd578116700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52430/0 (socket says 172.21.15.49:52430) 2023-10-08T09:22:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd575d9b700 1 -- 172.21.15.49:0/2729267488 learned_addr learned my addr 172.21.15.49:0/2729267488 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd57659c700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd57810a300 0x7fd578118020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.363+0000 7fd576d9d700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd578116c40 0x7fd5781170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.364+0000 7fd575d9b700 1 -- 172.21.15.49:0/2729267488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd57810a300 msgr2=0x7fd578118020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.364+0000 7fd575d9b700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd57810a300 0x7fd578118020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.364+0000 7fd575d9b700 1 -- 172.21.15.49:0/2729267488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd578116c40 msgr2=0x7fd5781170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.364+0000 7fd575d9b700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd578116c40 0x7fd5781170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.364+0000 7fd575d9b700 1 -- 172.21.15.49:0/2729267488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5680095f0 con 0x7fd57810ce20 2023-10-08T09:22:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.364+0000 7fd575d9b700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd578116700 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd57000ef10 tx=0x7fd57000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.365+0000 7fd5677fe700 1 -- 172.21.15.49:0/2729267488 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000ca70 con 0x7fd57810ce20 2023-10-08T09:22:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.365+0000 7fd5677fe700 1 -- 172.21.15.49:0/2729267488 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57000f040 con 0x7fd57810ce20 2023-10-08T09:22:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.365+0000 7fd5677fe700 1 -- 172.21.15.49:0/2729267488 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570007ea0 con 0x7fd57810ce20 2023-10-08T09:22:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.365+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5781205c0 con 0x7fd57810ce20 2023-10-08T09:22:53.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.365+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd578120a80 con 0x7fd57810ce20 2023-10-08T09:22:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.369+0000 7fd5677fe700 1 -- 172.21.15.49:0/2729267488 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd570007500 con 0x7fd57810ce20 2023-10-08T09:22:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.369+0000 7fd5677fe700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5600a14f0 0x7fd5600a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.370+0000 7fd5677fe700 1 -- 172.21.15.49:0/2729267488 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd5700cd2e0 con 0x7fd57810ce20 2023-10-08T09:22:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.370+0000 7fd57659c700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5600a14f0 0x7fd5600a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.370+0000 7fd588f47b80 1 Processor -- start 2023-10-08T09:22:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.370+0000 7fd57659c700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5600a14f0 0x7fd5600a39b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd578071f00 tx=0x7fd56c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd588f47b80 1 -- start start 2023-10-08T09:22:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a96e40 0x55d9b9a97260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a977a0 0x55d9b9a97cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9a98730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd588f47b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b99ff400 con 0x55d9b9a982b0 2023-10-08T09:22:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd588f47b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b99f0bc0 con 0x55d9b9a96e40 2023-10-08T09:22:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd588f47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b99ee4e0 con 0x55d9b9a977a0 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd575d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a977a0 0x55d9b9a97cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd575d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a977a0 0x55d9b9a97cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39470/0 (socket says 172.21.15.49:39470) 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd575d9b700 1 -- 172.21.15.49:0/2671366663 learned_addr learned my addr 172.21.15.49:0/2671366663 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.378+0000 7fd57659c700 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a96e40 0x55d9b9a97260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd575d9b700 1 -- 172.21.15.49:0/2671366663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a96e40 msgr2=0x55d9b9a97260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd575d9b700 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a96e40 0x55d9b9a97260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd575d9b700 1 -- 172.21.15.49:0/2671366663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 msgr2=0x55d9b9a98730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd575d9b700 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9a98730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd575d9b700 1 -- 172.21.15.49:0/2671366663 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9b9814d80 con 0x55d9b9a977a0 2023-10-08T09:22:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd575d9b700 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a977a0 0x55d9b9a97cd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd57001d000 tx=0x7fd570099d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a539ae14c6c38618 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd564ff9700 1 -- 172.21.15.49:0/2671366663 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5700cd170 con 0x55d9b9a977a0 2023-10-08T09:22:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd564ff9700 1 -- 172.21.15.49:0/2671366663 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57009a2d0 con 0x55d9b9a977a0 2023-10-08T09:22:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.379+0000 7fd564ff9700 1 -- 172.21.15.49:0/2671366663 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570098750 con 0x55d9b9a977a0 2023-10-08T09:22:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 -- 172.21.15.49:0/2671366663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a977a0 msgr2=0x55d9b9a97cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a977a0 0x55d9b9a97cd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd57001d000 tx=0x7fd570099d00 comp rx=0 tx=0).stop 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 -- 172.21.15.49:0/2671366663 shutdown_connections 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9a98730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a96e40 0x55d9b9a97260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 --2- 172.21.15.49:0/2671366663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a977a0 0x55d9b9a97cd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 -- 172.21.15.49:0/2671366663 >> 172.21.15.49:0/2671366663 conn(0x55d9b9a0b200 msgr2=0x55d9b9a918a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.380+0000 7fd588f47b80 1 -- 172.21.15.49:0/2671366663 shutdown_connections 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.381+0000 7fd588f47b80 1 -- 172.21.15.49:0/2671366663 wait complete. 2023-10-08T09:22:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.381+0000 7fd588f47b80 1 -- start start 2023-10-08T09:22:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.381+0000 7fd588f47b80 1 Processor -- start 2023-10-08T09:22:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.381+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.381+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.381+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.381+0000 7fd588f47b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b99e9dd0 con 0x55d9b9a982b0 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd588f47b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b99866b0 con 0x55d9b9a977a0 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd588f47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9986350 con 0x55d9b9a96e40 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd588f47b80 10 client.0 _ll_register_callbacks cb 0x55d9b97e37a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd576d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39478/0 (socket says 172.21.15.49:39478) 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 -- 172.21.15.49:0/3882539824 learned_addr learned my addr 172.21.15.49:0/3882539824 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd576d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52442/0 (socket says 172.21.15.49:52442) 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd575d9b700 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 -- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 msgr2=0x55d9b9aa0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 -- 172.21.15.49:0/3882539824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 msgr2=0x55d9b9aa7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.382+0000 7fd57659c700 1 -- 172.21.15.49:0/3882539824 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd578120a80 con 0x55d9b9a96e40 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.383+0000 7fd57659c700 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa08c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00efe0 tx=0x7fd56c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea3b1dcff55bd086 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.383+0000 7fd55dffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.383+0000 7fd55dffb700 1 -- 172.21.15.49:0/3882539824 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c015bb0 con 0x55d9b9a96e40 2023-10-08T09:22:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.383+0000 7fd55dffb700 1 -- 172.21.15.49:0/3882539824 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd56c015d10 con 0x55d9b9a96e40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.384+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9b99b1e50 con 0x55d9b9a96e40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd55dffb700 1 -- 172.21.15.49:0/3882539824 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd56c0059d0 con 0x55d9b9a96e40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 10 client.26257 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d9b97fdc00 con 0x55d9b9a96e40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd55dffb700 1 -- 172.21.15.49:0/3882539824 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9b99b1e50 con 0x55d9b9a96e40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd55dffb700 1 -- 172.21.15.49:0/3882539824 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd56c006750 con 0x55d9b9a96e40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 10 client.26257 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 20 client.26257 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9aac750 0x55d9b9aaeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 4 client.26257 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9b99e2db0 con 0x55d9b9aac750 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.385+0000 7fd575d9b700 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9aac750 0x55d9b9aaeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.386+0000 7fd55dffb700 10 client.26257 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.386+0000 7fd575d9b700 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9aac750 0x55d9b9aaeb70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a11575368b45b9f server_cookie=19ccb258346f3333 in_seq=0 out_seq=0 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd55dffb700 1 -- 172.21.15.49:0/3882539824 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d9b99e2db0 con 0x55d9b9aac750 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd55dffb700 10 client.26257 handle_command_reply: tid=0 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 client.26257 shutdown 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 msgr2=0x55d9b9aa08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa08c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00efe0 tx=0x7fd56c0047b0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 shutdown_connections 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9aac750 0x55d9b9aaeb70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa08c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd55dffb700 0 client.26257 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3882539824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 >> 172.21.15.49:0/3882539824 conn(0x55d9b9a0b200 msgr2=0x55d9b9a91010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 shutdown_connections 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 1 -- 172.21.15.49:0/3882539824 wait complete. 2023-10-08T09:22:53.391 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.387+0000 7fd588f47b80 20 client.26257 trim_cache size 0 max 16384 2023-10-08T09:22:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 Processor -- start 2023-10-08T09:22:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 -- start start 2023-10-08T09:22:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a982b0 0x55d9b9a91870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b986a8c0 con 0x55d9b9a977a0 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b99b1e50 con 0x55d9b9a982b0 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd588f47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9aa9b80 con 0x55d9b9a96e40 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd575d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd57659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd575d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52452/0 (socket says 172.21.15.49:52452) 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd57659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39484/0 (socket says 172.21.15.49:39484) 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd575d9b700 1 -- 172.21.15.49:0/3900699314 learned_addr learned my addr 172.21.15.49:0/3900699314 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd576d9d700 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a982b0 0x55d9b9a91870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.406+0000 7fd57659c700 1 -- 172.21.15.49:0/3900699314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a982b0 msgr2=0x55d9b9a91870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd57659c700 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a982b0 0x55d9b9a91870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd57659c700 1 -- 172.21.15.49:0/3900699314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a977a0 msgr2=0x55d9b9aa7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd57659c700 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd57659c700 1 -- 172.21.15.49:0/3900699314 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9b983b1f0 con 0x55d9b9a96e40 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd575d9b700 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd57659c700 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa84a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd56c021000 tx=0x7fd56c009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95e1976e43b5b6ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd5657fa700 1 -- 172.21.15.49:0/3900699314 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c02a8b0 con 0x55d9b9a96e40 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd5657fa700 1 -- 172.21.15.49:0/3900699314 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd56c015bb0 con 0x55d9b9a96e40 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.407+0000 7fd5657fa700 1 -- 172.21.15.49:0/3900699314 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c004280 con 0x55d9b9a96e40 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 -- 172.21.15.49:0/3900699314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 msgr2=0x55d9b9aa84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa84a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd56c021000 tx=0x7fd56c009240 comp rx=0 tx=0).stop 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 -- 172.21.15.49:0/3900699314 shutdown_connections 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa84a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a982b0 0x55d9b9a91870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 --2- 172.21.15.49:0/3900699314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 -- 172.21.15.49:0/3900699314 >> 172.21.15.49:0/3900699314 conn(0x55d9b9a0b200 msgr2=0x55d9b9ab3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 -- 172.21.15.49:0/3900699314 shutdown_connections 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 -- 172.21.15.49:0/3900699314 wait complete. 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.408+0000 7fd588f47b80 1 -- start start 2023-10-08T09:22:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 1 Processor -- start 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9aa7570 con 0x55d9b9a982b0 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9aa76e0 con 0x55d9b9a977a0 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9aa78a0 con 0x55d9b9a96e40 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd588f47b80 10 client.0 _ll_register_callbacks cb 0x7fd56c00eb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd57659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd576d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd575d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd57659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39490/0 (socket says 172.21.15.49:39490) 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd57659c700 1 -- 172.21.15.49:0/1924301060 learned_addr learned my addr 172.21.15.49:0/1924301060 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd575d9b700 1 -- 172.21.15.49:0/1924301060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 msgr2=0x55d9b9aa5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd575d9b700 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd575d9b700 1 -- 172.21.15.49:0/1924301060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 msgr2=0x55d9b9aa7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd575d9b700 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.409+0000 7fd575d9b700 1 -- 172.21.15.49:0/1924301060 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9b9814d80 con 0x55d9b9a977a0 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd57659c700 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd575d9b700 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa6020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5700967d0 tx=0x7fd570000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878a288960b0292c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd55dffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:22:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd55dffb700 1 -- 172.21.15.49:0/1924301060 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57000ce40 con 0x55d9b9a977a0 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd57009a2d0 con 0x55d9b9a977a0 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd55dffb700 1 -- 172.21.15.49:0/1924301060 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd570083070 con 0x55d9b9a977a0 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd55dffb700 1 -- 172.21.15.49:0/1924301060 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57009e420 con 0x55d9b9a977a0 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd588f47b80 10 client.26216 fetch_fsmap learned FSMap version 40 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d9b97fdc00 con 0x55d9b9a977a0 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd55dffb700 1 -- 172.21.15.49:0/1924301060 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57009a2d0 con 0x55d9b9a977a0 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd55dffb700 1 -- 172.21.15.49:0/1924301060 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd57007e370 con 0x55d9b9a977a0 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd588f47b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd588f47b80 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.410+0000 7fd588f47b80 20 client.26216 populate_metadata read hostname 'smithi049' 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.411+0000 7fd588f47b80 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9b01190 0x55d9b9b03580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:22:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.411+0000 7fd588f47b80 4 client.26216 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:22:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.411+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9b99e2ad0 con 0x55d9b9b01190 2023-10-08T09:22:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.411+0000 7fd57659c700 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9b01190 0x55d9b9b03580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:22:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.411+0000 7fd55dffb700 10 client.26216 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.411+0000 7fd57659c700 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9b01190 0x55d9b9b03580 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee10828f439a7c4 server_cookie=577484aee79547e4 in_seq=0 out_seq=0 2023-10-08T09:22:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.580+0000 7fd55dffb700 1 -- 172.21.15.49:0/1924301060 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9b99e2ad0 con 0x55d9b9b01190 2023-10-08T09:22:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.580+0000 7fd55dffb700 10 client.26216 handle_command_reply: tid=0 2023-10-08T09:22:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.580+0000 7fd588f47b80 1 client.26216 shutdown 2023-10-08T09:22:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.581+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 msgr2=0x55d9b9aa6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.581+0000 7fd588f47b80 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa6020 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5700967d0 tx=0x7fd570000f40 comp rx=0 tx=0).stop 2023-10-08T09:22:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.585+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 shutdown_connections 2023-10-08T09:22:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.585+0000 7fd588f47b80 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d9b9b01190 0x55d9b9b03580 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.585+0000 7fd588f47b80 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d9b9a96e40 0x55d9b9aa5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.585+0000 7fd588f47b80 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d9b9a982b0 0x55d9b9aa7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.585+0000 7fd588f47b80 1 --2- 172.21.15.49:0/1924301060 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d9b9a977a0 0x55d9b9aa6020 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.585+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 >> 172.21.15.49:0/1924301060 conn(0x55d9b9a0b200 msgr2=0x55d9b9a9e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.586+0000 7fd55dffb700 0 client.26216 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:22:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.587+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 shutdown_connections 2023-10-08T09:22:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.589+0000 7fd588f47b80 1 -- 172.21.15.49:0/1924301060 wait complete. 2023-10-08T09:22:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:22:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:22:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:22:53.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:22:53.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.590+0000 7fd588f47b80 20 client.26216 trim_cache size 0 max 16384 2023-10-08T09:22:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.591+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5600a14f0 msgr2=0x7fd5600a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.591+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5600a14f0 0x7fd5600a39b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd578071f00 tx=0x7fd56c009110 comp rx=0 tx=0).stop 2023-10-08T09:22:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.591+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 msgr2=0x7fd578116700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:22:53.595 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.591+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd578116700 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd57000ef10 tx=0x7fd57000c3a0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.594+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 shutdown_connections 2023-10-08T09:22:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.594+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5600a14f0 0x7fd5600a39b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.594+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd57810a300 0x7fd578118020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.594+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd578116c40 0x7fd5781170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.594+0000 7fd57cd5e700 1 --2- 172.21.15.49:0/2729267488 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd57810ce20 0x7fd578116700 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:22:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.594+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 >> 172.21.15.49:0/2729267488 conn(0x7fd57806d930 msgr2=0x7fd5781097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:22:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.595+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 shutdown_connections 2023-10-08T09:22:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:22:53.595+0000 7fd57cd5e700 1 -- 172.21.15.49:0/2729267488 wait complete. 2023-10-08T09:22:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 450.0 s, 584.1 tps, lat 170.003 ms stddev 193.087 2023-10-08T09:22:55.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:55 smithi049 ceph-mon[99485]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 478 op/s 2023-10-08T09:22:55.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:55 smithi184 ceph-mon[103675]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 478 op/s 2023-10-08T09:22:55.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:55 smithi132 ceph-mon[103389]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 478 op/s 2023-10-08T09:22:56.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:56 smithi049 ceph-mon[99485]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 592 op/s 2023-10-08T09:22:56.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:56 smithi184 ceph-mon[103675]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 592 op/s 2023-10-08T09:22:56.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:56 smithi132 ceph-mon[103389]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.4 MiB/s wr, 592 op/s 2023-10-08T09:22:58.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:22:58 smithi049 ceph-mon[99485]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 530 op/s 2023-10-08T09:22:58.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:22:58 smithi184 ceph-mon[103675]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 530 op/s 2023-10-08T09:22:58.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:22:58 smithi132 ceph-mon[103389]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 530 op/s 2023-10-08T09:22:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 455.0 s, 588.7 tps, lat 168.981 ms stddev 187.585 2023-10-08T09:23:01.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:01 smithi132 ceph-mon[103389]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 534 op/s 2023-10-08T09:23:01.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:01 smithi049 ceph-mon[99485]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 534 op/s 2023-10-08T09:23:01.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:01 smithi184 ceph-mon[103675]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.3 MiB/s wr, 534 op/s 2023-10-08T09:23:03.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:02 smithi049 ceph-mon[99485]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 517 op/s 2023-10-08T09:23:03.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:02 smithi184 ceph-mon[103675]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 517 op/s 2023-10-08T09:23:03.305 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:02 smithi132 ceph-mon[103389]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 517 op/s 2023-10-08T09:23:04.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 460.0 s, 581.7 tps, lat 175.964 ms stddev 190.115 2023-10-08T09:23:05.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:05 smithi132 ceph-mon[103389]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 469 op/s 2023-10-08T09:23:05.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:05 smithi049 ceph-mon[99485]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 469 op/s 2023-10-08T09:23:05.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:05 smithi184 ceph-mon[103675]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 469 op/s 2023-10-08T09:23:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.045+0000 7fb4a982f700 1 -- 172.21.15.49:0/4226571242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 msgr2=0x7fb4a410a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.045+0000 7fb4a982f700 1 --2- 172.21.15.49:0/4226571242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a410a250 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb494009990 tx=0x7fb494009ca0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 -- 172.21.15.49:0/4226571242 shutdown_connections 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 --2- 172.21.15.49:0/4226571242 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a4110f30 0x7fb4a4113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 --2- 172.21.15.49:0/4226571242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a4072630 0x7fb4a4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 --2- 172.21.15.49:0/4226571242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a410a250 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 -- 172.21.15.49:0/4226571242 >> 172.21.15.49:0/4226571242 conn(0x7fb4a406d6e0 msgr2=0x7fb4a406fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 -- 172.21.15.49:0/4226571242 shutdown_connections 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 -- 172.21.15.49:0/4226571242 wait complete. 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.046+0000 7fb4a982f700 1 Processor -- start 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a982f700 1 -- start start 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a982f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a4117ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a982f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a4110f30 0x7fb4a4118430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a982f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a411d440 0x7fb4a411f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a982f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a411fe10 con 0x7fb4a411d440 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a982f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a411ff80 con 0x7fb4a4110f30 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a982f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a4120140 con 0x7fb4a4109dd0 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a4117ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a4117ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39504/0 (socket says 172.21.15.49:39504) 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/1771364323 learned_addr learned my addr 172.21.15.49:0/1771364323 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a4110f30 0x7fb4a4118430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/1771364323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a4110f30 msgr2=0x7fb4a4118430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a4110f30 0x7fb4a4118430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a411d440 msgr2=0x7fb4a411f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a411d440 0x7fb4a411f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.047+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/1771364323 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940095f0 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.048+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a4117ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b530 tx=0x7fb49800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.048+0000 7fb4a882d700 1 -- 172.21.15.49:0/1771364323 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498011830 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.048+0000 7fb4a882d700 1 -- 172.21.15.49:0/1771364323 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb498011e50 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.048+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a4120420 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.048+0000 7fb4a882d700 1 -- 172.21.15.49:0/1771364323 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498010e70 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.048+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a41209b0 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.052+0000 7fb4a882d700 1 -- 172.21.15.49:0/1771364323 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb498010510 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.052+0000 7fb4a882d700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb48c0a1470 0x7fb48c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.053+0000 7fb4a882d700 1 -- 172.21.15.49:0/1771364323 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb4980cca80 con 0x7fb4a4109dd0 2023-10-08T09:23:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.053+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb48c0a1470 0x7fb48c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.056+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb48c0a1470 0x7fb48c0a3930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb49400b2e0 tx=0x7fb494005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.139+0000 7fb4b5a18b80 1 Processor -- start 2023-10-08T09:23:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.150+0000 7fb4b5a18b80 1 -- start start 2023-10-08T09:23:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.150+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.150+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.150+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.150+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb222f400 con 0x556fb22c6e40 2023-10-08T09:23:07.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.150+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb2220bc0 con 0x556fb22c82b0 2023-10-08T09:23:07.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.150+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb221e4e0 con 0x556fb22c77a0 2023-10-08T09:23:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.151+0000 7fb4a37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.151+0000 7fb4a37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42092/0 (socket says 172.21.15.49:42092) 2023-10-08T09:23:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.151+0000 7fb4a37fe700 1 -- 172.21.15.49:0/3326051265 learned_addr learned my addr 172.21.15.49:0/3326051265 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.161+0000 7fb4a37fe700 1 -- 172.21.15.49:0/3326051265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 msgr2=0x556fb22c7cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:23:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.161+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/3326051265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22c7cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.161+0000 7fb4a37fe700 1 -- 172.21.15.49:0/3326051265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 msgr2=0x556fb22c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.161+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/3326051265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.161+0000 7fb4a37fe700 1 -- 172.21.15.49:0/3326051265 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fb2044d80 con 0x556fb22c82b0 2023-10-08T09:23:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.173+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/3326051265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22c8730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e400 tx=0x7fb49c00e710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c91f7a0f41eef1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.180+0000 7fb489ffb700 1 -- 172.21.15.49:0/3326051265 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c018070 con 0x556fb22c82b0 2023-10-08T09:23:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.180+0000 7fb489ffb700 1 -- 172.21.15.49:0/3326051265 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb49c00f040 con 0x556fb22c82b0 2023-10-08T09:23:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.180+0000 7fb489ffb700 1 -- 172.21.15.49:0/3326051265 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c004280 con 0x556fb22c82b0 2023-10-08T09:23:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.181+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3326051265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 msgr2=0x556fb22c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.181+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3326051265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22c8730 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e400 tx=0x7fb49c00e710 comp rx=0 tx=0).stop 2023-10-08T09:23:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.186+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3326051265 shutdown_connections 2023-10-08T09:23:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.186+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3326051265 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22c8730 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.186+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3326051265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.186+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3326051265 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.186+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3326051265 >> 172.21.15.49:0/3326051265 conn(0x556fb223b200 msgr2=0x556fb22c18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.187+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3326051265 shutdown_connections 2023-10-08T09:23:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.201+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3326051265 wait complete. 2023-10-08T09:23:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.202+0000 7fb4b5a18b80 1 -- start start 2023-10-08T09:23:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.202+0000 7fb4b5a18b80 1 Processor -- start 2023-10-08T09:23:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.205+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c6e40 0x556fb22d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.205+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.205+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.205+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb2219dd0 con 0x556fb22c82b0 2023-10-08T09:23:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.205+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb21b66b0 con 0x556fb22c6e40 2023-10-08T09:23:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.205+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb21b6350 con 0x556fb22c77a0 2023-10-08T09:23:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.205+0000 7fb4b5a18b80 10 client.0 _ll_register_callbacks cb 0x556fb20137a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.206+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c6e40 0x556fb22d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.206+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c6e40 0x556fb22d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42098/0 (socket says 172.21.15.49:42098) 2023-10-08T09:23:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.206+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/2992396089 learned_addr learned my addr 172.21.15.49:0/2992396089 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.207+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.213+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.213+0000 7fb4a37fe700 1 -- 172.21.15.49:0/2992396089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 msgr2=0x556fb22d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.213+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.213+0000 7fb4a37fe700 1 -- 172.21.15.49:0/2992396089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c6e40 msgr2=0x556fb22d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.213+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c6e40 0x556fb22d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.213+0000 7fb4a37fe700 1 -- 172.21.15.49:0/2992396089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c00e140 con 0x556fb22c82b0 2023-10-08T09:23:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.213+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d7ec0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00b010 tx=0x7fb49c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dbfc1a81992cc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.218+0000 7fb482ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:23:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.232+0000 7fb482ffd700 1 -- 172.21.15.49:0/2992396089 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c018040 con 0x556fb22c82b0 2023-10-08T09:23:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.232+0000 7fb482ffd700 1 -- 172.21.15.49:0/2992396089 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb49c009580 con 0x556fb22c82b0 2023-10-08T09:23:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.232+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fb21e1e50 con 0x556fb22c82b0 2023-10-08T09:23:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.249+0000 7fb482ffd700 1 -- 172.21.15.49:0/2992396089 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb49c004430 con 0x556fb22c82b0 2023-10-08T09:23:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.250+0000 7fb4b5a18b80 10 client.16602 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.250+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556fb202dc00 con 0x556fb22c82b0 2023-10-08T09:23:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb482ffd700 1 -- 172.21.15.49:0/2992396089 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556fb21e1e50 con 0x556fb22c82b0 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb482ffd700 1 -- 172.21.15.49:0/2992396089 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb49c007cb0 con 0x556fb22c82b0 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb4b5a18b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb4b5a18b80 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb4b5a18b80 20 client.16602 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb22dc620 0x556fb22dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb4b5a18b80 4 client.16602 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.251+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fb2212db0 con 0x556fb22dc620 2023-10-08T09:23:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.252+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb22dc620 0x556fb22dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.253+0000 7fb482ffd700 10 client.16602 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.253+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb22dc620 0x556fb22dea40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d34ec46063458ea2 server_cookie=2787483ba711ff8b in_seq=0 out_seq=0 2023-10-08T09:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.257+0000 7fb482ffd700 1 -- 172.21.15.49:0/2992396089 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556fb2212db0 con 0x556fb22dc620 2023-10-08T09:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.257+0000 7fb482ffd700 10 client.16602 handle_command_reply: tid=0 2023-10-08T09:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.257+0000 7fb4b5a18b80 1 client.16602 shutdown 2023-10-08T09:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.268+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 msgr2=0x556fb22d7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.268+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d7ec0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00b010 tx=0x7fb49c003ce0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.269+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 shutdown_connections 2023-10-08T09:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.269+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb22dc620 0x556fb22dea40 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.269+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d7ec0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.269+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c6e40 0x556fb22d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.269+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/2992396089 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.269+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 >> 172.21.15.49:0/2992396089 conn(0x556fb223b200 msgr2=0x556fb22c1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:07.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.270+0000 7fb482ffd700 0 client.16602 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.276+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 shutdown_connections 2023-10-08T09:23:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.276+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/2992396089 wait complete. 2023-10-08T09:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.281+0000 7fb4b5a18b80 20 client.16602 trim_cache size 0 max 16384 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 Processor -- start 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 -- start start 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb209a8c0 con 0x556fb22c6e40 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb22d16d0 con 0x556fb22c82b0 2023-10-08T09:23:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.308+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb2212db0 con 0x556fb22c77a0 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52492/0 (socket says 172.21.15.49:52492) 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/827914685 learned_addr learned my addr 172.21.15.49:0/827914685 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/827914685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 msgr2=0x556fb22d7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/827914685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 msgr2=0x556fb22cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/827914685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fb206b1f0 con 0x556fb22c6e40 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:23:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c8690 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb49800bc40 tx=0x7fb498010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6d6a1bcaef1902 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.309+0000 7fb48a7fc700 1 -- 172.21.15.49:0/827914685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49809dbc0 con 0x556fb22c6e40 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.310+0000 7fb48a7fc700 1 -- 172.21.15.49:0/827914685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4980a1e00 con 0x556fb22c6e40 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.310+0000 7fb48a7fc700 1 -- 172.21.15.49:0/827914685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4980991c0 con 0x556fb22c6e40 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/827914685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 msgr2=0x556fb22c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c8690 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb49800bc40 tx=0x7fb498010910 comp rx=0 tx=0).stop 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/827914685 shutdown_connections 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c82b0 0x556fb22cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c6e40 0x556fb22c8690 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/827914685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c77a0 0x556fb22d7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/827914685 >> 172.21.15.49:0/827914685 conn(0x556fb223b200 msgr2=0x556fb22e33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/827914685 shutdown_connections 2023-10-08T09:23:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/827914685 wait complete. 2023-10-08T09:23:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 -- start start 2023-10-08T09:23:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 Processor -- start 2023-10-08T09:23:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.311+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c6e40 0x556fb22d5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c77a0 0x556fb22d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4b5a18b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb22d7160 con 0x556fb22c82b0 2023-10-08T09:23:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb22d72d0 con 0x556fb22c77a0 2023-10-08T09:23:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4b5a18b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fb22d7490 con 0x556fb22c6e40 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4b5a18b80 10 client.0 _ll_register_callbacks cb 0x7fb49c009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c6e40 0x556fb22d5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c6e40 0x556fb22d5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39532/0 (socket says 172.21.15.49:39532) 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/3765336032 learned_addr learned my addr 172.21.15.49:0/3765336032 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c77a0 0x556fb22d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c77a0 msgr2=0x556fb22d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c77a0 0x556fb22d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/3765336032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 msgr2=0x556fb22d6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 -- 172.21.15.49:0/3765336032 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a41209b0 con 0x556fb22c6e40 2023-10-08T09:23:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a37fe700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb4a2ffd700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c6e40 0x556fb22d5770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb498095f70 tx=0x7fb498099450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=991ab4db6c47028a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.312+0000 7fb482ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb482ffd700 1 -- 172.21.15.49:0/3765336032 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb498099160 con 0x556fb22c6e40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb49c00f040 con 0x556fb22c6e40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb482ffd700 1 -- 172.21.15.49:0/3765336032 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb49809dbc0 con 0x556fb22c6e40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb482ffd700 1 -- 172.21.15.49:0/3765336032 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4980652d0 con 0x556fb22c6e40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 10 client.26275 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556fb2043250 con 0x556fb22c6e40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb482ffd700 1 -- 172.21.15.49:0/3765336032 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb49c00f040 con 0x556fb22c6e40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb482ffd700 1 -- 172.21.15.49:0/3765336032 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb498095860 con 0x556fb22c6e40 2023-10-08T09:23:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 10 client.26275 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 10 client.26275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 20 client.26275 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb2330b30 0x556fb2332f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 4 client.26275 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.313+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556fb2212ad0 con 0x556fb2330b30 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.314+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb2330b30 0x556fb2332f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.314+0000 7fb482ffd700 10 client.26275 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.314+0000 7fb4a27fc700 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb2330b30 0x556fb2332f50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f928ddaf6f44b4f1 server_cookie=1752903f7dae8a6e in_seq=0 out_seq=0 2023-10-08T09:23:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.396+0000 7fb482ffd700 1 -- 172.21.15.49:0/3765336032 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556fb2212ad0 con 0x556fb2330b30 2023-10-08T09:23:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.396+0000 7fb482ffd700 10 client.26275 handle_command_reply: tid=0 2023-10-08T09:23:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.396+0000 7fb4b5a18b80 1 client.26275 shutdown 2023-10-08T09:23:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c6e40 msgr2=0x556fb22d5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c6e40 0x556fb22d5770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb498095f70 tx=0x7fb498099450 comp rx=0 tx=0).stop 2023-10-08T09:23:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 shutdown_connections 2023-10-08T09:23:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556fb2330b30 0x556fb2332f50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556fb22c6e40 0x556fb22d5770 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb498095f70 tx=0x7fb498099450 comp rx=0 tx=0).stop 2023-10-08T09:23:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556fb22c82b0 0x556fb22d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 --2- 172.21.15.49:0/3765336032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556fb22c77a0 0x556fb22d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 >> 172.21.15.49:0/3765336032 conn(0x556fb223b200 msgr2=0x556fb22bdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:07.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.402+0000 7fb482ffd700 0 client.26275 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.409+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 shutdown_connections 2023-10-08T09:23:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.409+0000 7fb4b5a18b80 1 -- 172.21.15.49:0/3765336032 wait complete. 2023-10-08T09:23:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.409+0000 7fb4b5a18b80 20 client.26275 trim_cache size 0 max 16384 2023-10-08T09:23:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:23:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:23:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:23:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:23:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb48c0a1470 msgr2=0x7fb48c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb48c0a1470 0x7fb48c0a3930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb49400b2e0 tx=0x7fb494005fd0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 msgr2=0x7fb4a4117ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a4117ef0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b530 tx=0x7fb49800b840 comp rx=0 tx=0).stop 2023-10-08T09:23:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 shutdown_connections 2023-10-08T09:23:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb48c0a1470 0x7fb48c0a3930 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4a4109dd0 0x7fb4a4117ef0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb4a411d440 0x7fb4a411f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 --2- 172.21.15.49:0/1771364323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4a4110f30 0x7fb4a4118430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.411+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 >> 172.21.15.49:0/1771364323 conn(0x7fb4a406d6e0 msgr2=0x7fb4a41117a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.412+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 shutdown_connections 2023-10-08T09:23:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:07.415+0000 7fb4a982f700 1 -- 172.21.15.49:0/1771364323 wait complete. 2023-10-08T09:23:07.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:07 smithi132 ceph-mon[103389]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.0 MiB/s wr, 570 op/s 2023-10-08T09:23:07.639 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:07 smithi049 ceph-mon[99485]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.0 MiB/s wr, 570 op/s 2023-10-08T09:23:07.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:07 smithi184 ceph-mon[103675]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.0 MiB/s wr, 570 op/s 2023-10-08T09:23:09.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 465.0 s, 510.6 tps, lat 189.381 ms stddev 221.199 2023-10-08T09:23:09.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:09 smithi132 ceph-mon[103389]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 544 op/s 2023-10-08T09:23:09.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:09 smithi049 ceph-mon[99485]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 544 op/s 2023-10-08T09:23:09.753 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:09 smithi184 ceph-mon[103675]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.2 MiB/s wr, 544 op/s 2023-10-08T09:23:11.636 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:11 smithi049 ceph-mon[99485]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 484 op/s 2023-10-08T09:23:11.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:11 smithi184 ceph-mon[103675]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 484 op/s 2023-10-08T09:23:11.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:11 smithi132 ceph-mon[103389]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.0 MiB/s wr, 484 op/s 2023-10-08T09:23:13.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:12 smithi049 ceph-mon[99485]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 538 op/s 2023-10-08T09:23:13.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:12 smithi184 ceph-mon[103675]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 538 op/s 2023-10-08T09:23:13.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:12 smithi132 ceph-mon[103389]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 538 op/s 2023-10-08T09:23:14.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 470.0 s, 536.8 tps, lat 191.107 ms stddev 230.240 2023-10-08T09:23:15.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:15 smithi049 ceph-mon[99485]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 476 op/s 2023-10-08T09:23:15.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:15 smithi184 ceph-mon[103675]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 476 op/s 2023-10-08T09:23:15.803 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:15 smithi132 ceph-mon[103389]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.4 MiB/s wr, 476 op/s 2023-10-08T09:23:16.181 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:23:16.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.531+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1016812105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfa41077f0 msgr2=0x7fbfa4109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:16.533 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.531+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1016812105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfa41077f0 0x7fbfa4109c20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009990 tx=0x7fbf98009ca0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.543+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1016812105 shutdown_connections 2023-10-08T09:23:16.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.543+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1016812105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfa41077f0 0x7fbfa4109c20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.543+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1016812105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa410cb30 0x7fbfa410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.543+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1016812105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 0x7fbfa410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.543+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1016812105 >> 172.21.15.49:0/1016812105 conn(0x7fbfa406d790 msgr2=0x7fbfa406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:16.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.557+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1016812105 shutdown_connections 2023-10-08T09:23:16.559 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.557+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1016812105 wait complete. 2023-10-08T09:23:16.560 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.558+0000 7fbfaabc7700 1 Processor -- start 2023-10-08T09:23:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.563+0000 7fbfaabc7700 1 -- start start 2023-10-08T09:23:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.563+0000 7fbfaabc7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfa41077f0 0x7fbfa41a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.563+0000 7fbfaabc7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 0x7fbfa41a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.565 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.563+0000 7fbfaabc7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa410cb30 0x7fbfa41acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.563+0000 7fbfaabc7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa41ad100 con 0x7fbfa410a160 2023-10-08T09:23:16.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.563+0000 7fbfaabc7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa410ef20 con 0x7fbfa410cb30 2023-10-08T09:23:16.566 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.563+0000 7fbfaabc7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa410f0b0 con 0x7fbfa41077f0 2023-10-08T09:23:16.567 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 0x7fbfa41a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 0x7fbfa41a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52512/0 (socket says 172.21.15.49:52512) 2023-10-08T09:23:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1356909979 learned_addr learned my addr 172.21.15.49:0/1356909979 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1356909979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfa41077f0 msgr2=0x7fbfa41a55d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:23:16.568 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfa41077f0 0x7fbfa41a55d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1356909979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa410cb30 msgr2=0x7fbfa41acb30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:23:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa410cb30 0x7fbfa41acb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.564+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1356909979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf980095f0 con 0x7fbfa410a160 2023-10-08T09:23:16.569 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.565+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 0x7fbfa41a5b10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00ef10 tx=0x7fbf9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:16.576 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.571+0000 7fbfa1ffb700 1 -- 172.21.15.49:0/1356909979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00cad0 con 0x7fbfa410a160 2023-10-08T09:23:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.571+0000 7fbfa1ffb700 1 -- 172.21.15.49:0/1356909979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c00f040 con 0x7fbfa410a160 2023-10-08T09:23:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.571+0000 7fbfa1ffb700 1 -- 172.21.15.49:0/1356909979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0146e0 con 0x7fbfa410a160 2023-10-08T09:23:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.572+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa410f330 con 0x7fbfa410a160 2023-10-08T09:23:16.578 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.572+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa410f7c0 con 0x7fbfa410a160 2023-10-08T09:23:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.580+0000 7fbfa1ffb700 1 -- 172.21.15.49:0/1356909979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbf9c004ad0 con 0x7fbfa410a160 2023-10-08T09:23:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.581+0000 7fbfa1ffb700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf8c0a1680 0x7fbf8c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.583 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.581+0000 7fbfa1ffb700 1 -- 172.21.15.49:0/1356909979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbf9c0cc9e0 con 0x7fbfa410a160 2023-10-08T09:23:16.584 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.581+0000 7fbfa8963700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf8c0a1680 0x7fbf8c0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.587+0000 7fbfb6db0b80 1 Processor -- start 2023-10-08T09:23:16.589 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.587+0000 7fbfa8963700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf8c0a1680 0x7fbf8c0a3b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf980095c0 tx=0x7fbf98009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:16.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfb6db0b80 1 -- start start 2023-10-08T09:23:16.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f500480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.596 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f5009c0 0x55618f501270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f501800 0x55618f501c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f468460 con 0x55618f501800 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f459c20 con 0x55618f5009c0 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f457540 con 0x55618f500060 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfa9164700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f501800 0x55618f501c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfa9164700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f501800 0x55618f501c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52520/0 (socket says 172.21.15.49:52520) 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfa9164700 1 -- 172.21.15.49:0/3623305602 learned_addr learned my addr 172.21.15.49:0/3623305602 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:16.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfa8963700 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f500480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.592+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f5009c0 0x55618f501270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.593+0000 7fbfa8963700 1 -- 172.21.15.49:0/3623305602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f5009c0 msgr2=0x55618f501270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:16.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.593+0000 7fbfa8963700 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f5009c0 0x55618f501270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.593+0000 7fbfa8963700 1 -- 172.21.15.49:0/3623305602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f501800 msgr2=0x55618f501c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:16.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.593+0000 7fbfa8963700 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f501800 0x55618f501c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.593+0000 7fbfa8963700 1 -- 172.21.15.49:0/3623305602 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618f27dde0 con 0x55618f500060 2023-10-08T09:23:16.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.593+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f5009c0 0x55618f501270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:23:16.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.598+0000 7fbfa8963700 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f500480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b2e0 tx=0x7fbf9801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709bd4664abd94e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:16.600 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.599+0000 7fbf8affd700 1 -- 172.21.15.49:0/3623305602 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9801d070 con 0x55618f500060 2023-10-08T09:23:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.599+0000 7fbf8affd700 1 -- 172.21.15.49:0/3623305602 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9800bea0 con 0x55618f500060 2023-10-08T09:23:16.601 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.599+0000 7fbf8affd700 1 -- 172.21.15.49:0/3623305602 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800f750 con 0x55618f500060 2023-10-08T09:23:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.600+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3623305602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 msgr2=0x55618f500480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:16.602 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.600+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f500480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b2e0 tx=0x7fbf9801d000 comp rx=0 tx=0).stop 2023-10-08T09:23:16.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.604+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3623305602 shutdown_connections 2023-10-08T09:23:16.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.604+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f500480 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.604+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f501800 0x55618f501c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.604+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3623305602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f5009c0 0x55618f501270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.606 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.604+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3623305602 >> 172.21.15.49:0/3623305602 conn(0x55618f474260 msgr2=0x55618f4fae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:16.614 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.613+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3623305602 shutdown_connections 2023-10-08T09:23:16.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.613+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3623305602 wait complete. 2023-10-08T09:23:16.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.613+0000 7fbfb6db0b80 1 -- start start 2023-10-08T09:23:16.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.613+0000 7fbfb6db0b80 1 Processor -- start 2023-10-08T09:23:16.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f509e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.616 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f452e30 con 0x55618f5009c0 2023-10-08T09:23:16.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f3ef710 con 0x55618f501800 2023-10-08T09:23:16.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f3ef3b0 con 0x55618f500060 2023-10-08T09:23:16.617 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfb6db0b80 10 client.0 _ll_register_callbacks cb 0x55618f24c800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:16.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfa9164700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfa9164700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42138/0 (socket says 172.21.15.49:42138) 2023-10-08T09:23:16.618 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.614+0000 7fbfa9164700 1 -- 172.21.15.49:0/3218262569 learned_addr learned my addr 172.21.15.49:0/3218262569 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:16.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbfa9164700 1 -- 172.21.15.49:0/3218262569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 msgr2=0x55618f509e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:23:16.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbfa9164700 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f509e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbfa9164700 1 -- 172.21.15.49:0/3218262569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 msgr2=0x55618f50a370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:23:16.619 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbfa9164700 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50a370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbfa9164700 1 -- 172.21.15.49:0/3218262569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa410dae0 con 0x55618f501800 2023-10-08T09:23:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbfa9164700 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5113b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e480 tx=0x7fbf9400e840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d705d67aa4745d9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbf83fff700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:23:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbf83fff700 1 -- 172.21.15.49:0/3218262569 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94009e40 con 0x55618f501800 2023-10-08T09:23:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbf83fff700 1 -- 172.21.15.49:0/3218262569 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbf94013680 con 0x55618f501800 2023-10-08T09:23:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55618f41aeb0 con 0x55618f501800 2023-10-08T09:23:16.620 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.615+0000 7fbf83fff700 1 -- 172.21.15.49:0/3218262569 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf94013840 con 0x55618f501800 2023-10-08T09:23:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.620+0000 7fbfb6db0b80 10 client.26234 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:16.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.620+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55618f266c60 con 0x55618f501800 2023-10-08T09:23:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.629+0000 7fbf83fff700 1 -- 172.21.15.49:0/3218262569 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55618f41aeb0 con 0x55618f501800 2023-10-08T09:23:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.629+0000 7fbf83fff700 1 -- 172.21.15.49:0/3218262569 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbf94011070 con 0x55618f501800 2023-10-08T09:23:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.636+0000 7fbfb6db0b80 10 client.26234 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:16.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.636+0000 7fbfb6db0b80 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:16.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.636+0000 7fbfb6db0b80 20 client.26234 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:16.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.636+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f515bf0 0x55618f518010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.636+0000 7fbfb6db0b80 4 client.26234 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:23:16.640 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.636+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55618f44be10 con 0x55618f515bf0 2023-10-08T09:23:16.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.638+0000 7fbfa8963700 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f515bf0 0x55618f518010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.647+0000 7fbfa8963700 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f515bf0 0x55618f518010 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1829b17a55a94 server_cookie=2d46a9f7bf6779c8 in_seq=0 out_seq=0 2023-10-08T09:23:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.647+0000 7fbf83fff700 10 client.26234 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.647+0000 7fbf83fff700 1 -- 172.21.15.49:0/3218262569 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55618f44be10 con 0x55618f515bf0 2023-10-08T09:23:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.647+0000 7fbf83fff700 10 client.26234 handle_command_reply: tid=0 2023-10-08T09:23:16.653 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.648+0000 7fbfb6db0b80 1 client.26234 shutdown 2023-10-08T09:23:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.659+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 msgr2=0x55618f5113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:16.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.659+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5113b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400e480 tx=0x7fbf9400e840 comp rx=0 tx=0).stop 2023-10-08T09:23:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.664+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 shutdown_connections 2023-10-08T09:23:16.665 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.664+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f515bf0 0x55618f518010 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.664+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f509e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.664+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5113b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.664+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/3218262569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.664+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 >> 172.21.15.49:0/3218262569 conn(0x55618f474260 msgr2=0x55618f4fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:16.666 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.664+0000 7fbf83fff700 0 client.26234 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:16.673 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.671+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 shutdown_connections 2023-10-08T09:23:16.675 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.673+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/3218262569 wait complete. 2023-10-08T09:23:16.688 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.673+0000 7fbfb6db0b80 20 client.26234 trim_cache size 0 max 16384 2023-10-08T09:23:16.710 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.708+0000 7fbfb6db0b80 1 Processor -- start 2023-10-08T09:23:16.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.709+0000 7fbfb6db0b80 1 -- start start 2023-10-08T09:23:16.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.709+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f500060 0x55618f509340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.709+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f511370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.709+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f501800 0x55618f4f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.709+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f2d3920 con 0x55618f5009c0 2023-10-08T09:23:16.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.709+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f44be10 con 0x55618f500060 2023-10-08T09:23:16.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.709+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f41aeb0 con 0x55618f501800 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.710+0000 7fbfa9164700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f501800 0x55618f4f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.710+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f511370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.710+0000 7fbfa9164700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f501800 0x55618f4f9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39562/0 (socket says 172.21.15.49:39562) 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.710+0000 7fbfa9164700 1 -- 172.21.15.49:0/50209290 learned_addr learned my addr 172.21.15.49:0/50209290 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.710+0000 7fbfa8963700 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f500060 0x55618f509340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.711+0000 7fbfa3fff700 1 -- 172.21.15.49:0/50209290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f501800 msgr2=0x55618f4f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.711+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f501800 0x55618f4f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.711+0000 7fbfa3fff700 1 -- 172.21.15.49:0/50209290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f500060 msgr2=0x55618f509340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.711+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f500060 0x55618f509340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.711+0000 7fbfa3fff700 1 -- 172.21.15.49:0/50209290 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618f2a4250 con 0x55618f5009c0 2023-10-08T09:23:16.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.711+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f511370 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00eee0 tx=0x7fbf9c099850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae43b6e1062abffb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.711+0000 7fbf8b7fe700 1 -- 172.21.15.49:0/50209290 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c099360 con 0x55618f5009c0 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.712+0000 7fbf8b7fe700 1 -- 172.21.15.49:0/50209290 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9c09a100 con 0x55618f5009c0 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.712+0000 7fbf8b7fe700 1 -- 172.21.15.49:0/50209290 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00cc30 con 0x55618f5009c0 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.712+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/50209290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 msgr2=0x55618f511370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.712+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f511370 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00eee0 tx=0x7fbf9c099850 comp rx=0 tx=0).stop 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/50209290 shutdown_connections 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f501800 0x55618f4f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f500060 0x55618f509340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.717 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/50209290 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f511370 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/50209290 >> 172.21.15.49:0/50209290 conn(0x55618f474260 msgr2=0x55618f4f6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:16.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/50209290 shutdown_connections 2023-10-08T09:23:16.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/50209290 wait complete. 2023-10-08T09:23:16.718 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 -- start start 2023-10-08T09:23:16.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.713+0000 7fbfb6db0b80 1 Processor -- start 2023-10-08T09:23:16.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f511410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.723 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfb6db0b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f510780 con 0x55618f5009c0 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f5108f0 con 0x55618f501800 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfb6db0b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618f569a90 con 0x55618f500060 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfb6db0b80 10 client.0 _ll_register_callbacks cb 0x7fbf940099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfa3fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52534/0 (socket says 172.21.15.49:52534) 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1136283058 learned_addr learned my addr 172.21.15.49:0/1136283058 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:16.724 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.714+0000 7fbfa8963700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f511410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa9164700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1136283058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 msgr2=0x55618f511410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:16.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f511410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 msgr2=0x55618f5100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:16.725 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa3fff700 1 -- 172.21.15.49:0/1136283058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c0b7e70 con 0x55618f5009c0 2023-10-08T09:23:16.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa9164700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:23:16.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfa3fff700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50fa20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0a17d0 tx=0x7fbf9c09db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2c09eff9d2020e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:16.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbf83fff700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:23:16.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbf83fff700 1 -- 172.21.15.49:0/1136283058 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0b2270 con 0x55618f5009c0 2023-10-08T09:23:16.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbf83fff700 1 -- 172.21.15.49:0/1136283058 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbf9c0a1d00 con 0x55618f5009c0 2023-10-08T09:23:16.726 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf9800bea0 con 0x55618f5009c0 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbf83fff700 1 -- 172.21.15.49:0/1136283058 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9c099a00 con 0x55618f5009c0 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.715+0000 7fbfb6db0b80 10 client.16623 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55618f27c2b0 con 0x55618f5009c0 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbf83fff700 1 -- 172.21.15.49:0/1136283058 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf9800bea0 con 0x55618f5009c0 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbf83fff700 1 -- 172.21.15.49:0/1136283058 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbf9c098960 con 0x55618f5009c0 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfb6db0b80 10 client.16623 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfb6db0b80 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfb6db0b80 20 client.16623 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:16.727 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f56b1b0 0x55618f56d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfb6db0b80 4 client.16623 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55618f44bb30 con 0x55618f56b1b0 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.716+0000 7fbfa8963700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f56b1b0 0x55618f56d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbf83fff700 10 client.16623 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfa8963700 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f56b1b0 0x55618f56d5d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44788e1a7b91effe server_cookie=2df2814860422ae3 in_seq=0 out_seq=0 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbf83fff700 1 -- 172.21.15.49:0/1136283058 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55618f44bb30 con 0x55618f56b1b0 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbf83fff700 10 client.16623 handle_command_reply: tid=0 2023-10-08T09:23:16.728 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 client.16623 shutdown 2023-10-08T09:23:16.729 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 msgr2=0x55618f50fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:16.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50fa20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0a17d0 tx=0x7fbf9c09db50 comp rx=0 tx=0).stop 2023-10-08T09:23:16.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 shutdown_connections 2023-10-08T09:23:16.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55618f56b1b0 0x55618f56d5d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55618f500060 0x55618f511410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55618f501800 0x55618f5100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.730 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 --2- 172.21.15.49:0/1136283058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55618f5009c0 0x55618f50fa20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 >> 172.21.15.49:0/1136283058 conn(0x55618f474260 msgr2=0x55618f509dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:16.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbf83fff700 0 client.16623 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:16.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 shutdown_connections 2023-10-08T09:23:16.731 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.717+0000 7fbfb6db0b80 1 -- 172.21.15.49:0/1136283058 wait complete. 2023-10-08T09:23:16.732 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.718+0000 7fbfb6db0b80 20 client.16623 trim_cache size 0 max 16384 2023-10-08T09:23:16.732 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:23:16.732 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:23:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:23:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:23:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:23:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:23:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:23:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:23:16.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:23:16.734 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:23:16.734 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.718+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf8c0a1680 msgr2=0x7fbf8c0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:16.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.718+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf8c0a1680 0x7fbf8c0a3b40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf980095c0 tx=0x7fbf98009550 comp rx=0 tx=0).stop 2023-10-08T09:23:16.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.718+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 msgr2=0x7fbfa41a5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:16.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.718+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 0x7fbfa41a5b10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00ef10 tx=0x7fbf9c00c3a0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.719+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 shutdown_connections 2023-10-08T09:23:16.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.719+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbf8c0a1680 0x7fbf8c0a3b40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.735 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.719+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfa41077f0 0x7fbfa41a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.719+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa410cb30 0x7fbfa41acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.719+0000 7fbfaabc7700 1 --2- 172.21.15.49:0/1356909979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbfa410a160 0x7fbfa41a5b10 secure :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00ef10 tx=0x7fbf9c00c3a0 comp rx=0 tx=0).stop 2023-10-08T09:23:16.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.719+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 >> 172.21.15.49:0/1356909979 conn(0x7fbfa406d790 msgr2=0x7fbfa410b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:16.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.723+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 shutdown_connections 2023-10-08T09:23:16.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:16.723+0000 7fbfaabc7700 1 -- 172.21.15.49:0/1356909979 wait complete. 2023-10-08T09:23:16.741 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:23:17.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:17 smithi049 ceph-mon[99485]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.5 MiB/s wr, 541 op/s 2023-10-08T09:23:17.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:17 smithi184 ceph-mon[103675]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.5 MiB/s wr, 541 op/s 2023-10-08T09:23:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:17 smithi132 ceph-mon[103389]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.5 MiB/s wr, 541 op/s 2023-10-08T09:23:18.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:18 smithi184 ceph-mon[103675]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 532 op/s 2023-10-08T09:23:18.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:18 smithi132 ceph-mon[103389]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 532 op/s 2023-10-08T09:23:18.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:18 smithi049 ceph-mon[99485]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 532 op/s 2023-10-08T09:23:19.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 475.0 s, 596.6 tps, lat 162.210 ms stddev 178.512 2023-10-08T09:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 -- 172.21.15.49:0/807383648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c108150 msgr2=0x7fed6c10eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 --2- 172.21.15.49:0/807383648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c108150 0x7fed6c10eb30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009990 tx=0x7fed5c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 -- 172.21.15.49:0/807383648 shutdown_connections 2023-10-08T09:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 --2- 172.21.15.49:0/807383648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c10f070 0x7fed6c111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 --2- 172.21.15.49:0/807383648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c1077f0 0x7fed6c107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 --2- 172.21.15.49:0/807383648 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c108150 0x7fed6c10eb30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 -- 172.21.15.49:0/807383648 >> 172.21.15.49:0/807383648 conn(0x7fed6c06d770 msgr2=0x7fed6c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 -- 172.21.15.49:0/807383648 shutdown_connections 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.628+0000 7fed720cf700 1 -- 172.21.15.49:0/807383648 wait complete. 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 Processor -- start 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 -- start start 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c1077f0 0x7fed6c1a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c108150 0x7fed6c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c10f070 0x7fed6c1a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1a8ba0 con 0x7fed6c1077f0 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1a8ce0 con 0x7fed6c10f070 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed720cf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1a8e70 con 0x7fed6c108150 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.629+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c1077f0 0x7fed6c1a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c108150 0x7fed6c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c108150 0x7fed6c1a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39584/0 (socket says 172.21.15.49:39584) 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c1077f0 0x7fed6c1a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52548/0 (socket says 172.21.15.49:52548) 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 -- 172.21.15.49:0/4232326595 learned_addr learned my addr 172.21.15.49:0/4232326595 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6bfff700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c10f070 0x7fed6c1a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 -- 172.21.15.49:0/4232326595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c10f070 msgr2=0x7fed6c1a85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c10f070 0x7fed6c1a85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 -- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c1077f0 msgr2=0x7fed6c1a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c1077f0 0x7fed6c1a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 -- 172.21.15.49:0/4232326595 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0095f0 con 0x7fed6c108150 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6bfff700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c10f070 0x7fed6c1a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed6affd700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c108150 0x7fed6c1a15b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098e0 tx=0x7fed5c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed68ff9700 1 -- 172.21.15.49:0/4232326595 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c01d070 con 0x7fed6c108150 2023-10-08T09:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed68ff9700 1 -- 172.21.15.49:0/4232326595 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c022470 con 0x7fed6c108150 2023-10-08T09:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c1a90f0 con 0x7fed6c108150 2023-10-08T09:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed68ff9700 1 -- 172.21.15.49:0/4232326595 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c017650 con 0x7fed6c108150 2023-10-08T09:23:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.630+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c1a9510 con 0x7fed6c108150 2023-10-08T09:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.633+0000 7fed68ff9700 1 -- 172.21.15.49:0/4232326595 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fed5c022850 con 0x7fed6c108150 2023-10-08T09:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.634+0000 7fed68ff9700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fed540a1480 0x7fed540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.634+0000 7fed68ff9700 1 -- 172.21.15.49:0/4232326595 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fed5c0ce550 con 0x7fed6c108150 2023-10-08T09:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.634+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fed540a1480 0x7fed540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.635+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fed540a1480 0x7fed540a3940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fed6400f540 tx=0x7fed6400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.635+0000 7fed7e2b8b80 1 Processor -- start 2023-10-08T09:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.635+0000 7fed7e2b8b80 1 -- start start 2023-10-08T09:23:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0be40 0x55a49ea0c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0c7a0 0x55a49ea0ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e974400 con 0x55a49ea0be40 2023-10-08T09:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e965bc0 con 0x55a49ea0d2b0 2023-10-08T09:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e9634e0 con 0x55a49ea0c7a0 2023-10-08T09:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0be40 0x55a49ea0c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6affd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0c7a0 0x55a49ea0ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0be40 0x55a49ea0c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52556/0 (socket says 172.21.15.49:52556) 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 -- 172.21.15.49:0/2954129129 learned_addr learned my addr 172.21.15.49:0/2954129129 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6bfff700 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 -- 172.21.15.49:0/2954129129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0c7a0 msgr2=0x55a49ea0ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0c7a0 0x55a49ea0ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 -- 172.21.15.49:0/2954129129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 msgr2=0x55a49ea0d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.636+0000 7fed6b7fe700 1 -- 172.21.15.49:0/2954129129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49e789d80 con 0x55a49ea0be40 2023-10-08T09:23:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.637+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0be40 0x55a49ea0c260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fed64003d20 tx=0x7fed64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20567c719ad87e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.638+0000 7fed51ffb700 1 -- 172.21.15.49:0/2954129129 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400e040 con 0x55a49ea0be40 2023-10-08T09:23:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.638+0000 7fed51ffb700 1 -- 172.21.15.49:0/2954129129 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6401b070 con 0x55a49ea0be40 2023-10-08T09:23:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.638+0000 7fed51ffb700 1 -- 172.21.15.49:0/2954129129 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64008ea0 con 0x55a49ea0be40 2023-10-08T09:23:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.639+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/2954129129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0be40 msgr2=0x55a49ea0c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.639+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0be40 0x55a49ea0c260 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fed64003d20 tx=0x7fed64014000 comp rx=0 tx=0).stop 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.647+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/2954129129 shutdown_connections 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.647+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.647+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0be40 0x55a49ea0c260 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.647+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/2954129129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0c7a0 0x55a49ea0ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.647+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/2954129129 >> 172.21.15.49:0/2954129129 conn(0x55a49e980200 msgr2=0x55a49ea068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.647+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/2954129129 shutdown_connections 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.651+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/2954129129 wait complete. 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.651+0000 7fed7e2b8b80 1 -- start start 2023-10-08T09:23:20.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.651+0000 7fed7e2b8b80 1 Processor -- start 2023-10-08T09:23:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.664+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea1e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.664+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea1e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.664+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.664+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e95edd0 con 0x55a49ea0d2b0 2023-10-08T09:23:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.664+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e8fb6b0 con 0x55a49ea0c7a0 2023-10-08T09:23:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.664+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e8fb350 con 0x55a49ea0be40 2023-10-08T09:23:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.664+0000 7fed7e2b8b80 10 client.0 _ll_register_callbacks cb 0x55a49e7587a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.676+0000 7fed6affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea1e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.677+0000 7fed6affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea1e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42176/0 (socket says 172.21.15.49:42176) 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.677+0000 7fed6affd700 1 -- 172.21.15.49:0/3892458492 learned_addr learned my addr 172.21.15.49:0/3892458492 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.677+0000 7fed6bfff700 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.677+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea1e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.681+0000 7fed6b7fe700 1 -- 172.21.15.49:0/3892458492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 msgr2=0x55a49ea1e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.681+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea1e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.681+0000 7fed6b7fe700 1 -- 172.21.15.49:0/3892458492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 msgr2=0x55a49ea25660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.681+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea25660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.681+0000 7fed6b7fe700 1 -- 172.21.15.49:0/3892458492 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64003a00 con 0x55a49ea0be40 2023-10-08T09:23:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.681+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea1e0e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed64009690 tx=0x7fed64009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d7d07a3611ac6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.683+0000 7fed4affd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:23:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.683+0000 7fed4affd700 1 -- 172.21.15.49:0/3892458492 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400e070 con 0x55a49ea0be40 2023-10-08T09:23:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.683+0000 7fed4affd700 1 -- 172.21.15.49:0/3892458492 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fed64015450 con 0x55a49ea0be40 2023-10-08T09:23:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.684+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a49e926e50 con 0x55a49ea0be40 2023-10-08T09:23:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.684+0000 7fed4affd700 1 -- 172.21.15.49:0/3892458492 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed64004a90 con 0x55a49ea0be40 2023-10-08T09:23:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.684+0000 7fed7e2b8b80 10 client.26299 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.684+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a49e772c00 con 0x55a49ea0be40 2023-10-08T09:23:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.684+0000 7fed4affd700 1 -- 172.21.15.49:0/3892458492 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a49e926e50 con 0x55a49ea0be40 2023-10-08T09:23:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.685+0000 7fed4affd700 1 -- 172.21.15.49:0/3892458492 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed64015d20 con 0x55a49ea0be40 2023-10-08T09:23:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.685+0000 7fed7e2b8b80 10 client.26299 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.685+0000 7fed7e2b8b80 10 client.26299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.685+0000 7fed7e2b8b80 20 client.26299 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.685+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea18810 0x55a49ea1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.685+0000 7fed7e2b8b80 4 client.26299 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:23:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.685+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a49e957db0 con 0x55a49ea18810 2023-10-08T09:23:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.689+0000 7fed6affd700 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea18810 0x55a49ea1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.719+0000 7fed4affd700 10 client.26299 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.719+0000 7fed6affd700 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea18810 0x55a49ea1ac00 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50f4bc8aa36485e server_cookie=bfaf48ff9b08ebd0 in_seq=0 out_seq=0 2023-10-08T09:23:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.724+0000 7fed4affd700 1 -- 172.21.15.49:0/3892458492 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a49e957db0 con 0x55a49ea18810 2023-10-08T09:23:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.724+0000 7fed4affd700 10 client.26299 handle_command_reply: tid=0 2023-10-08T09:23:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.724+0000 7fed7e2b8b80 1 client.26299 shutdown 2023-10-08T09:23:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 msgr2=0x55a49ea1e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea1e0e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed64009690 tx=0x7fed64009260 comp rx=0 tx=0).stop 2023-10-08T09:23:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 shutdown_connections 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea18810 0x55a49ea1ac00 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea1e0e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed4affd700 0 client.26299 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3892458492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea1e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 >> 172.21.15.49:0/3892458492 conn(0x55a49e980200 msgr2=0x55a49ea06010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 shutdown_connections 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3892458492 wait complete. 2023-10-08T09:23:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.725+0000 7fed7e2b8b80 20 client.26299 trim_cache size 0 max 16384 2023-10-08T09:23:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.751+0000 7fed7e2b8b80 1 Processor -- start 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.751+0000 7fed7e2b8b80 1 -- start start 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea0c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea25620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e7df8c0 con 0x55a49ea0d2b0 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49ea0cb80 con 0x55a49ea0c7a0 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49e957db0 con 0x55a49ea0be40 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea0c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea0c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39604/0 (socket says 172.21.15.49:39604) 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52572/0 (socket says 172.21.15.49:52572) 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6b7fe700 1 -- 172.21.15.49:0/844625169 learned_addr learned my addr 172.21.15.49:0/844625169 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6bfff700 1 -- 172.21.15.49:0/844625169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 msgr2=0x55a49ea0c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6bfff700 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea0c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6bfff700 1 -- 172.21.15.49:0/844625169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 msgr2=0x55a49ea25620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6bfff700 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea25620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.752+0000 7fed6bfff700 1 -- 172.21.15.49:0/844625169 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49e7b01f0 con 0x55a49ea0d2b0 2023-10-08T09:23:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.753+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:23:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.753+0000 7fed6bfff700 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d690 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fed60009da0 tx=0x7fed60009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8321030fcc20b39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.753+0000 7fed527fc700 1 -- 172.21.15.49:0/844625169 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60020070 con 0x55a49ea0d2b0 2023-10-08T09:23:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.753+0000 7fed527fc700 1 -- 172.21.15.49:0/844625169 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed60018040 con 0x55a49ea0d2b0 2023-10-08T09:23:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.753+0000 7fed527fc700 1 -- 172.21.15.49:0/844625169 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60017050 con 0x55a49ea0d2b0 2023-10-08T09:23:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.754+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/844625169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 msgr2=0x55a49ea0d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.754+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d690 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fed60009da0 tx=0x7fed60009260 comp rx=0 tx=0).stop 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.754+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/844625169 shutdown_connections 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.754+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea0c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.754+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea0d2b0 0x55a49ea0d690 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.754+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/844625169 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0c7a0 0x55a49ea25620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.754+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/844625169 >> 172.21.15.49:0/844625169 conn(0x55a49e980200 msgr2=0x55a49ea138a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.755+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/844625169 shutdown_connections 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.755+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/844625169 wait complete. 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.755+0000 7fed7e2b8b80 1 -- start start 2023-10-08T09:23:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.755+0000 7fed7e2b8b80 1 Processor -- start 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea23750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed7e2b8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea256d0 0x55a49ea23c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49ea24370 con 0x55a49ea256d0 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49ea244e0 con 0x55a49ea0d2b0 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed7e2b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a49ea85c60 con 0x55a49ea0be40 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed7e2b8b80 10 client.0 _ll_register_callbacks cb 0x7fed60009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed6affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea23750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed6affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea23750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42190/0 (socket says 172.21.15.49:42190) 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed6b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea250f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39608/0 (socket says 172.21.15.49:39608) 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.756+0000 7fed6affd700 1 -- 172.21.15.49:0/3174357138 learned_addr learned my addr 172.21.15.49:0/3174357138 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6bfff700 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea256d0 0x55a49ea23c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6b7fe700 1 -- 172.21.15.49:0/3174357138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 msgr2=0x55a49ea23750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea23750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6b7fe700 1 -- 172.21.15.49:0/3174357138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea256d0 msgr2=0x55a49ea23c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea256d0 0x55a49ea23c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6b7fe700 1 -- 172.21.15.49:0/3174357138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a49e772c00 con 0x55a49ea0be40 2023-10-08T09:23:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6bfff700 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea256d0 0x55a49ea23c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:23:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed6b7fe700 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea250f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed64010010 tx=0x7fed64004850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e22218f2d37216 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed4affd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:23:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed4affd700 1 -- 172.21.15.49:0/3174357138 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400e040 con 0x55a49ea0be40 2023-10-08T09:23:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed4affd700 1 -- 172.21.15.49:0/3174357138 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fed64008930 con 0x55a49ea0be40 2023-10-08T09:23:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.757+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed6401b070 con 0x55a49ea0be40 2023-10-08T09:23:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.758+0000 7fed4affd700 1 -- 172.21.15.49:0/3174357138 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed64022ac0 con 0x55a49ea0be40 2023-10-08T09:23:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 10 client.26305 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7fed60009a50 con 0x55a49ea0be40 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed4affd700 1 -- 172.21.15.49:0/3174357138 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6401b070 con 0x55a49ea0be40 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed4affd700 1 -- 172.21.15.49:0/3174357138 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fed64004310 con 0x55a49ea0be40 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 10 client.26305 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 10 client.26305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 20 client.26305 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea870e0 0x55a49ea894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 4 client.26305 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.759+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a49e957ad0 con 0x55a49ea870e0 2023-10-08T09:23:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.760+0000 7fed6affd700 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea870e0 0x55a49ea894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.760+0000 7fed6affd700 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea870e0 0x55a49ea894d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e36f89b3d55b44 server_cookie=43561a9886810540 in_seq=0 out_seq=0 2023-10-08T09:23:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.760+0000 7fed4affd700 10 client.26305 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed4affd700 1 -- 172.21.15.49:0/3174357138 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a49e957ad0 con 0x55a49ea870e0 2023-10-08T09:23:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed4affd700 10 client.26305 handle_command_reply: tid=0 2023-10-08T09:23:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 client.26305 shutdown 2023-10-08T09:23:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 msgr2=0x55a49ea250f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea250f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed64010010 tx=0x7fed64004850 comp rx=0 tx=0).stop 2023-10-08T09:23:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 shutdown_connections 2023-10-08T09:23:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a49ea870e0 0x55a49ea894d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a49ea0be40 0x55a49ea250f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed4affd700 0 client.26305 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a49ea256d0 0x55a49ea23c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 --2- 172.21.15.49:0/3174357138 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a49ea0d2b0 0x55a49ea23750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.774+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 >> 172.21.15.49:0/3174357138 conn(0x55a49e980200 msgr2=0x55a49ea1d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.775+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 shutdown_connections 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.775+0000 7fed7e2b8b80 1 -- 172.21.15.49:0/3174357138 wait complete. 2023-10-08T09:23:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.775+0000 7fed7e2b8b80 20 client.26305 trim_cache size 0 max 16384 2023-10-08T09:23:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:23:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:23:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:23:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:23:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.775+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fed540a1480 msgr2=0x7fed540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.775+0000 7fed720cf700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fed540a1480 0x7fed540a3940 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fed6400f540 tx=0x7fed6400b040 comp rx=0 tx=0).stop 2023-10-08T09:23:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.775+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c108150 msgr2=0x7fed6c1a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.775+0000 7fed720cf700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c108150 0x7fed6c1a15b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0098e0 tx=0x7fed5c005720 comp rx=0 tx=0).stop 2023-10-08T09:23:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.777+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 shutdown_connections 2023-10-08T09:23:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.777+0000 7fed720cf700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fed540a1480 0x7fed540a3940 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.777+0000 7fed720cf700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed6c10f070 0x7fed6c1a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.777+0000 7fed720cf700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed6c1077f0 0x7fed6c1a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.777+0000 7fed720cf700 1 --2- 172.21.15.49:0/4232326595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed6c108150 0x7fed6c1a15b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.777+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 >> 172.21.15.49:0/4232326595 conn(0x7fed6c06d770 msgr2=0x7fed6c06f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.778+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 shutdown_connections 2023-10-08T09:23:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:20.778+0000 7fed720cf700 1 -- 172.21.15.49:0/4232326595 wait complete. 2023-10-08T09:23:21.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:21 smithi049 ceph-mon[99485]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 478 op/s 2023-10-08T09:23:21.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:21 smithi184 ceph-mon[103675]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 478 op/s 2023-10-08T09:23:21.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:21 smithi132 ceph-mon[103389]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.7 MiB/s wr, 478 op/s 2023-10-08T09:23:23.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:22 smithi049 ceph-mon[99485]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-10-08T09:23:23.250 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:22 smithi184 ceph-mon[103675]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-10-08T09:23:23.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:22 smithi132 ceph-mon[103389]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.8 MiB/s wr, 588 op/s 2023-10-08T09:23:24.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 480.0 s, 523.5 tps, lat 191.626 ms stddev 250.451 2023-10-08T09:23:25.631 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:25 smithi049 ceph-mon[99485]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 473 op/s 2023-10-08T09:23:25.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:25 smithi184 ceph-mon[103675]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 473 op/s 2023-10-08T09:23:25.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:25 smithi132 ceph-mon[103389]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 473 op/s 2023-10-08T09:23:26.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:26 smithi049 ceph-mon[99485]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 587 op/s 2023-10-08T09:23:26.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:26 smithi184 ceph-mon[103675]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 587 op/s 2023-10-08T09:23:26.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:26 smithi132 ceph-mon[103389]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.2 MiB/s wr, 587 op/s 2023-10-08T09:23:28.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:28 smithi184 ceph-mon[103675]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2023-10-08T09:23:28.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:28 smithi132 ceph-mon[103389]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2023-10-08T09:23:28.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:28 smithi049 ceph-mon[99485]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 584 op/s 2023-10-08T09:23:29.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 485.0 s, 526.2 tps, lat 196.990 ms stddev 242.497 2023-10-08T09:23:31.634 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:31 smithi049 ceph-mon[99485]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 506 op/s 2023-10-08T09:23:31.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:31 smithi184 ceph-mon[103675]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 506 op/s 2023-10-08T09:23:31.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:31 smithi132 ceph-mon[103389]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.5 MiB/s wr, 506 op/s 2023-10-08T09:23:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:32 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:23:33.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:32 smithi049 ceph-mon[99485]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2023-10-08T09:23:33.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:32 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:23:33.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:32 smithi184 ceph-mon[103675]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2023-10-08T09:23:33.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:32 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:23:33.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:32 smithi132 ceph-mon[103389]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 608 op/s 2023-10-08T09:23:34.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:33 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:23:34.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:33 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:23:34.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:33 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:23:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.132+0000 7fd20cf2f700 1 -- 172.21.15.49:0/1312038087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd208075510 msgr2=0x7fd2080779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.132+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/1312038087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd208075510 0x7fd2080779a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00b210 tx=0x7fd1fc00b520 comp rx=0 tx=0).stop 2023-10-08T09:23:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.143+0000 7fd20cf2f700 1 -- 172.21.15.49:0/1312038087 shutdown_connections 2023-10-08T09:23:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.143+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/1312038087 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2080725e0 0x7fd208072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.143+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/1312038087 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208077ee0 0x7fd20807a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.143+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/1312038087 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd208075510 0x7fd2080779a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.143+0000 7fd20cf2f700 1 -- 172.21.15.49:0/1312038087 >> 172.21.15.49:0/1312038087 conn(0x7fd20806d770 msgr2=0x7fd20806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.159+0000 7fd20cf2f700 1 -- 172.21.15.49:0/1312038087 shutdown_connections 2023-10-08T09:23:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.161+0000 7fd20cf2f700 1 -- 172.21.15.49:0/1312038087 wait complete. 2023-10-08T09:23:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 Processor -- start 2023-10-08T09:23:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 -- start start 2023-10-08T09:23:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2080725e0 0x7fd2081b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208075510 0x7fd2081b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd208077ee0 0x7fd2081b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081b58a0 con 0x7fd208075510 2023-10-08T09:23:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081b5a10 con 0x7fd2080725e0 2023-10-08T09:23:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20cf2f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081b5bd0 con 0x7fd208077ee0 2023-10-08T09:23:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2080725e0 0x7fd2081b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208075510 0x7fd2081b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd20659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2080725e0 0x7fd2081b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42202/0 (socket says 172.21.15.49:42202) 2023-10-08T09:23:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.162+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208075510 0x7fd2081b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52590/0 (socket says 172.21.15.49:52590) 2023-10-08T09:23:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.163+0000 7fd20659c700 1 -- 172.21.15.49:0/2465593615 learned_addr learned my addr 172.21.15.49:0/2465593615 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.165+0000 7fd205d9b700 1 -- 172.21.15.49:0/2465593615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd208077ee0 msgr2=0x7fd2081b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.165+0000 7fd205d9b700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd208077ee0 0x7fd2081b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.165+0000 7fd205d9b700 1 -- 172.21.15.49:0/2465593615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2080725e0 msgr2=0x7fd2081b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.165+0000 7fd205d9b700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2080725e0 0x7fd2081b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.165+0000 7fd205d9b700 1 -- 172.21.15.49:0/2465593615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc009e70 con 0x7fd208075510 2023-10-08T09:23:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.176+0000 7fd205d9b700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208075510 0x7fd2081b2a70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc000c00 tx=0x7fd1fc0126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.178+0000 7fd1f77fe700 1 -- 172.21.15.49:0/2465593615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00e040 con 0x7fd208075510 2023-10-08T09:23:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.178+0000 7fd1f77fe700 1 -- 172.21.15.49:0/2465593615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1fc012e40 con 0x7fd208075510 2023-10-08T09:23:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.178+0000 7fd1f77fe700 1 -- 172.21.15.49:0/2465593615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc01a630 con 0x7fd208075510 2023-10-08T09:23:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.182+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2081b5e50 con 0x7fd208075510 2023-10-08T09:23:34.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.182+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2081b6300 con 0x7fd208075510 2023-10-08T09:23:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.187+0000 7fd219118b80 1 Processor -- start 2023-10-08T09:23:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.194+0000 7fd1f77fe700 1 -- 172.21.15.49:0/2465593615 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd1fc01c400 con 0x7fd208075510 2023-10-08T09:23:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.195+0000 7fd1f77fe700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd1f00a1680 0x7fd1f00a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.195+0000 7fd1f77fe700 1 -- 172.21.15.49:0/2465593615 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd1fc0cd350 con 0x7fd208075510 2023-10-08T09:23:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.199+0000 7fd206d9d700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd1f00a1680 0x7fd1f00a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.208+0000 7fd219118b80 1 -- start start 2023-10-08T09:23:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.208+0000 7fd219118b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97bd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.208+0000 7fd219118b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.208+0000 7fd219118b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.208+0000 7fd219118b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d9725400 con 0x55d2d97bd7a0 2023-10-08T09:23:34.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.208+0000 7fd219118b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d9716bc0 con 0x55d2d97be2b0 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.208+0000 7fd219118b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d97144e0 con 0x55d2d97bce40 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.210+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.210+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97bdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52596/0 (socket says 172.21.15.49:52596) 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.210+0000 7fd205d9b700 1 -- 172.21.15.49:0/1440757135 learned_addr learned my addr 172.21.15.49:0/1440757135 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.212+0000 7fd205d9b700 1 -- 172.21.15.49:0/1440757135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 msgr2=0x55d2d97bd260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.212+0000 7fd205d9b700 1 --2- 172.21.15.49:0/1440757135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97bd260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.212+0000 7fd205d9b700 1 -- 172.21.15.49:0/1440757135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 msgr2=0x55d2d97be730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.212+0000 7fd205d9b700 1 --2- 172.21.15.49:0/1440757135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97be730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.212+0000 7fd205d9b700 1 -- 172.21.15.49:0/1440757135 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2d953ad80 con 0x55d2d97bd7a0 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.212+0000 7fd205d9b700 1 --2- 172.21.15.49:0/1440757135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97bdcd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005950 tx=0x7fd1fc005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31c5610863ebb4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:34.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.213+0000 7fd1f4ff9700 1 -- 172.21.15.49:0/1440757135 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc09a370 con 0x55d2d97bd7a0 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.213+0000 7fd1f4ff9700 1 -- 172.21.15.49:0/1440757135 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1fc09e4c0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.213+0000 7fd1f4ff9700 1 -- 172.21.15.49:0/1440757135 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0cd020 con 0x55d2d97bd7a0 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd219118b80 1 -- 172.21.15.49:0/1440757135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 msgr2=0x55d2d97bdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd219118b80 1 --2- 172.21.15.49:0/1440757135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97bdcd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005950 tx=0x7fd1fc005ab0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd206d9d700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd1f00a1680 0x7fd1f00a3b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009730 tx=0x7fd1f8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd219118b80 1 -- 172.21.15.49:0/1440757135 shutdown_connections 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd219118b80 1 --2- 172.21.15.49:0/1440757135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97bd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd219118b80 1 --2- 172.21.15.49:0/1440757135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97be730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd219118b80 1 --2- 172.21.15.49:0/1440757135 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97bdcd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.214+0000 7fd219118b80 1 -- 172.21.15.49:0/1440757135 >> 172.21.15.49:0/1440757135 conn(0x55d2d9731200 msgr2=0x55d2d97b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.215+0000 7fd219118b80 1 -- 172.21.15.49:0/1440757135 shutdown_connections 2023-10-08T09:23:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.216+0000 7fd219118b80 1 -- 172.21.15.49:0/1440757135 wait complete. 2023-10-08T09:23:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.217+0000 7fd219118b80 1 -- start start 2023-10-08T09:23:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.217+0000 7fd219118b80 1 Processor -- start 2023-10-08T09:23:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.218+0000 7fd219118b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.218+0000 7fd219118b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.218+0000 7fd219118b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.218+0000 7fd219118b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d970fdd0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.218+0000 7fd219118b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d96ac6b0 con 0x55d2d97be2b0 2023-10-08T09:23:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.218+0000 7fd219118b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d96ac350 con 0x55d2d97bce40 2023-10-08T09:23:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.218+0000 7fd219118b80 10 client.0 _ll_register_callbacks cb 0x55d2d95097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.220+0000 7fd20659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.220+0000 7fd20659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39636/0 (socket says 172.21.15.49:39636) 2023-10-08T09:23:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.220+0000 7fd20659c700 1 -- 172.21.15.49:0/1948263162 learned_addr learned my addr 172.21.15.49:0/1948263162 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.229+0000 7fd205d9b700 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.231+0000 7fd206d9d700 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.234+0000 7fd206d9d700 1 -- 172.21.15.49:0/1948263162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 msgr2=0x55d2d97c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.234+0000 7fd206d9d700 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.234+0000 7fd206d9d700 1 -- 172.21.15.49:0/1948263162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 msgr2=0x55d2d97c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.234+0000 7fd206d9d700 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.234+0000 7fd206d9d700 1 -- 172.21.15.49:0/1948263162 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2081b6300 con 0x55d2d97be2b0 2023-10-08T09:23:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.234+0000 7fd206d9d700 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97cdbd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8006fd0 tx=0x7fd1f8004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df8bde8d3177d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.239+0000 7fd1edffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:23:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.239+0000 7fd1edffb700 1 -- 172.21.15.49:0/1948263162 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f801a070 con 0x55d2d97be2b0 2023-10-08T09:23:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.239+0000 7fd1edffb700 1 -- 172.21.15.49:0/1948263162 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd1f8012b20 con 0x55d2d97be2b0 2023-10-08T09:23:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.239+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2d96d7e50 con 0x55d2d97be2b0 2023-10-08T09:23:34.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:33 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:23:34.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:33 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:23:34.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:33 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:23:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.252+0000 7fd1edffb700 1 -- 172.21.15.49:0/1948263162 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f80152e0 con 0x55d2d97be2b0 2023-10-08T09:23:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.252+0000 7fd219118b80 10 client.26264 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.252+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d2d9523c00 con 0x55d2d97be2b0 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.261+0000 7fd1edffb700 1 -- 172.21.15.49:0/1948263162 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2d96d7e50 con 0x55d2d97be2b0 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.261+0000 7fd1edffb700 1 -- 172.21.15.49:0/1948263162 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd1f8026070 con 0x55d2d97be2b0 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.262+0000 7fd219118b80 10 client.26264 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.262+0000 7fd219118b80 10 client.26264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.262+0000 7fd219118b80 20 client.26264 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.262+0000 7fd219118b80 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d97d2510 0x55d2d97d4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.262+0000 7fd219118b80 4 client.26264 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:23:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.262+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2d9708db0 con 0x55d2d97d2510 2023-10-08T09:23:34.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.265+0000 7fd20659c700 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d97d2510 0x55d2d97d4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.275+0000 7fd20659c700 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d97d2510 0x55d2d97d4930 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f6b15e20152809 server_cookie=552f0eefb4407f5a in_seq=0 out_seq=0 2023-10-08T09:23:34.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.275+0000 7fd1edffb700 10 client.26264 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.281+0000 7fd1edffb700 1 -- 172.21.15.49:0/1948263162 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d2d9708db0 con 0x55d2d97d2510 2023-10-08T09:23:34.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.281+0000 7fd1edffb700 10 client.26264 handle_command_reply: tid=0 2023-10-08T09:23:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.282+0000 7fd219118b80 1 client.26264 shutdown 2023-10-08T09:23:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 msgr2=0x55d2d97cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97cdbd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8006fd0 tx=0x7fd1f8004860 comp rx=0 tx=0).stop 2023-10-08T09:23:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 shutdown_connections 2023-10-08T09:23:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d97d2510 0x55d2d97d4930 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97bce40 0x55d2d97c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97be2b0 0x55d2d97cdbd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd1edffb700 0 client.26264 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 --2- 172.21.15.49:0/1948263162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 >> 172.21.15.49:0/1948263162 conn(0x55d2d9731200 msgr2=0x55d2d97b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 shutdown_connections 2023-10-08T09:23:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 1 -- 172.21.15.49:0/1948263162 wait complete. 2023-10-08T09:23:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.283+0000 7fd219118b80 20 client.26264 trim_cache size 0 max 16384 2023-10-08T09:23:34.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:33 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:23:34.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:33 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:23:34.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:33 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:23:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 Processor -- start 2023-10-08T09:23:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 -- start start 2023-10-08T09:23:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bce40 0x55d2d97d4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97b7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d95908c0 con 0x55d2d97bce40 2023-10-08T09:23:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d96d7e50 con 0x55d2d97bd7a0 2023-10-08T09:23:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.313+0000 7fd219118b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d97af670 con 0x55d2d97be2b0 2023-10-08T09:23:34.317 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 490.0 s, 527.2 tps, lat 174.038 ms stddev 191.773 2023-10-08T09:23:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.319+0000 7fd206d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.319+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.319+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42224/0 (socket says 172.21.15.49:42224) 2023-10-08T09:23:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.319+0000 7fd205d9b700 1 -- 172.21.15.49:0/3813356226 learned_addr learned my addr 172.21.15.49:0/3813356226 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.323+0000 7fd205d9b700 1 -- 172.21.15.49:0/3813356226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 msgr2=0x55d2d97b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.323+0000 7fd205d9b700 1 --2- 172.21.15.49:0/3813356226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97b7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.323+0000 7fd205d9b700 1 -- 172.21.15.49:0/3813356226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bce40 msgr2=0x55d2d97d4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.323+0000 7fd205d9b700 1 --2- 172.21.15.49:0/3813356226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bce40 0x55d2d97d4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.323+0000 7fd205d9b700 1 -- 172.21.15.49:0/3813356226 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2d95611f0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.323+0000 7fd205d9b700 1 --2- 172.21.15.49:0/3813356226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cdb90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc09a070 tx=0x7fd1fc0960b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2c10c83154d89a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.324+0000 7fd1f57fa700 1 -- 172.21.15.49:0/3813356226 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc01c6b0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.324+0000 7fd1f57fa700 1 -- 172.21.15.49:0/3813356226 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1fc098710 con 0x55d2d97bd7a0 2023-10-08T09:23:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.324+0000 7fd1f57fa700 1 -- 172.21.15.49:0/3813356226 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc09a5d0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.325+0000 7fd219118b80 1 -- 172.21.15.49:0/3813356226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bd7a0 msgr2=0x55d2d97cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.325+0000 7fd219118b80 1 --2- 172.21.15.49:0/3813356226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cdb90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc09a070 tx=0x7fd1fc0960b0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.332+0000 7fd219118b80 1 -- 172.21.15.49:0/3813356226 shutdown_connections 2023-10-08T09:23:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.332+0000 7fd219118b80 1 --2- 172.21.15.49:0/3813356226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.332+0000 7fd219118b80 1 --2- 172.21.15.49:0/3813356226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bce40 0x55d2d97d4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.332+0000 7fd219118b80 1 --2- 172.21.15.49:0/3813356226 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cdb90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.332+0000 7fd219118b80 1 -- 172.21.15.49:0/3813356226 >> 172.21.15.49:0/3813356226 conn(0x55d2d9731200 msgr2=0x55d2d97d9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.332+0000 7fd219118b80 1 -- 172.21.15.49:0/3813356226 shutdown_connections 2023-10-08T09:23:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.341+0000 7fd219118b80 1 -- 172.21.15.49:0/3813356226 wait complete. 2023-10-08T09:23:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.341+0000 7fd219118b80 1 -- start start 2023-10-08T09:23:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.341+0000 7fd219118b80 1 Processor -- start 2023-10-08T09:23:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd219118b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bce40 0x55d2d97cb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd219118b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd219118b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd219118b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d97cd380 con 0x55d2d97bd7a0 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd219118b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d97cd4f0 con 0x55d2d97bce40 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd219118b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2d97cd6b0 con 0x55d2d97be2b0 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd219118b80 10 client.0 _ll_register_callbacks cb 0x55d2d95097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd20659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bce40 0x55d2d97cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd205d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52614/0 (socket says 172.21.15.49:52614) 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd205d9b700 1 -- 172.21.15.49:0/1602717225 learned_addr learned my addr 172.21.15.49:0/1602717225 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.342+0000 7fd206d9d700 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd205d9b700 1 -- 172.21.15.49:0/1602717225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 msgr2=0x55d2d97cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd205d9b700 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd205d9b700 1 -- 172.21.15.49:0/1602717225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bce40 msgr2=0x55d2d97cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd205d9b700 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bce40 0x55d2d97cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd205d9b700 1 -- 172.21.15.49:0/1602717225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2d953ad80 con 0x55d2d97bd7a0 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd205d9b700 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cbe80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0a2140 tx=0x7fd1fc099eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b06d0b92cc85408 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd1edffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd1edffb700 1 -- 172.21.15.49:0/1602717225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc09a370 con 0x55d2d97bd7a0 2023-10-08T09:23:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd1edffb700 1 -- 172.21.15.49:0/1602717225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd1fc0a2670 con 0x55d2d97bd7a0 2023-10-08T09:23:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1fc09e4c0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd1edffb700 1 -- 172.21.15.49:0/1602717225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1fc09a5d0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd219118b80 10 client.16650 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.343+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d2d9539250 con 0x55d2d97bd7a0 2023-10-08T09:23:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.344+0000 7fd1edffb700 1 -- 172.21.15.49:0/1602717225 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1fc09e4c0 con 0x55d2d97bd7a0 2023-10-08T09:23:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd1edffb700 1 -- 172.21.15.49:0/1602717225 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd1fc0a0300 con 0x55d2d97bd7a0 2023-10-08T09:23:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd219118b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd219118b80 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd219118b80 20 client.16650 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd219118b80 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d9826d30 0x55d2d9829120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd219118b80 4 client.16650 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:23:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d2d9708ad0 con 0x55d2d9826d30 2023-10-08T09:23:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.346+0000 7fd20659c700 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d9826d30 0x55d2d9829120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.347+0000 7fd1edffb700 10 client.16650 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.347+0000 7fd20659c700 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d9826d30 0x55d2d9829120 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c23cba3538fe30e server_cookie=249caa6ec080ba0 in_seq=0 out_seq=0 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd1edffb700 1 -- 172.21.15.49:0/1602717225 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d2d9708ad0 con 0x55d2d9826d30 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd1edffb700 10 client.16650 handle_command_reply: tid=0 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 client.16650 shutdown 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 msgr2=0x55d2d97cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cbe80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0a2140 tx=0x7fd1fc099eb0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 shutdown_connections 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d2d9826d30 0x55d2d9829120 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d2d97be2b0 0x55d2d97cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d2d97bce40 0x55d2d97cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 --2- 172.21.15.49:0/1602717225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d2d97bd7a0 0x55d2d97cbe80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 >> 172.21.15.49:0/1602717225 conn(0x55d2d9731200 msgr2=0x55d2d97b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd1edffb700 0 client.16650 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 shutdown_connections 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 1 -- 172.21.15.49:0/1602717225 wait complete. 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.359+0000 7fd219118b80 20 client.16650 trim_cache size 0 max 16384 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:23:34.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:23:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd1f00a1680 msgr2=0x7fd1f00a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd1f00a1680 0x7fd1f00a3b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009730 tx=0x7fd1f8006d20 comp rx=0 tx=0).stop 2023-10-08T09:23:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208075510 msgr2=0x7fd2081b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208075510 0x7fd2081b2a70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc000c00 tx=0x7fd1fc0126d0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 shutdown_connections 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd1f00a1680 0x7fd1f00a3b40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd208077ee0 0x7fd2081b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd2080725e0 0x7fd2081b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 --2- 172.21.15.49:0/2465593615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd208075510 0x7fd2081b2a70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 >> 172.21.15.49:0/2465593615 conn(0x7fd20806d770 msgr2=0x7fd208075ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 shutdown_connections 2023-10-08T09:23:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:34.360+0000 7fd20cf2f700 1 -- 172.21.15.49:0/2465593615 wait complete. 2023-10-08T09:23:35.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:34 smithi184 ceph-mon[103675]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2023-10-08T09:23:35.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:34 smithi132 ceph-mon[103389]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2023-10-08T09:23:35.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:34 smithi049 ceph-mon[99485]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.6 MiB/s wr, 510 op/s 2023-10-08T09:23:37.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:37 smithi049 ceph-mon[99485]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.9 MiB/s wr, 629 op/s 2023-10-08T09:23:37.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:37 smithi184 ceph-mon[103675]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.9 MiB/s wr, 629 op/s 2023-10-08T09:23:37.801 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:37 smithi132 ceph-mon[103389]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.9 MiB/s wr, 629 op/s 2023-10-08T09:23:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 495.0 s, 542.2 tps, lat 194.787 ms stddev 227.842 2023-10-08T09:23:39.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:39 smithi049 ceph-mon[99485]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.9 MiB/s wr, 580 op/s 2023-10-08T09:23:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:39 smithi184 ceph-mon[103675]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.9 MiB/s wr, 580 op/s 2023-10-08T09:23:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:39 smithi132 ceph-mon[103389]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.9 MiB/s wr, 580 op/s 2023-10-08T09:23:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:41 smithi049 ceph-mon[99485]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 540 op/s 2023-10-08T09:23:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:41 smithi184 ceph-mon[103675]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 540 op/s 2023-10-08T09:23:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:41 smithi132 ceph-mon[103389]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.1 MiB/s wr, 540 op/s 2023-10-08T09:23:43.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:42 smithi049 ceph-mon[99485]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.2 MiB/s wr, 603 op/s 2023-10-08T09:23:43.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:42 smithi184 ceph-mon[103675]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.2 MiB/s wr, 603 op/s 2023-10-08T09:23:43.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:42 smithi132 ceph-mon[103389]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 7.2 MiB/s wr, 603 op/s 2023-10-08T09:23:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 500.0 s, 578.3 tps, lat 173.331 ms stddev 190.962 2023-10-08T09:23:45.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:45 smithi049 ceph-mon[99485]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 533 op/s 2023-10-08T09:23:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:45 smithi184 ceph-mon[103675]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 533 op/s 2023-10-08T09:23:45.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:45 smithi132 ceph-mon[103389]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.2 MiB/s wr, 533 op/s 2023-10-08T09:23:46.742 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:23:47.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.109+0000 7f62179c9700 1 -- 172.21.15.49:0/2574964619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 msgr2=0x7f621010c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.109+0000 7f62179c9700 1 --2- 172.21.15.49:0/2574964619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f621010c5f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f620800b0a0 tx=0x7f620800b3b0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.119+0000 7f62179c9700 1 -- 172.21.15.49:0/2574964619 shutdown_connections 2023-10-08T09:23:47.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.119+0000 7f62179c9700 1 --2- 172.21.15.49:0/2574964619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f621010cb30 0x7f621010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.120 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.119+0000 7f62179c9700 1 --2- 172.21.15.49:0/2574964619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62101077f0 0x7f6210109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.119+0000 7f62179c9700 1 --2- 172.21.15.49:0/2574964619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f621010c5f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.119+0000 7f62179c9700 1 -- 172.21.15.49:0/2574964619 >> 172.21.15.49:0/2574964619 conn(0x7f621006d790 msgr2=0x7f621006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.144 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.130+0000 7f62179c9700 1 -- 172.21.15.49:0/2574964619 shutdown_connections 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.143+0000 7f62179c9700 1 -- 172.21.15.49:0/2574964619 wait complete. 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.143+0000 7f62179c9700 1 Processor -- start 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.150+0000 7f62179c9700 1 -- start start 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.150+0000 7f62179c9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62101077f0 0x7f62101a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.151+0000 7f62179c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f62101a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.151+0000 7f62179c9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621010cb30 0x7f6210078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.151+0000 7f62179c9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210079050 con 0x7f621010a160 2023-10-08T09:23:47.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.151+0000 7f62179c9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100791c0 con 0x7f621010cb30 2023-10-08T09:23:47.152 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.151+0000 7f62179c9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6210079330 con 0x7f62101077f0 2023-10-08T09:23:47.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.155+0000 7f6214f64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f62101a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.155+0000 7f6214f64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f62101a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52626/0 (socket says 172.21.15.49:52626) 2023-10-08T09:23:47.155 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.155+0000 7f6214f64700 1 -- 172.21.15.49:0/508032281 learned_addr learned my addr 172.21.15.49:0/508032281 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.162+0000 7f6215765700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62101077f0 0x7f62101a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.163+0000 7f6215f66700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621010cb30 0x7f6210078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f6214f64700 1 -- 172.21.15.49:0/508032281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62101077f0 msgr2=0x7f62101a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f6214f64700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62101077f0 0x7f62101a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f6214f64700 1 -- 172.21.15.49:0/508032281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621010cb30 msgr2=0x7f6210078a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f6214f64700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621010cb30 0x7f6210078a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.166 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f6214f64700 1 -- 172.21.15.49:0/508032281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6208009d00 con 0x7f621010a160 2023-10-08T09:23:47.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f6214f64700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f62101a5b00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f620800b070 tx=0x7f6208012620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f62067fc700 1 -- 172.21.15.49:0/508032281 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800e040 con 0x7f621010a160 2023-10-08T09:23:47.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f62067fc700 1 -- 172.21.15.49:0/508032281 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6208012d80 con 0x7f621010a160 2023-10-08T09:23:47.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62100795b0 con 0x7f621010a160 2023-10-08T09:23:47.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f62067fc700 1 -- 172.21.15.49:0/508032281 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62080056d0 con 0x7f621010a160 2023-10-08T09:23:47.167 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.165+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6210079ab0 con 0x7f621010a160 2023-10-08T09:23:47.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.171+0000 7f6223bb2b80 1 Processor -- start 2023-10-08T09:23:47.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.172+0000 7f62067fc700 1 -- 172.21.15.49:0/508032281 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6208003ca0 con 0x7f621010a160 2023-10-08T09:23:47.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.172+0000 7f62067fc700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61fc0a1550 0x7f61fc0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.173+0000 7f62067fc700 1 -- 172.21.15.49:0/508032281 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f62080ce3f0 con 0x7f621010a160 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.182+0000 7f6215f66700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61fc0a1550 0x7f61fc0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.186+0000 7f6223bb2b80 1 -- start start 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.186+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.186+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23a9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.186+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.186+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f2310460 con 0x5576f23a89c0 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.186+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f2301c20 con 0x5576f23a8060 2023-10-08T09:23:47.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.186+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f22ff540 con 0x5576f23a9800 2023-10-08T09:23:47.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.189+0000 7f6214f64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.189+0000 7f6214f64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52632/0 (socket says 172.21.15.49:52632) 2023-10-08T09:23:47.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.189+0000 7f6214f64700 1 -- 172.21.15.49:0/3421605724 learned_addr learned my addr 172.21.15.49:0/3421605724 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.189+0000 7f6215765700 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.202+0000 7f6215f66700 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.202+0000 7f6215765700 1 -- 172.21.15.49:0/3421605724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 msgr2=0x5576f23a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.202+0000 7f6215765700 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.202+0000 7f6215765700 1 -- 172.21.15.49:0/3421605724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 msgr2=0x5576f23a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.202+0000 7f6215765700 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.203 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.202+0000 7f6215765700 1 -- 172.21.15.49:0/3421605724 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576f2125de0 con 0x5576f23a8060 2023-10-08T09:23:47.206 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.206+0000 7f6215765700 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23a8480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f620c005950 tx=0x7f620c00aeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c59f1f5f0d1009a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.207 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.207+0000 7f6215f66700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61fc0a1550 0x7f61fc0a3a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6200009bf0 tx=0x7f62000092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.215+0000 7f61fb7fe700 1 -- 172.21.15.49:0/3421605724 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00fcf0 con 0x5576f23a8060 2023-10-08T09:23:47.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.215+0000 7f61fb7fe700 1 -- 172.21.15.49:0/3421605724 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620c00fe50 con 0x5576f23a8060 2023-10-08T09:23:47.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.215+0000 7f61fb7fe700 1 -- 172.21.15.49:0/3421605724 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c018440 con 0x5576f23a8060 2023-10-08T09:23:47.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.216+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3421605724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 msgr2=0x5576f23a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.216 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.216+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23a8480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f620c005950 tx=0x7f620c00aeb0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.216+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3421605724 shutdown_connections 2023-10-08T09:23:47.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.216+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.216+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23a8480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.216+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3421605724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.217 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.216+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3421605724 >> 172.21.15.49:0/3421605724 conn(0x5576f231c260 msgr2=0x5576f23a2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.229+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3421605724 shutdown_connections 2023-10-08T09:23:47.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.236+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3421605724 wait complete. 2023-10-08T09:23:47.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.236+0000 7f6223bb2b80 1 -- start start 2023-10-08T09:23:47.237 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.236+0000 7f6223bb2b80 1 Processor -- start 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.242+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.242+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.242+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.242+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f22fae30 con 0x5576f23a89c0 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.242+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f2297710 con 0x5576f23a8060 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.242+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f22973b0 con 0x5576f23a9800 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.242+0000 7f6223bb2b80 10 client.0 _ll_register_callbacks cb 0x5576f20f4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:47.244 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6215f66700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23c33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52638/0 (socket says 172.21.15.49:52638) 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 -- 172.21.15.49:0/2615962965 learned_addr learned my addr 172.21.15.49:0/2615962965 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 -- 172.21.15.49:0/2615962965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 msgr2=0x5576f23ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 -- 172.21.15.49:0/2615962965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 msgr2=0x5576f23c2ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23c2ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 -- 172.21.15.49:0/2615962965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c00a830 con 0x5576f23a89c0 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6215f66700 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f6214f64700 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23c33e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6208005280 tx=0x7f6208003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db280509d429042e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.245 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.243+0000 7f61f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:23:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.246+0000 7f61f8ff9700 1 -- 172.21.15.49:0/2615962965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620809b500 con 0x5576f23a89c0 2023-10-08T09:23:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.246+0000 7f61f8ff9700 1 -- 172.21.15.49:0/2615962965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f620809f650 con 0x5576f23a89c0 2023-10-08T09:23:47.247 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.246+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5576f22c2eb0 con 0x5576f23a89c0 2023-10-08T09:23:47.248 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.246+0000 7f61f8ff9700 1 -- 172.21.15.49:0/2615962965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62080a3800 con 0x5576f23a89c0 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.246+0000 7f6223bb2b80 10 client.16662 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.246+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5576f210ec60 con 0x5576f23a89c0 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f61f8ff9700 1 -- 172.21.15.49:0/2615962965 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5576f22c2eb0 con 0x5576f23a89c0 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f61f8ff9700 1 -- 172.21.15.49:0/2615962965 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f620809a9f0 con 0x5576f23a89c0 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f6223bb2b80 10 client.16662 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f6223bb2b80 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f6223bb2b80 20 client.16662 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f23ce9c0 0x5576f23d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f6223bb2b80 4 client.16662 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:23:47.249 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576f22f3e10 con 0x5576f23ce9c0 2023-10-08T09:23:47.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.247+0000 7f6215765700 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f23ce9c0 0x5576f23d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.248+0000 7f61f8ff9700 10 client.16662 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.248+0000 7f6215765700 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f23ce9c0 0x5576f23d0de0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c7b7a460b80e38 server_cookie=bc0a2dd1906682c7 in_seq=0 out_seq=0 2023-10-08T09:23:47.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.249+0000 7f61f8ff9700 1 -- 172.21.15.49:0/2615962965 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5576f22f3e10 con 0x5576f23ce9c0 2023-10-08T09:23:47.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.249+0000 7f61f8ff9700 10 client.16662 handle_command_reply: tid=0 2023-10-08T09:23:47.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.249+0000 7f6223bb2b80 1 client.16662 shutdown 2023-10-08T09:23:47.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.249+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 msgr2=0x5576f23c33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.249+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23c33e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6208005280 tx=0x7f6208003c10 comp rx=0 tx=0).stop 2023-10-08T09:23:47.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 shutdown_connections 2023-10-08T09:23:47.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f23ce9c0 0x5576f23d0de0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a9800 0x5576f23ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a8060 0x5576f23c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.257 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/2615962965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a89c0 0x5576f23c33e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 >> 172.21.15.49:0/2615962965 conn(0x5576f231c260 msgr2=0x5576f23a24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f61f8ff9700 0 client.16662 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 shutdown_connections 2023-10-08T09:23:47.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.256+0000 7f6223bb2b80 1 -- 172.21.15.49:0/2615962965 wait complete. 2023-10-08T09:23:47.258 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.257+0000 7f6223bb2b80 20 client.16662 trim_cache size 0 max 16384 2023-10-08T09:23:47.292 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.291+0000 7f6223bb2b80 1 Processor -- start 2023-10-08T09:23:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6223bb2b80 1 -- start start 2023-10-08T09:23:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a8060 0x5576f23ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a89c0 0x5576f23acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a9800 0x5576f23ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f217b920 con 0x5576f23a9800 2023-10-08T09:23:47.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f22f3e10 con 0x5576f23a89c0 2023-10-08T09:23:47.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f22c2eb0 con 0x5576f23a8060 2023-10-08T09:23:47.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.296+0000 7f6215f66700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a9800 0x5576f23ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.297+0000 7f6215f66700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a9800 0x5576f23ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52644/0 (socket says 172.21.15.49:52644) 2023-10-08T09:23:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.297+0000 7f6215f66700 1 -- 172.21.15.49:0/1567367093 learned_addr learned my addr 172.21.15.49:0/1567367093 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.297+0000 7f6214f64700 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a89c0 0x5576f23acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.307+0000 7f6215765700 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a8060 0x5576f23ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.307+0000 7f6215f66700 1 -- 172.21.15.49:0/1567367093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a8060 msgr2=0x5576f23ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.307+0000 7f6215f66700 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a8060 0x5576f23ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.307+0000 7f6215f66700 1 -- 172.21.15.49:0/1567367093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a89c0 msgr2=0x5576f23acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.307+0000 7f6215f66700 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a89c0 0x5576f23acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.307+0000 7f6215f66700 1 -- 172.21.15.49:0/1567367093 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576f214c250 con 0x5576f23a9800 2023-10-08T09:23:47.323 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.319+0000 7f6215f66700 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a9800 0x5576f23ca2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6200011a40 tx=0x7f620000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3fe6353146af1e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.329+0000 7f61fbfff700 1 -- 172.21.15.49:0/1567367093 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200015aa0 con 0x5576f23a9800 2023-10-08T09:23:47.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.329+0000 7f61fbfff700 1 -- 172.21.15.49:0/1567367093 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620001b050 con 0x5576f23a9800 2023-10-08T09:23:47.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.329+0000 7f61fbfff700 1 -- 172.21.15.49:0/1567367093 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620001f370 con 0x5576f23a9800 2023-10-08T09:23:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.330+0000 7f6223bb2b80 1 -- 172.21.15.49:0/1567367093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a9800 msgr2=0x5576f23ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.330+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a9800 0x5576f23ca2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6200011a40 tx=0x7f620000e000 comp rx=0 tx=0).stop 2023-10-08T09:23:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.342+0000 7f6223bb2b80 1 -- 172.21.15.49:0/1567367093 shutdown_connections 2023-10-08T09:23:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.342+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a8060 0x5576f23ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.352 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.342+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a9800 0x5576f23ca2f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.342+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/1567367093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a89c0 0x5576f23acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.342+0000 7f6223bb2b80 1 -- 172.21.15.49:0/1567367093 >> 172.21.15.49:0/1567367093 conn(0x5576f231c260 msgr2=0x5576f23afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.353 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.351+0000 7f6223bb2b80 1 -- 172.21.15.49:0/1567367093 shutdown_connections 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.363+0000 7f6223bb2b80 1 -- 172.21.15.49:0/1567367093 wait complete. 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.363+0000 7f6223bb2b80 1 -- start start 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.363+0000 7f6223bb2b80 1 Processor -- start 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.366+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a8060 0x5576f23ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.366+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a89c0 0x5576f23c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.366+0000 7f6223bb2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a9800 0x5576f23c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.366+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f23c93d0 con 0x5576f23a8060 2023-10-08T09:23:47.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.366+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f23c9540 con 0x5576f23a9800 2023-10-08T09:23:47.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.366+0000 7f6223bb2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f2422800 con 0x5576f23a89c0 2023-10-08T09:23:47.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.366+0000 7f6223bb2b80 10 client.0 _ll_register_callbacks cb 0x7f62080971b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:47.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215f66700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a9800 0x5576f23c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215f66700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a9800 0x5576f23c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42266/0 (socket says 172.21.15.49:42266) 2023-10-08T09:23:47.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215f66700 1 -- 172.21.15.49:0/3754025587 learned_addr learned my addr 172.21.15.49:0/3754025587 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215765700 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a8060 0x5576f23ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215765700 1 -- 172.21.15.49:0/3754025587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a89c0 msgr2=0x5576f23c87b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:23:47.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215765700 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a89c0 0x5576f23c87b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215765700 1 -- 172.21.15.49:0/3754025587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a9800 msgr2=0x5576f23c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215765700 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a9800 0x5576f23c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.367+0000 7f6215765700 1 -- 172.21.15.49:0/3754025587 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6200011720 con 0x5576f23a8060 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.368+0000 7f6215765700 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a8060 0x5576f23ca1a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5576f23d1710 tx=0x7f620c00fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9992f14d15f69054 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.369+0000 7f61f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.369+0000 7f61f8ff9700 1 -- 172.21.15.49:0/3754025587 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0260b0 con 0x5576f23a8060 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.369+0000 7f61f8ff9700 1 -- 172.21.15.49:0/3754025587 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f620c026210 con 0x5576f23a8060 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.369+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f620c00fe50 con 0x5576f23a8060 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f61f8ff9700 1 -- 172.21.15.49:0/3754025587 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620c0263f0 con 0x5576f23a8060 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 10 client.16668 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:47.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5576f21242b0 con 0x5576f23a8060 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f61f8ff9700 1 -- 172.21.15.49:0/3754025587 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f620c00fe50 con 0x5576f23a8060 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f61f8ff9700 1 -- 172.21.15.49:0/3754025587 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f620c00abd0 con 0x5576f23a8060 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 10 client.16668 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 20 client.16668 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f2424150 0x5576f2426570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 4 client.16668 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:23:47.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.370+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5576f22f3b30 con 0x5576f2424150 2023-10-08T09:23:47.388 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.379+0000 7f6214f64700 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f2424150 0x5576f2426570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.395+0000 7f6214f64700 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f2424150 0x5576f2426570 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba3dca5b749d9b6 server_cookie=8d7c193ce6cbdf4e in_seq=0 out_seq=0 2023-10-08T09:23:47.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.396+0000 7f61f8ff9700 10 client.16668 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.400+0000 7f61f8ff9700 1 -- 172.21.15.49:0/3754025587 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5576f22f3b30 con 0x5576f2424150 2023-10-08T09:23:47.411 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.400+0000 7f61f8ff9700 10 client.16668 handle_command_reply: tid=0 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.400+0000 7f6223bb2b80 1 client.16668 shutdown 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.401+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a8060 msgr2=0x5576f23ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.401+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a8060 0x5576f23ca1a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5576f23d1710 tx=0x7f620c00fba0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.410+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 shutdown_connections 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.410+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5576f2424150 0x5576f2426570 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.410+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5576f23a9800 0x5576f23c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.410+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5576f23a8060 0x5576f23ca1a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.410+0000 7f6223bb2b80 1 --2- 172.21.15.49:0/3754025587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5576f23a89c0 0x5576f23c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.410+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 >> 172.21.15.49:0/3754025587 conn(0x5576f231c260 msgr2=0x5576f23ac800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.410+0000 7f61f8ff9700 0 client.16668 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.419 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.418+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 shutdown_connections 2023-10-08T09:23:47.433 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.428+0000 7f6223bb2b80 1 -- 172.21.15.49:0/3754025587 wait complete. 2023-10-08T09:23:47.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.435+0000 7f6223bb2b80 20 client.16668 trim_cache size 0 max 16384 2023-10-08T09:23:47.438 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:23:47.438 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.437+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61fc0a1550 msgr2=0x7f61fc0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.439 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.437+0000 7f62179c9700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61fc0a1550 0x7f61fc0a3a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6200009bf0 tx=0x7f62000092b0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.437+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 msgr2=0x7f62101a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.440 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.437+0000 7f62179c9700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f62101a5b00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f620800b070 tx=0x7f6208012620 comp rx=0 tx=0).stop 2023-10-08T09:23:47.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.440+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 shutdown_connections 2023-10-08T09:23:47.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.441+0000 7f62179c9700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f61fc0a1550 0x7f61fc0a3a10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.441+0000 7f62179c9700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62101077f0 0x7f62101a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.441+0000 7f62179c9700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f621010cb30 0x7f6210078a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.441+0000 7f62179c9700 1 --2- 172.21.15.49:0/508032281 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f621010a160 0x7f62101a5b00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.441+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 >> 172.21.15.49:0/508032281 conn(0x7f621006d790 msgr2=0x7f6210108dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.442+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 shutdown_connections 2023-10-08T09:23:47.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:23:47.447+0000 7f62179c9700 1 -- 172.21.15.49:0/508032281 wait complete. 2023-10-08T09:23:47.482 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:23:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 -- 172.21.15.49:0/802823766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 msgr2=0x7feee8109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 --2- 172.21.15.49:0/802823766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 0x7feee8109c20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7feed0009990 tx=0x7feed0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 -- 172.21.15.49:0/802823766 shutdown_connections 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 --2- 172.21.15.49:0/802823766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 0x7feee8109c20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 --2- 172.21.15.49:0/802823766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee810cb30 0x7feee810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 --2- 172.21.15.49:0/802823766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feee810a160 0x7feee810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 -- 172.21.15.49:0/802823766 >> 172.21.15.49:0/802823766 conn(0x7feee806d770 msgr2=0x7feee806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.572+0000 7feeedefb700 1 -- 172.21.15.49:0/802823766 shutdown_connections 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.573+0000 7feeedefb700 1 -- 172.21.15.49:0/802823766 wait complete. 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.573+0000 7feeedefb700 1 Processor -- start 2023-10-08T09:23:47.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feeedefb700 1 -- start start 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feeedefb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 0x7feee81a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feeedefb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee810a160 0x7feee81a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feeedefb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feee810cb30 0x7feee81acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feeedefb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee81ad0b0 con 0x7feee810cb30 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feeedefb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee81ad1f0 con 0x7feee810a160 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feeedefb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee81ad380 con 0x7feee81077f0 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 0x7feee81a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feee810cb30 0x7feee81acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 0x7feee81a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39696/0 (socket says 172.21.15.49:39696) 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee77fe700 1 -- 172.21.15.49:0/1280401372 learned_addr learned my addr 172.21.15.49:0/1280401372 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee6ffd700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee810a160 0x7feee81a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee7fff700 1 -- 172.21.15.49:0/1280401372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 msgr2=0x7feee81a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee7fff700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 0x7feee81a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee7fff700 1 -- 172.21.15.49:0/1280401372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee810a160 msgr2=0x7feee81a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.574+0000 7feee7fff700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee810a160 0x7feee81a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.575+0000 7feee7fff700 1 -- 172.21.15.49:0/1280401372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00095f0 con 0x7feee810cb30 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.575+0000 7feee7fff700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feee810cb30 0x7feee81acae0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7feee000be20 tx=0x7feee0009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.575+0000 7feee4ff9700 1 -- 172.21.15.49:0/1280401372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee000cb90 con 0x7feee810cb30 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.575+0000 7feee4ff9700 1 -- 172.21.15.49:0/1280401372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feee000ccf0 con 0x7feee810cb30 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.575+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee81ad660 con 0x7feee810cb30 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.575+0000 7feee4ff9700 1 -- 172.21.15.49:0/1280401372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0012670 con 0x7feee810cb30 2023-10-08T09:23:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.575+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee81adb70 con 0x7feee810cb30 2023-10-08T09:23:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.577+0000 7feee4ff9700 1 -- 172.21.15.49:0/1280401372 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feee0014440 con 0x7feee810cb30 2023-10-08T09:23:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.578+0000 7feee4ff9700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feed40a1550 0x7feed40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.578+0000 7feee4ff9700 1 -- 172.21.15.49:0/1280401372 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feee0006e40 con 0x7feee810cb30 2023-10-08T09:23:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.578+0000 7feee77fe700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feed40a1550 0x7feed40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.579+0000 7feee77fe700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feed40a1550 0x7feed40a3a10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feed000b7b0 tx=0x7feed0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 Processor -- start 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 -- start start 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e29260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e29cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e2a2b0 0x55be22e2a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22d91400 con 0x55be22e2a2b0 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22d82bc0 con 0x55be22e297a0 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22d804e0 con 0x55be22e28e40 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feee77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e29260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feee7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e2a2b0 0x55be22e2a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.582 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feee77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e29260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39704/0 (socket says 172.21.15.49:39704) 2023-10-08T09:23:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feee7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e2a2b0 0x55be22e2a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52674/0 (socket says 172.21.15.49:52674) 2023-10-08T09:23:47.583 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.580+0000 7feee77fe700 1 -- 172.21.15.49:0/1576899901 learned_addr learned my addr 172.21.15.49:0/1576899901 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feee77fe700 1 -- 172.21.15.49:0/1576899901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 msgr2=0x55be22e29cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feee77fe700 1 --2- 172.21.15.49:0/1576899901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e29cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feee77fe700 1 -- 172.21.15.49:0/1576899901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e2a2b0 msgr2=0x55be22e2a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feee77fe700 1 --2- 172.21.15.49:0/1576899901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e2a2b0 0x55be22e2a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feee77fe700 1 -- 172.21.15.49:0/1576899901 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be22ba6d80 con 0x55be22e28e40 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feee77fe700 1 --2- 172.21.15.49:0/1576899901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e29260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feed0005cd0 tx=0x7feed0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=841740139484d32c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feeddffb700 1 -- 172.21.15.49:0/1576899901 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed002b070 con 0x55be22e28e40 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feeddffb700 1 -- 172.21.15.49:0/1576899901 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feed001e040 con 0x55be22e28e40 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.581+0000 7feeddffb700 1 -- 172.21.15.49:0/1576899901 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0025ca0 con 0x55be22e28e40 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 -- 172.21.15.49:0/1576899901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 msgr2=0x55be22e29260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/1576899901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e29260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feed0005cd0 tx=0x7feed0020000 comp rx=0 tx=0).stop 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 -- 172.21.15.49:0/1576899901 shutdown_connections 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/1576899901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e29260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/1576899901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e2a2b0 0x55be22e2a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/1576899901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e29cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 -- 172.21.15.49:0/1576899901 >> 172.21.15.49:0/1576899901 conn(0x55be22d9d200 msgr2=0x55be22e238a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 -- 172.21.15.49:0/1576899901 shutdown_connections 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 -- 172.21.15.49:0/1576899901 wait complete. 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.582+0000 7feefa0e4b80 1 -- start start 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 1 Processor -- start 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e297a0 0x55be22e32d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e39d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22d7bdd0 con 0x55be22e297a0 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22d186b0 con 0x55be22e2a2b0 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22d18350 con 0x55be22e28e40 2023-10-08T09:23:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feefa0e4b80 10 client.0 _ll_register_callbacks cb 0x55be22b757a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e327d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39710/0 (socket says 172.21.15.49:39710) 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 -- 172.21.15.49:0/399317048 learned_addr learned my addr 172.21.15.49:0/399317048 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee6ffd700 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e297a0 0x55be22e32d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 -- 172.21.15.49:0/399317048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 msgr2=0x55be22e39d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e39d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 -- 172.21.15.49:0/399317048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e297a0 msgr2=0x55be22e32d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e297a0 0x55be22e32d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.583+0000 7feee77fe700 1 -- 172.21.15.49:0/399317048 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee810da70 con 0x55be22e28e40 2023-10-08T09:23:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.584+0000 7feee77fe700 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e327d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2e0 tx=0x7feed0025eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6033d0d752da0e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.584+0000 7feec6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:23:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.585+0000 7feec6ffd700 1 -- 172.21.15.49:0/399317048 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed003c070 con 0x55be22e28e40 2023-10-08T09:23:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.585+0000 7feec6ffd700 1 -- 172.21.15.49:0/399317048 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feed000fc00 con 0x55be22e28e40 2023-10-08T09:23:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.585+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be22d43e50 con 0x55be22e28e40 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.585+0000 7feec6ffd700 1 -- 172.21.15.49:0/399317048 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7feed002b050 con 0x55be22e28e40 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.585+0000 7feefa0e4b80 10 client.26326 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.585+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55be22b8fc00 con 0x55be22e28e40 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.585+0000 7feec6ffd700 1 -- 172.21.15.49:0/399317048 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be22d43e50 con 0x55be22e28e40 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.586+0000 7feec6ffd700 1 -- 172.21.15.49:0/399317048 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feed003f030 con 0x55be22e28e40 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.586+0000 7feefa0e4b80 10 client.26326 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.586+0000 7feefa0e4b80 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.586+0000 7feefa0e4b80 20 client.26326 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.586+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e3e1a0 0x55be22e40590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.587+0000 7feefa0e4b80 4 client.26326 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.587+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be22d74db0 con 0x55be22e3e1a0 2023-10-08T09:23:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.587+0000 7feee6ffd700 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e3e1a0 0x55be22e40590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.587+0000 7feec6ffd700 10 client.26326 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.587+0000 7feee6ffd700 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e3e1a0 0x55be22e40590 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65722d2ec0218b3e server_cookie=372c5716a362ad8b in_seq=0 out_seq=0 2023-10-08T09:23:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.588+0000 7feec6ffd700 1 -- 172.21.15.49:0/399317048 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be22d74db0 con 0x55be22e3e1a0 2023-10-08T09:23:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.588+0000 7feec6ffd700 10 client.26326 handle_command_reply: tid=0 2023-10-08T09:23:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.588+0000 7feefa0e4b80 1 client.26326 shutdown 2023-10-08T09:23:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.588+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 msgr2=0x55be22e327d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.588+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e327d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2e0 tx=0x7feed0025eb0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 shutdown_connections 2023-10-08T09:23:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e3e1a0 0x55be22e40590 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e28e40 0x55be22e327d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e39d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/399317048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e297a0 0x55be22e32d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feec6ffd700 0 client.26326 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 >> 172.21.15.49:0/399317048 conn(0x55be22d9d200 msgr2=0x55be22e23010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 shutdown_connections 2023-10-08T09:23:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 1 -- 172.21.15.49:0/399317048 wait complete. 2023-10-08T09:23:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.592+0000 7feefa0e4b80 20 client.26326 trim_cache size 0 max 16384 2023-10-08T09:23:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 Processor -- start 2023-10-08T09:23:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 -- start start 2023-10-08T09:23:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e3a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e297a0 0x55be22e39cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e2a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22bfc8c0 con 0x55be22e28e40 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22e310b0 con 0x55be22e2a2b0 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22e295e0 con 0x55be22e297a0 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feee77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e3a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feee77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e3a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52684/0 (socket says 172.21.15.49:52684) 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feee77fe700 1 -- 172.21.15.49:0/383063017 learned_addr learned my addr 172.21.15.49:0/383063017 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feee6ffd700 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e297a0 0x55be22e39cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.612+0000 7feee7fff700 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e2a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.613+0000 7feee77fe700 1 -- 172.21.15.49:0/383063017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e297a0 msgr2=0x55be22e39cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.613+0000 7feee77fe700 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e297a0 0x55be22e39cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.613+0000 7feee77fe700 1 -- 172.21.15.49:0/383063017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 msgr2=0x55be22e2a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.613+0000 7feee77fe700 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e2a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.613+0000 7feee77fe700 1 -- 172.21.15.49:0/383063017 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be22bcd1f0 con 0x55be22e28e40 2023-10-08T09:23:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.613+0000 7feee7fff700 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e2a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:23:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.613+0000 7feee77fe700 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e3a3b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feed0022480 tx=0x7feed0005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ddd7584b67986 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.618+0000 7feede7fc700 1 -- 172.21.15.49:0/383063017 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed003c030 con 0x55be22e28e40 2023-10-08T09:23:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.618+0000 7feede7fc700 1 -- 172.21.15.49:0/383063017 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feed002c040 con 0x55be22e28e40 2023-10-08T09:23:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.618+0000 7feede7fc700 1 -- 172.21.15.49:0/383063017 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed002b070 con 0x55be22e28e40 2023-10-08T09:23:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.618+0000 7feefa0e4b80 1 -- 172.21.15.49:0/383063017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 msgr2=0x55be22e3a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.618+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e3a3b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7feed0022480 tx=0x7feed0005cd0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.619+0000 7feefa0e4b80 1 -- 172.21.15.49:0/383063017 shutdown_connections 2023-10-08T09:23:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.619+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e2a2b0 0x55be22e2a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.619+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e3a3b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.619+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/383063017 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e297a0 0x55be22e39cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.619+0000 7feefa0e4b80 1 -- 172.21.15.49:0/383063017 >> 172.21.15.49:0/383063017 conn(0x55be22d9d200 msgr2=0x55be22e44ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.622+0000 7feefa0e4b80 1 -- 172.21.15.49:0/383063017 shutdown_connections 2023-10-08T09:23:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.622+0000 7feefa0e4b80 1 -- 172.21.15.49:0/383063017 wait complete. 2023-10-08T09:23:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.622+0000 7feefa0e4b80 1 -- start start 2023-10-08T09:23:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.622+0000 7feefa0e4b80 1 Processor -- start 2023-10-08T09:23:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.622+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.622+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e37b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feefa0e4b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e2a2b0 0x55be22e38a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22e38fc0 con 0x55be22e28e40 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22e39130 con 0x55be22e297a0 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feefa0e4b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be22e392f0 con 0x55be22e2a2b0 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feefa0e4b80 10 client.0 _ll_register_callbacks cb 0x7feee0095b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e37b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e37b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42304/0 (socket says 172.21.15.49:42304) 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee6ffd700 1 -- 172.21.15.49:0/3442974379 learned_addr learned my addr 172.21.15.49:0/3442974379 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee7fff700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e2a2b0 0x55be22e38a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee77fe700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee7fff700 1 -- 172.21.15.49:0/3442974379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 msgr2=0x55be22e37b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee7fff700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e37b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee7fff700 1 -- 172.21.15.49:0/3442974379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 msgr2=0x55be22e375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee7fff700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e375d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee7fff700 1 -- 172.21.15.49:0/3442974379 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be22b8fc00 con 0x55be22e2a2b0 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.623+0000 7feee77fe700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feee7fff700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e2a2b0 0x55be22e38a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feee000bfd0 tx=0x7feee0099f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9004f7c42c3b49ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feec6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feec6ffd700 1 -- 172.21.15.49:0/3442974379 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00a1b70 con 0x55be22e2a2b0 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feec6ffd700 1 -- 172.21.15.49:0/3442974379 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7feee0099a40 con 0x55be22e2a2b0 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feed001e040 con 0x55be22e2a2b0 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feec6ffd700 1 -- 172.21.15.49:0/3442974379 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7feee0099c00 con 0x55be22e2a2b0 2023-10-08T09:23:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 10 client.26332 fetch_fsmap learned FSMap version 40 2023-10-08T09:23:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55be22ba5250 con 0x55be22e2a2b0 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feec6ffd700 1 -- 172.21.15.49:0/3442974379 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feee007c020 con 0x55be22e2a2b0 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feec6ffd700 1 -- 172.21.15.49:0/3442974379 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7feee00a16e0 con 0x55be22e2a2b0 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 10 client.26332 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 10 client.26332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 20 client.26332 populate_metadata read hostname 'smithi049' 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e8c8e0 0x55be22e8ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 4 client.26332 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.624+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55be22d74ad0 con 0x55be22e8c8e0 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.625+0000 7feee77fe700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e8c8e0 0x55be22e8ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.625+0000 7feec6ffd700 10 client.26332 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.625+0000 7feee77fe700 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e8c8e0 0x55be22e8ed00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b5ad9cf65e9bec server_cookie=7af2608c91799478 in_seq=0 out_seq=0 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feec6ffd700 1 -- 172.21.15.49:0/3442974379 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55be22d74ad0 con 0x55be22e8c8e0 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feec6ffd700 10 client.26332 handle_command_reply: tid=0 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 client.26332 shutdown 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e2a2b0 msgr2=0x55be22e38a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e2a2b0 0x55be22e38a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feee000bfd0 tx=0x7feee0099f60 comp rx=0 tx=0).stop 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 shutdown_connections 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55be22e8c8e0 0x55be22e8ed00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55be22e2a2b0 0x55be22e38a80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be22e28e40 0x55be22e375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feec6ffd700 0 client.26332 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 --2- 172.21.15.49:0/3442974379 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55be22e297a0 0x55be22e37b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 >> 172.21.15.49:0/3442974379 conn(0x55be22d9d200 msgr2=0x55be22e3a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 shutdown_connections 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 1 -- 172.21.15.49:0/3442974379 wait complete. 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.634+0000 7feefa0e4b80 20 client.26332 trim_cache size 0 max 16384 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:23:47.637 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:23:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:23:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feed40a1550 msgr2=0x7feed40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feed40a1550 0x7feed40a3a10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feed000b7b0 tx=0x7feed0005fb0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feee810cb30 msgr2=0x7feee81acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feee810cb30 0x7feee81acae0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7feee000be20 tx=0x7feee0009ed0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 shutdown_connections 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feed40a1550 0x7feed40a3a10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feee81077f0 0x7feee81a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feee810cb30 0x7feee81acae0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 --2- 172.21.15.49:0/1280401372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feee810a160 0x7feee81a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 >> 172.21.15.49:0/1280401372 conn(0x7feee806d770 msgr2=0x7feee810b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 shutdown_connections 2023-10-08T09:23:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:23:47.635+0000 7feeedefb700 1 -- 172.21.15.49:0/1280401372 wait complete. 2023-10-08T09:23:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:47 smithi184 ceph-mon[103675]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.2 MiB/s wr, 614 op/s 2023-10-08T09:23:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:47 smithi132 ceph-mon[103389]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.2 MiB/s wr, 614 op/s 2023-10-08T09:23:47.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:47 smithi049 ceph-mon[99485]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 7.2 MiB/s wr, 614 op/s 2023-10-08T09:23:48.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:48 smithi184 ceph-mon[103675]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 537 op/s 2023-10-08T09:23:48.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:48 smithi132 ceph-mon[103389]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 537 op/s 2023-10-08T09:23:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:48 smithi049 ceph-mon[99485]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 537 op/s 2023-10-08T09:23:49.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 505.0 s, 564.5 tps, lat 182.399 ms stddev 230.418 2023-10-08T09:23:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:51 smithi184 ceph-mon[103675]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 502 op/s 2023-10-08T09:23:51.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:51 smithi132 ceph-mon[103389]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 502 op/s 2023-10-08T09:23:51.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:51 smithi049 ceph-mon[99485]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.7 MiB/s wr, 502 op/s 2023-10-08T09:23:53.139 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:52 smithi049 ceph-mon[99485]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 560 op/s 2023-10-08T09:23:53.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:52 smithi184 ceph-mon[103675]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 560 op/s 2023-10-08T09:23:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:52 smithi132 ceph-mon[103389]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 560 op/s 2023-10-08T09:23:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 510.0 s, 521.3 tps, lat 187.812 ms stddev 228.971 2023-10-08T09:23:55.752 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:55 smithi184 ceph-mon[103675]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 533 op/s 2023-10-08T09:23:55.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:55 smithi132 ceph-mon[103389]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 533 op/s 2023-10-08T09:23:55.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:55 smithi049 ceph-mon[99485]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 533 op/s 2023-10-08T09:23:57.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:57 smithi184 ceph-mon[103675]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 576 op/s 2023-10-08T09:23:57.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:57 smithi132 ceph-mon[103389]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 576 op/s 2023-10-08T09:23:57.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:57 smithi049 ceph-mon[99485]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 576 op/s 2023-10-08T09:23:58.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:23:58 smithi132 ceph-mon[103389]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 522 op/s 2023-10-08T09:23:58.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:23:58 smithi049 ceph-mon[99485]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 522 op/s 2023-10-08T09:23:59.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:23:58 smithi184 ceph-mon[103675]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 522 op/s 2023-10-08T09:23:59.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 515.0 s, 722.1 tps, lat 137.709 ms stddev 154.065 2023-10-08T09:24:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 -- 172.21.15.49:0/560678490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec108920 msgr2=0x7f5cec10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 --2- 172.21.15.49:0/560678490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec108920 0x7f5cec10f1d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400b0d0 tx=0x7f5ce400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 -- 172.21.15.49:0/560678490 shutdown_connections 2023-10-08T09:24:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 --2- 172.21.15.49:0/560678490 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec108920 0x7f5cec10f1d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 --2- 172.21.15.49:0/560678490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cec106d90 0x7f5cec1071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 --2- 172.21.15.49:0/560678490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 0x7f5cec1083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 -- 172.21.15.49:0/560678490 >> 172.21.15.49:0/560678490 conn(0x7f5cec075d20 msgr2=0x7f5cec078160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 -- 172.21.15.49:0/560678490 shutdown_connections 2023-10-08T09:24:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.922+0000 7f5cf235b700 1 -- 172.21.15.49:0/560678490 wait complete. 2023-10-08T09:24:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 Processor -- start 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 -- start start 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec106d90 0x7f5cec19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 0x7f5cec19d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cec108920 0x7f5cec1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a48e0 con 0x7f5cec106d90 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a4a20 con 0x7f5cec108920 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.923+0000 7f5cf235b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1a4bb0 con 0x7f5cec107f80 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec106d90 0x7f5cec19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 0x7f5cec19d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 0x7f5cec19d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39740/0 (socket says 172.21.15.49:39740) 2023-10-08T09:24:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec106d90 0x7f5cec19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52704/0 (socket says 172.21.15.49:52704) 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3378129376 learned_addr learned my addr 172.21.15.49:0/3378129376 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3378129376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cec108920 msgr2=0x7f5cec1a4310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cec108920 0x7f5cec1a4310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec106d90 msgr2=0x7f5cec19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec106d90 0x7f5cec19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3378129376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce4009d00 con 0x7f5cec107f80 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 0x7f5cec19d2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000e9a0 tx=0x7f5ce000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ce97fa700 1 -- 172.21.15.49:0/3378129376 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce000cbd0 con 0x7f5cec107f80 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ce97fa700 1 -- 172.21.15.49:0/3378129376 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce000cd30 con 0x7f5cec107f80 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec1a4e90 con 0x7f5cec107f80 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.924+0000 7f5ce97fa700 1 -- 172.21.15.49:0/3378129376 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0018710 con 0x7f5cec107f80 2023-10-08T09:24:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.925+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec1a53a0 con 0x7f5cec107f80 2023-10-08T09:24:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.927+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec106d90 0x7f5cec19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:24:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.927+0000 7f5ce97fa700 1 -- 172.21.15.49:0/3378129376 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5ce0010430 con 0x7f5cec107f80 2023-10-08T09:24:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.928+0000 7f5ce97fa700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5cd40a1480 0x7f5cd40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.928+0000 7f5ce97fa700 1 -- 172.21.15.49:0/3378129376 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5ce0014070 con 0x7f5cec107f80 2023-10-08T09:24:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.928+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5cd40a1480 0x7f5cd40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 Processor -- start 2023-10-08T09:24:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 -- start start 2023-10-08T09:24:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5cd40a1480 0x7f5cd40a3940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5cec107de0 tx=0x7f5cdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d1e40 0x559c9d8d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d83a400 con 0x559c9d8d1e40 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d82bbc0 con 0x559c9d8d27a0 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.929+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d8294e0 con 0x559c9d8d32b0 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39748/0 (socket says 172.21.15.49:39748) 2023-10-08T09:24:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 -- 172.21.15.49:0/3650255121 learned_addr learned my addr 172.21.15.49:0/3650255121 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 -- 172.21.15.49:0/3650255121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 msgr2=0x559c9d8d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 --2- 172.21.15.49:0/3650255121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 -- 172.21.15.49:0/3650255121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d1e40 msgr2=0x559c9d8d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 --2- 172.21.15.49:0/3650255121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d1e40 0x559c9d8d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 -- 172.21.15.49:0/3650255121 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c9d64fd80 con 0x559c9d8d32b0 2023-10-08T09:24:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.930+0000 7f5cf08f8700 1 --2- 172.21.15.49:0/3650255121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3730 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4007b60 tx=0x7f5ce4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c6f1be89367b1c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.931+0000 7f5cd27fc700 1 -- 172.21.15.49:0/3650255121 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400e040 con 0x559c9d8d32b0 2023-10-08T09:24:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.931+0000 7f5cd27fc700 1 -- 172.21.15.49:0/3650255121 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce400baa0 con 0x559c9d8d32b0 2023-10-08T09:24:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.931+0000 7f5cd27fc700 1 -- 172.21.15.49:0/3650255121 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4005330 con 0x559c9d8d32b0 2023-10-08T09:24:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3650255121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d32b0 msgr2=0x559c9d8d3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3650255121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3730 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4007b60 tx=0x7f5ce4009b80 comp rx=0 tx=0).stop 2023-10-08T09:24:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3650255121 shutdown_connections 2023-10-08T09:24:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3650255121 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3730 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3650255121 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d1e40 0x559c9d8d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3650255121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3650255121 >> 172.21.15.49:0/3650255121 conn(0x559c9d846200 msgr2=0x559c9d8cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3650255121 shutdown_connections 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3650255121 wait complete. 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 -- start start 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.932+0000 7f5cfe544b80 1 Processor -- start 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8dbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d32b0 0x559c9d8e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d824dd0 con 0x559c9d8d32b0 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d7c16b0 con 0x559c9d8d27a0 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d7c1350 con 0x559c9d8d1e40 2023-10-08T09:24:00.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cfe544b80 10 client.0 _ll_register_callbacks cb 0x559c9d61e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8dbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8dbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42330/0 (socket says 172.21.15.49:42330) 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3763319970 learned_addr learned my addr 172.21.15.49:0/3763319970 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cebfff700 1 -- 172.21.15.49:0/3763319970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 msgr2=0x559c9d8dbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8dbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cebfff700 1 -- 172.21.15.49:0/3763319970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d32b0 msgr2=0x559c9d8e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d32b0 0x559c9d8e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.933+0000 7f5cebfff700 1 -- 172.21.15.49:0/3763319970 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec10fb20 con 0x559c9d8d1e40 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8db9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc010ce0 tx=0x7f5cdc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60644c3f300747d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:00.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5ccb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3763319970 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc012b20 con 0x559c9d8d1e40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3763319970 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5cdc012c80 con 0x559c9d8d1e40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c9d7ece50 con 0x559c9d8d1e40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3763319970 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cdc012e40 con 0x559c9d8d1e40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cfe544b80 10 client.26344 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559c9d638c00 con 0x559c9d8d1e40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3763319970 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c9d7ece50 con 0x559c9d8d1e40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3763319970 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5cdc01b050 con 0x559c9d8d1e40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cfe544b80 10 client.26344 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cfe544b80 10 client.26344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cfe544b80 20 client.26344 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.934+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d8e7770 0x559c9d8e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.935+0000 7f5cfe544b80 4 client.26344 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:24:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.935+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c9d81ddb0 con 0x559c9d8e7770 2023-10-08T09:24:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.935+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d8e7770 0x559c9d8e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.935+0000 7f5ccb7fe700 10 client.26344 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.935+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d8e7770 0x559c9d8e9b90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af25b6a1ed5e71ad server_cookie=9373222a89c60293 in_seq=0 out_seq=0 2023-10-08T09:24:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3763319970 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c9d81ddb0 con 0x559c9d8e7770 2023-10-08T09:24:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5ccb7fe700 10 client.26344 handle_command_reply: tid=0 2023-10-08T09:24:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 client.26344 shutdown 2023-10-08T09:24:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 msgr2=0x559c9d8db9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8db9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc010ce0 tx=0x7f5cdc004750 comp rx=0 tx=0).stop 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 shutdown_connections 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d8e7770 0x559c9d8e9b90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8db9f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d32b0 0x559c9d8e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5ccb7fe700 0 client.26344 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3763319970 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d27a0 0x559c9d8dbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 >> 172.21.15.49:0/3763319970 conn(0x559c9d846200 msgr2=0x559c9d8cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 shutdown_connections 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.936+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3763319970 wait complete. 2023-10-08T09:24:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.937+0000 7f5cfe544b80 20 client.26344 trim_cache size 0 max 16384 2023-10-08T09:24:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 Processor -- start 2023-10-08T09:24:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 -- start start 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8d9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8e2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d6a58c0 con 0x559c9d8d27a0 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d7ece50 con 0x559c9d8d32b0 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d8e4ba0 con 0x559c9d8d1e40 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8e2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8d9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8d9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39758/0 (socket says 172.21.15.49:39758) 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 -- 172.21.15.49:0/928639558 learned_addr learned my addr 172.21.15.49:0/928639558 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5ceb7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8e2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52726/0 (socket says 172.21.15.49:52726) 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cf08f8700 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 -- 172.21.15.49:0/928639558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 msgr2=0x559c9d8d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 -- 172.21.15.49:0/928639558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 msgr2=0x559c9d8e2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8e2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.955+0000 7f5cebfff700 1 -- 172.21.15.49:0/928639558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c9d6761f0 con 0x559c9d8d1e40 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.956+0000 7f5cebfff700 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8d9a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc005d80 tx=0x7f5cdc005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e761c01ee7952570 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.956+0000 7f5cd2ffd700 1 -- 172.21.15.49:0/928639558 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0279b0 con 0x559c9d8d1e40 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.956+0000 7f5cd2ffd700 1 -- 172.21.15.49:0/928639558 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cdc027410 con 0x559c9d8d1e40 2023-10-08T09:24:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.956+0000 7f5cd2ffd700 1 -- 172.21.15.49:0/928639558 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc01eb60 con 0x559c9d8d1e40 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- 172.21.15.49:0/928639558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 msgr2=0x559c9d8d9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8d9a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc005d80 tx=0x7f5cdc005b40 comp rx=0 tx=0).stop 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- 172.21.15.49:0/928639558 shutdown_connections 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8d9a00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/928639558 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8e2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- 172.21.15.49:0/928639558 >> 172.21.15.49:0/928639558 conn(0x559c9d846200 msgr2=0x559c9d8ee4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- 172.21.15.49:0/928639558 shutdown_connections 2023-10-08T09:24:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- 172.21.15.49:0/928639558 wait complete. 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- start start 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 Processor -- start 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8decd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8df210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d8e2720 con 0x559c9d8d27a0 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d8e2890 con 0x559c9d8d32b0 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.957+0000 7f5cfe544b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c9d8e2a50 con 0x559c9d8d1e40 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5cfe544b80 10 client.0 _ll_register_callbacks cb 0x7f5cdc010910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8decd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5cebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8decd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39764/0 (socket says 172.21.15.49:39764) 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5cebfff700 1 -- 172.21.15.49:0/3538229782 learned_addr learned my addr 172.21.15.49:0/3538229782 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8df210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5cf08f8700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3538229782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 msgr2=0x559c9d8decd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8decd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 msgr2=0x559c9d8e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ceb7fe700 1 -- 172.21.15.49:0/3538229782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c9d638c00 con 0x559c9d8d27a0 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8decd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ceb7fe700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8df210 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5ce009a100 tx=0x7f5ce0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e465dc1425285bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ccb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:24:00.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.958+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3538229782 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00b2ea0 con 0x559c9d8d27a0 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.959+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ce400baa0 con 0x559c9d8d27a0 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.959+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3538229782 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5ce009e550 con 0x559c9d8d27a0 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3538229782 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ce00a2790 con 0x559c9d8d27a0 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 10 client.16695 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559c9d64e250 con 0x559c9d8d27a0 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3538229782 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ce400baa0 con 0x559c9d8d27a0 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3538229782 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5ce009e150 con 0x559c9d8d27a0 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 10 client.16695 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 20 client.16695 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d93c540 0x559c9d93e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:00.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 4 client.16695 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:24:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c9d81dad0 con 0x559c9d93c540 2023-10-08T09:24:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.960+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d93c540 0x559c9d93e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.961+0000 7f5ccb7fe700 10 client.16695 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.961+0000 7f5cebfff700 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d93c540 0x559c9d93e9e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837820bdf97d0352 server_cookie=f5dad05189ddba61 in_seq=0 out_seq=0 2023-10-08T09:24:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5ccb7fe700 1 -- 172.21.15.49:0/3538229782 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c9d81dad0 con 0x559c9d93c540 2023-10-08T09:24:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5ccb7fe700 10 client.16695 handle_command_reply: tid=0 2023-10-08T09:24:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 client.16695 shutdown 2023-10-08T09:24:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 msgr2=0x559c9d8df210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8df210 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5ce009a100 tx=0x7f5ce0010d50 comp rx=0 tx=0).stop 2023-10-08T09:24:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 shutdown_connections 2023-10-08T09:24:00.977 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559c9d93c540 0x559c9d93e9e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559c9d8d1e40 0x559c9d8decd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559c9d8d32b0 0x559c9d8e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 --2- 172.21.15.49:0/3538229782 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559c9d8d27a0 0x559c9d8df210 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5ccb7fe700 0 client.16695 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 >> 172.21.15.49:0/3538229782 conn(0x559c9d846200 msgr2=0x559c9d8c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 shutdown_connections 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.975+0000 7f5cfe544b80 1 -- 172.21.15.49:0/3538229782 wait complete. 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cfe544b80 20 client.16695 trim_cache size 0 max 16384 2023-10-08T09:24:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:24:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:24:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:24:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5cd40a1480 msgr2=0x7f5cd40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5cd40a1480 0x7f5cd40a3940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5cec107de0 tx=0x7f5cdc009260 comp rx=0 tx=0).stop 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 msgr2=0x7f5cec19d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 0x7f5cec19d2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000e9a0 tx=0x7f5ce000ed60 comp rx=0 tx=0).stop 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 shutdown_connections 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5cd40a1480 0x7f5cd40a3940 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cec108920 0x7f5cec1a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cec106d90 0x7f5cec19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 --2- 172.21.15.49:0/3378129376 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5cec107f80 0x7f5cec19d2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 >> 172.21.15.49:0/3378129376 conn(0x7f5cec075d20 msgr2=0x7f5cec10b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 shutdown_connections 2023-10-08T09:24:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:00.976+0000 7f5cf235b700 1 -- 172.21.15.49:0/3378129376 wait complete. 2023-10-08T09:24:01.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:01 smithi049 ceph-mon[99485]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-08T09:24:01.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:01 smithi184 ceph-mon[103675]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-08T09:24:01.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:01 smithi132 ceph-mon[103389]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-08T09:24:03.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:02 smithi049 ceph-mon[99485]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.7 MiB/s wr, 557 op/s 2023-10-08T09:24:03.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:02 smithi184 ceph-mon[103675]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.7 MiB/s wr, 557 op/s 2023-10-08T09:24:03.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:02 smithi132 ceph-mon[103389]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.7 MiB/s wr, 557 op/s 2023-10-08T09:24:04.315 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 520.0 s, 549.7 tps, lat 183.499 ms stddev 205.509 2023-10-08T09:24:05.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:05 smithi184 ceph-mon[103675]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 521 op/s 2023-10-08T09:24:05.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:05 smithi132 ceph-mon[103389]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 521 op/s 2023-10-08T09:24:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:05 smithi049 ceph-mon[99485]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.4 MiB/s wr, 521 op/s 2023-10-08T09:24:07.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:07 smithi184 ceph-mon[103675]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 570 op/s 2023-10-08T09:24:07.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:07 smithi132 ceph-mon[103389]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 570 op/s 2023-10-08T09:24:07.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:07 smithi049 ceph-mon[99485]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 570 op/s 2023-10-08T09:24:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 525.0 s, 514.2 tps, lat 195.230 ms stddev 224.324 2023-10-08T09:24:09.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:09 smithi184 ceph-mon[103675]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 MiB/s wr, 516 op/s 2023-10-08T09:24:09.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:09 smithi132 ceph-mon[103389]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 MiB/s wr, 516 op/s 2023-10-08T09:24:09.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:09 smithi049 ceph-mon[99485]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 6.0 MiB/s wr, 516 op/s 2023-10-08T09:24:11.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:11 smithi184 ceph-mon[103675]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.7 MiB/s wr, 499 op/s 2023-10-08T09:24:11.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:11 smithi132 ceph-mon[103389]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.7 MiB/s wr, 499 op/s 2023-10-08T09:24:11.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:11 smithi049 ceph-mon[99485]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.7 MiB/s wr, 499 op/s 2023-10-08T09:24:13.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:12 smithi049 ceph-mon[99485]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.4 MiB/s wr, 545 op/s 2023-10-08T09:24:13.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:12 smithi184 ceph-mon[103675]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.4 MiB/s wr, 545 op/s 2023-10-08T09:24:13.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:12 smithi132 ceph-mon[103389]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.4 MiB/s wr, 545 op/s 2023-10-08T09:24:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.157+0000 7fb33c32c700 1 -- 172.21.15.49:0/3271310671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340fee00 msgr2=0x7fb3340ff220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.157+0000 7fb33c32c700 1 --2- 172.21.15.49:0/3271310671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340fee00 0x7fb3340ff220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb330009920 tx=0x7fb330009c30 comp rx=0 tx=0).stop 2023-10-08T09:24:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.160+0000 7fb33c32c700 1 -- 172.21.15.49:0/3271310671 shutdown_connections 2023-10-08T09:24:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.160+0000 7fb33c32c700 1 --2- 172.21.15.49:0/3271310671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb334100990 0x7fb334107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.160+0000 7fb33c32c700 1 --2- 172.21.15.49:0/3271310671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340fee00 0x7fb3340ff220 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.160+0000 7fb33c32c700 1 --2- 172.21.15.49:0/3271310671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3340ffff0 0x7fb334100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.160+0000 7fb33c32c700 1 -- 172.21.15.49:0/3271310671 >> 172.21.15.49:0/3271310671 conn(0x7fb3340fa370 msgr2=0x7fb3340fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.173+0000 7fb33c32c700 1 -- 172.21.15.49:0/3271310671 shutdown_connections 2023-10-08T09:24:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.173+0000 7fb33c32c700 1 -- 172.21.15.49:0/3271310671 wait complete. 2023-10-08T09:24:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.174+0000 7fb33c32c700 1 Processor -- start 2023-10-08T09:24:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.174+0000 7fb33c32c700 1 -- start start 2023-10-08T09:24:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.174+0000 7fb33c32c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3340fee00 0x7fb33410f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.174+0000 7fb33c32c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340ffff0 0x7fb33410fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.174+0000 7fb33c32c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb334100990 0x7fb334116e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.174+0000 7fb33c32c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334117430 con 0x7fb3340fee00 2023-10-08T09:24:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb33c32c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334117570 con 0x7fb3340ffff0 2023-10-08T09:24:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb33c32c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334117700 con 0x7fb334100990 2023-10-08T09:24:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb3398c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340ffff0 0x7fb33410fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb3398c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340ffff0 0x7fb33410fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42366/0 (socket says 172.21.15.49:42366) 2023-10-08T09:24:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb3398c7700 1 -- 172.21.15.49:0/2254416951 learned_addr learned my addr 172.21.15.49:0/2254416951 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb33a0c8700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3340fee00 0x7fb33410f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb33a0c8700 1 -- 172.21.15.49:0/2254416951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb334100990 msgr2=0x7fb334116e60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:24:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb33a0c8700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb334100990 0x7fb334116e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb33a0c8700 1 -- 172.21.15.49:0/2254416951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340ffff0 msgr2=0x7fb33410fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.175+0000 7fb33a0c8700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340ffff0 0x7fb33410fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb33a0c8700 1 -- 172.21.15.49:0/2254416951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330009520 con 0x7fb3340fee00 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb3398c7700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340ffff0 0x7fb33410fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb33a0c8700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3340fee00 0x7fb33410f900 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb330000c00 tx=0x7fb33000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb32b7fe700 1 -- 172.21.15.49:0/2254416951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33001c070 con 0x7fb3340fee00 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb32b7fe700 1 -- 172.21.15.49:0/2254416951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb330005500 con 0x7fb3340fee00 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb32b7fe700 1 -- 172.21.15.49:0/2254416951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330003a80 con 0x7fb3340fee00 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb334117980 con 0x7fb3340fee00 2023-10-08T09:24:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.176+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33406a210 con 0x7fb3340fee00 2023-10-08T09:24:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 Processor -- start 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 -- start start 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cae40 0x55bff99cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cc2b0 0x55bff99cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff9933400 con 0x55bff99cae40 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff9924bc0 con 0x55bff99cb7a0 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb348515b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff99224e0 con 0x55bff99cc2b0 2023-10-08T09:24:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb33a8c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cae40 0x55bff99cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb33a8c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cae40 0x55bff99cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52758/0 (socket says 172.21.15.49:52758) 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.181+0000 7fb33a8c9700 1 -- 172.21.15.49:0/1577595949 learned_addr learned my addr 172.21.15.49:0/1577595949 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.182+0000 7fb3398c7700 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cc2b0 0x55bff99cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.182+0000 7fb33a8c9700 1 -- 172.21.15.49:0/1577595949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cc2b0 msgr2=0x55bff99cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.182+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cc2b0 0x55bff99cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.182+0000 7fb33a8c9700 1 -- 172.21.15.49:0/1577595949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 msgr2=0x55bff99cbcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.182+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99cbcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.182+0000 7fb33a8c9700 1 -- 172.21.15.49:0/1577595949 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bff9748d80 con 0x55bff99cae40 2023-10-08T09:24:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.182+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cae40 0x55bff99cb260 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00e4e0 tx=0x7fb32c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81854cbfa6d07e41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.184+0000 7fb32b7fe700 1 -- 172.21.15.49:0/2254416951 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb330020030 con 0x7fb3340fee00 2023-10-08T09:24:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.184+0000 7fb32b7fe700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb3200a1550 0x7fb3200a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.184+0000 7fb33a0c8700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb3200a1550 0x7fb3200a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.184+0000 7fb32b7fe700 1 -- 172.21.15.49:0/2254416951 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb3300ce1d0 con 0x7fb3340fee00 2023-10-08T09:24:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.185+0000 7fb328d82700 1 -- 172.21.15.49:0/1577595949 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c017070 con 0x55bff99cae40 2023-10-08T09:24:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.185+0000 7fb328d82700 1 -- 172.21.15.49:0/1577595949 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb32c00e050 con 0x55bff99cae40 2023-10-08T09:24:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.185+0000 7fb328d82700 1 -- 172.21.15.49:0/1577595949 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c007cb0 con 0x55bff99cae40 2023-10-08T09:24:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.186+0000 7fb348515b80 1 -- 172.21.15.49:0/1577595949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cae40 msgr2=0x55bff99cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.186+0000 7fb33a0c8700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb3200a1550 0x7fb3200a3a10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb330005950 tx=0x7fb3300909b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:14.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.186+0000 7fb348515b80 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cae40 0x55bff99cb260 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00e4e0 tx=0x7fb32c0099e0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.188+0000 7fb348515b80 1 -- 172.21.15.49:0/1577595949 shutdown_connections 2023-10-08T09:24:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.188+0000 7fb348515b80 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cc2b0 0x55bff99cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.188+0000 7fb348515b80 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cae40 0x55bff99cb260 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.188+0000 7fb348515b80 1 --2- 172.21.15.49:0/1577595949 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.188+0000 7fb348515b80 1 -- 172.21.15.49:0/1577595949 >> 172.21.15.49:0/1577595949 conn(0x55bff993f200 msgr2=0x55bff99c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.189+0000 7fb348515b80 1 -- 172.21.15.49:0/1577595949 shutdown_connections 2023-10-08T09:24:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.189+0000 7fb348515b80 1 -- 172.21.15.49:0/1577595949 wait complete. 2023-10-08T09:24:14.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.189+0000 7fb348515b80 1 -- start start 2023-10-08T09:24:14.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.189+0000 7fb348515b80 1 Processor -- start 2023-10-08T09:24:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb348515b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb348515b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb348515b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99dbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb348515b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff991ddd0 con 0x55bff99cc2b0 2023-10-08T09:24:14.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb348515b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff98ba6b0 con 0x55bff99cb7a0 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb348515b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff98ba350 con 0x55bff99cae40 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb348515b80 10 client.0 _ll_register_callbacks cb 0x55bff97177a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb33a8c9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb3398c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb33a8c9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39798/0 (socket says 172.21.15.49:39798) 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb33a8c9700 1 -- 172.21.15.49:0/3242380001 learned_addr learned my addr 172.21.15.49:0/3242380001 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99dbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb3398c7700 1 -- 172.21.15.49:0/3242380001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 msgr2=0x55bff99d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb3398c7700 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb3398c7700 1 -- 172.21.15.49:0/3242380001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 msgr2=0x55bff99dbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb3398c7700 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99dbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb3398c7700 1 -- 172.21.15.49:0/3242380001 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c00e1c0 con 0x55bff99cb7a0 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb3398c7700 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d4df0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3240119b0 tx=0x7fb324011cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6187829308fea3c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.190+0000 7fb31dffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb31dffb700 1 -- 172.21.15.49:0/3242380001 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400f940 con 0x55bff99cb7a0 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bff98e5e50 con 0x55bff99cb7a0 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb31dffb700 1 -- 172.21.15.49:0/3242380001 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb32400faa0 con 0x55bff99cb7a0 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb31dffb700 1 -- 172.21.15.49:0/3242380001 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32400fc60 con 0x55bff99cb7a0 2023-10-08T09:24:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb348515b80 10 client.26312 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55bff9731c00 con 0x55bff99cb7a0 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb31dffb700 1 -- 172.21.15.49:0/3242380001 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bff98e5e50 con 0x55bff99cb7a0 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb31dffb700 1 -- 172.21.15.49:0/3242380001 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb32401b020 con 0x55bff99cb7a0 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb348515b80 10 client.26312 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.191+0000 7fb348515b80 10 client.26312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.192+0000 7fb348515b80 20 client.26312 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.192+0000 7fb348515b80 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff99e07e0 0x55bff99e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.192+0000 7fb348515b80 4 client.26312 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.192+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bff9916db0 con 0x55bff99e07e0 2023-10-08T09:24:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.192+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff99e07e0 0x55bff99e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.192+0000 7fb31dffb700 10 client.26312 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.192+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff99e07e0 0x55bff99e2c00 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2740794558431b server_cookie=3aeba18f9bcdba8 in_seq=0 out_seq=0 2023-10-08T09:24:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.193+0000 7fb31dffb700 1 -- 172.21.15.49:0/3242380001 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x7fb32c02a730 con 0x55bff99e07e0 2023-10-08T09:24:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.193+0000 7fb31dffb700 10 client.26312 handle_command_reply: tid=0 2023-10-08T09:24:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.193+0000 7fb348515b80 1 client.26312 shutdown 2023-10-08T09:24:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.194+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 msgr2=0x55bff99d4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.197 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.194+0000 7fb348515b80 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d4df0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3240119b0 tx=0x7fb324011cc0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 shutdown_connections 2023-10-08T09:24:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff99e07e0 0x55bff99e2c00 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99dbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 --2- 172.21.15.49:0/3242380001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d4df0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 >> 172.21.15.49:0/3242380001 conn(0x55bff993f200 msgr2=0x55bff99c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb31dffb700 0 client.26312 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 shutdown_connections 2023-10-08T09:24:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 1 -- 172.21.15.49:0/3242380001 wait complete. 2023-10-08T09:24:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.201+0000 7fb348515b80 20 client.26312 trim_cache size 0 max 16384 2023-10-08T09:24:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.224+0000 7fb348515b80 1 Processor -- start 2023-10-08T09:24:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb348515b80 1 -- start start 2023-10-08T09:24:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb348515b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb348515b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99dbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb348515b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb348515b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff979e8c0 con 0x55bff99cc2b0 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb348515b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff99cbb80 con 0x55bff99cb7a0 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb348515b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff98e5e50 con 0x55bff99cae40 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52774/0 (socket says 172.21.15.49:52774) 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 -- 172.21.15.49:0/2292484310 learned_addr learned my addr 172.21.15.49:0/2292484310 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 -- 172.21.15.49:0/2292484310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 msgr2=0x55bff99d3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 -- 172.21.15.49:0/2292484310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 msgr2=0x55bff99dbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99dbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.225+0000 7fb33a8c9700 1 -- 172.21.15.49:0/2292484310 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bff976f1f0 con 0x55bff99cc2b0 2023-10-08T09:24:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.226+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99d48b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55bff99c73b0 tx=0x7fb32c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77319d4ed814687 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.226+0000 7fb329583700 1 -- 172.21.15.49:0/2292484310 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c017040 con 0x55bff99cc2b0 2023-10-08T09:24:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.226+0000 7fb329583700 1 -- 172.21.15.49:0/2292484310 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb32c026030 con 0x55bff99cc2b0 2023-10-08T09:24:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.226+0000 7fb329583700 1 -- 172.21.15.49:0/2292484310 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0079e0 con 0x55bff99cc2b0 2023-10-08T09:24:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.226+0000 7fb348515b80 1 -- 172.21.15.49:0/2292484310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 msgr2=0x55bff99d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.226+0000 7fb348515b80 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99d48b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x55bff99c73b0 tx=0x7fb32c00b040 comp rx=0 tx=0).stop 2023-10-08T09:24:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- 172.21.15.49:0/2292484310 shutdown_connections 2023-10-08T09:24:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99d48b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 --2- 172.21.15.49:0/2292484310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99dbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- 172.21.15.49:0/2292484310 >> 172.21.15.49:0/2292484310 conn(0x55bff993f200 msgr2=0x55bff99c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- 172.21.15.49:0/2292484310 shutdown_connections 2023-10-08T09:24:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- 172.21.15.49:0/2292484310 wait complete. 2023-10-08T09:24:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- start start 2023-10-08T09:24:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 Processor -- start 2023-10-08T09:24:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff99db180 con 0x55bff99cc2b0 2023-10-08T09:24:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff99db2f0 con 0x55bff99cb7a0 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bff99db4b0 con 0x55bff99cae40 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.227+0000 7fb348515b80 10 client.0 _ll_register_callbacks cb 0x7fb3240112c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42392/0 (socket says 172.21.15.49:42392) 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 -- 172.21.15.49:0/276070223 learned_addr learned my addr 172.21.15.49:0/276070223 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 -- 172.21.15.49:0/276070223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 msgr2=0x55bff99d97e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d97e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 -- 172.21.15.49:0/276070223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 msgr2=0x55bff99dac40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:24:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99dac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.228+0000 7fb3398c7700 1 -- 172.21.15.49:0/276070223 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c028040 con 0x55bff99cb7a0 2023-10-08T09:24:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb3398c7700 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d9d20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb324014e70 tx=0x7fb32400fca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55ed316736d99d28 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb31dffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:24:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb31dffb700 1 -- 172.21.15.49:0/276070223 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324009980 con 0x55bff99cb7a0 2023-10-08T09:24:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb32c00e050 con 0x55bff99cb7a0 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb31dffb700 1 -- 172.21.15.49:0/276070223 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb324000e40 con 0x55bff99cb7a0 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb31dffb700 1 -- 172.21.15.49:0/276070223 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32400f520 con 0x55bff99cb7a0 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb348515b80 10 client.26315 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55bff9747250 con 0x55bff99cb7a0 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb31dffb700 1 -- 172.21.15.49:0/276070223 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb324032020 con 0x55bff99cb7a0 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.229+0000 7fb31dffb700 1 -- 172.21.15.49:0/276070223 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb32401b070 con 0x55bff99cb7a0 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb348515b80 10 client.26315 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb348515b80 10 client.26315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb348515b80 20 client.26315 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb348515b80 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff9a35100 0x55bff9a37590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb348515b80 4 client.26315 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:24:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bff9916ad0 con 0x55bff9a35100 2023-10-08T09:24:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff9a35100 0x55bff9a37590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb31dffb700 10 client.26315 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.230+0000 7fb33a8c9700 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff9a35100 0x55bff9a37590 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc1aa0497fbbc67a server_cookie=ec9a7c73f25b1a5d in_seq=0 out_seq=0 2023-10-08T09:24:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.239+0000 7fb31dffb700 1 -- 172.21.15.49:0/276070223 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bff9916ad0 con 0x55bff9a35100 2023-10-08T09:24:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.239+0000 7fb31dffb700 10 client.26315 handle_command_reply: tid=0 2023-10-08T09:24:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.239+0000 7fb348515b80 1 client.26315 shutdown 2023-10-08T09:24:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 msgr2=0x55bff99d9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d9d20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb324014e70 tx=0x7fb32400fca0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 shutdown_connections 2023-10-08T09:24:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bff9a35100 0x55bff9a37590 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bff99cae40 0x55bff99d97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bff99cc2b0 0x55bff99dac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 --2- 172.21.15.49:0/276070223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bff99cb7a0 0x55bff99d9d20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 >> 172.21.15.49:0/276070223 conn(0x55bff993f200 msgr2=0x55bff99d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb31dffb700 0 client.26315 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 shutdown_connections 2023-10-08T09:24:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.240+0000 7fb348515b80 1 -- 172.21.15.49:0/276070223 wait complete. 2023-10-08T09:24:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb348515b80 20 client.26315 trim_cache size 0 max 16384 2023-10-08T09:24:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:24:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:24:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:24:14.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:24:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb3200a1550 msgr2=0x7fb3200a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb3200a1550 0x7fb3200a3a10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb330005950 tx=0x7fb3300909b0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3340fee00 msgr2=0x7fb33410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3340fee00 0x7fb33410f900 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb330000c00 tx=0x7fb33000b790 comp rx=0 tx=0).stop 2023-10-08T09:24:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 shutdown_connections 2023-10-08T09:24:14.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb3200a1550 0x7fb3200a3a10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb334100990 0x7fb334116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3340fee00 0x7fb33410f900 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 --2- 172.21.15.49:0/2254416951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3340ffff0 0x7fb33410fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 >> 172.21.15.49:0/2254416951 conn(0x7fb3340fa370 msgr2=0x7fb334103220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 shutdown_connections 2023-10-08T09:24:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:14.241+0000 7fb33c32c700 1 -- 172.21.15.49:0/2254416951 wait complete. 2023-10-08T09:24:14.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 530.0 s, 530.9 tps, lat 190.027 ms stddev 205.491 2023-10-08T09:24:15.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:15 smithi184 ceph-mon[103675]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2023-10-08T09:24:15.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:15 smithi132 ceph-mon[103389]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2023-10-08T09:24:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:15 smithi049 ceph-mon[99485]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 MiB/s wr, 518 op/s 2023-10-08T09:24:17.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:24:17.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:17 smithi184 ceph-mon[103675]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2023-10-08T09:24:17.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:17 smithi132 ceph-mon[103389]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2023-10-08T09:24:17.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.795+0000 7f906918b700 1 -- 172.21.15.49:0/1022198667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 msgr2=0x7f9064072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.799 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.795+0000 7f906918b700 1 --2- 172.21.15.49:0/1022198667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064072c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9054009990 tx=0x7f9054009ca0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.801+0000 7f906918b700 1 -- 172.21.15.49:0/1022198667 shutdown_connections 2023-10-08T09:24:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.801+0000 7f906918b700 1 --2- 172.21.15.49:0/1022198667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064072c80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.801+0000 7f906918b700 1 --2- 172.21.15.49:0/1022198667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f906410cd90 0x7f906410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.801+0000 7f906918b700 1 --2- 172.21.15.49:0/1022198667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9064073250 0x7f90640736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.801+0000 7f906918b700 1 -- 172.21.15.49:0/1022198667 >> 172.21.15.49:0/1022198667 conn(0x7f90640fbd30 msgr2=0x7f90640fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:17.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.804+0000 7f906918b700 1 -- 172.21.15.49:0/1022198667 shutdown_connections 2023-10-08T09:24:17.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.805+0000 7f906918b700 1 -- 172.21.15.49:0/1022198667 wait complete. 2023-10-08T09:24:17.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 Processor -- start 2023-10-08T09:24:17.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 -- start start 2023-10-08T09:24:17.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9064073250 0x7f9064109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064109990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f906410cd90 0x7f90641ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906410ca20 con 0x7f9064073250 2023-10-08T09:24:17.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906410cb60 con 0x7f906410cd90 2023-10-08T09:24:17.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.806+0000 7f906918b700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906406a210 con 0x7f9064074830 2023-10-08T09:24:17.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.807+0000 7f906259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064109990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064109990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39822/0 (socket says 172.21.15.49:39822) 2023-10-08T09:24:17.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 -- 172.21.15.49:0/1506903477 learned_addr learned my addr 172.21.15.49:0/1506903477 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:17.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f9062d9d700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9064073250 0x7f9064109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 -- 172.21.15.49:0/1506903477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f906410cd90 msgr2=0x7f90641ad000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:24:17.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f906410cd90 0x7f90641ad000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 -- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9064073250 msgr2=0x7f9064109450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9064073250 0x7f9064109450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 -- 172.21.15.49:0/1506903477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540095f0 con 0x7f9064074830 2023-10-08T09:24:17.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.808+0000 7f906259c700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064109990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f905800b550 tx=0x7f905800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:17.816 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.815+0000 7f904bfff700 1 -- 172.21.15.49:0/1506903477 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058011830 con 0x7f9064074830 2023-10-08T09:24:17.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.815+0000 7f904bfff700 1 -- 172.21.15.49:0/1506903477 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9058011e50 con 0x7f9064074830 2023-10-08T09:24:17.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.815+0000 7f904bfff700 1 -- 172.21.15.49:0/1506903477 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905800f2e0 con 0x7f9064074830 2023-10-08T09:24:17.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.816+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906406a4f0 con 0x7f9064074830 2023-10-08T09:24:17.817 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.816+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906406a9b0 con 0x7f9064074830 2023-10-08T09:24:17.821 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.821+0000 7f9075374b80 1 Processor -- start 2023-10-08T09:24:17.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.828+0000 7f904bfff700 1 -- 172.21.15.49:0/1506903477 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9058011990 con 0x7f9064074830 2023-10-08T09:24:17.829 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.829+0000 7f904bfff700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f904c0a1600 0x7f904c0a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f904bfff700 1 -- 172.21.15.49:0/1506903477 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f90580ccad0 con 0x7f9064074830 2023-10-08T09:24:17.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f9075374b80 1 -- start start 2023-10-08T09:24:17.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f9075374b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672134060 0x562672134480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f9075374b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5626721349c0 0x562672135270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f9075374b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672135800 0x562672135c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.831 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f9075374b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267209c460 con 0x562672134060 2023-10-08T09:24:17.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f9075374b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267208dc20 con 0x5626721349c0 2023-10-08T09:24:17.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.830+0000 7f9075374b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267208b540 con 0x562672135800 2023-10-08T09:24:17.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f904c0a1600 0x7f904c0a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.832 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5626721349c0 0x562672135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672135800 0x562672135c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672135800 0x562672135c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39832/0 (socket says 172.21.15.49:39832) 2023-10-08T09:24:17.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 -- 172.21.15.49:0/1708691595 learned_addr learned my addr 172.21.15.49:0/1708691595 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:17.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f9062d9d700 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672134060 0x562672134480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f904c0a1600 0x7f904c0a3ac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f905c010560 tx=0x7f905c010920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:17.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 -- 172.21.15.49:0/1708691595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5626721349c0 msgr2=0x562672135270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5626721349c0 0x562672135270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 -- 172.21.15.49:0/1708691595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672134060 msgr2=0x562672134480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672134060 0x562672134480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.831+0000 7f906359e700 1 -- 172.21.15.49:0/1708691595 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562671eb1de0 con 0x562672135800 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.832+0000 7f906359e700 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672135800 0x562672135c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c730 tx=0x7f905c018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4893826876948b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.832+0000 7f90497fa700 1 -- 172.21.15.49:0/1708691595 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c010230 con 0x562672135800 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.832+0000 7f90497fa700 1 -- 172.21.15.49:0/1708691595 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905c016070 con 0x562672135800 2023-10-08T09:24:17.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.832+0000 7f90497fa700 1 -- 172.21.15.49:0/1708691595 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c01c620 con 0x562672135800 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 -- 172.21.15.49:0/1708691595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672135800 msgr2=0x562672135c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672135800 0x562672135c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c730 tx=0x7f905c018000 comp rx=0 tx=0).stop 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 -- 172.21.15.49:0/1708691595 shutdown_connections 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672135800 0x562672135c90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672134060 0x562672134480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 --2- 172.21.15.49:0/1708691595 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5626721349c0 0x562672135270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 -- 172.21.15.49:0/1708691595 >> 172.21.15.49:0/1708691595 conn(0x5626720a8260 msgr2=0x56267212ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.833+0000 7f9075374b80 1 -- 172.21.15.49:0/1708691595 shutdown_connections 2023-10-08T09:24:17.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.834+0000 7f9075374b80 1 -- 172.21.15.49:0/1708691595 wait complete. 2023-10-08T09:24:17.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.834+0000 7f9075374b80 1 -- start start 2023-10-08T09:24:17.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.834+0000 7f9075374b80 1 Processor -- start 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f9075374b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672134060 0x56267213dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f9075374b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x56267213e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f9075374b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672143430 0x562672145820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f9075374b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562672086e30 con 0x562672135800 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f9075374b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x562672023710 con 0x562672134060 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f9075374b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626720233b0 con 0x562672143430 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f9075374b80 10 client.0 _ll_register_callbacks cb 0x562671e80800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f906259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x56267213e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f906259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x56267213e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52800/0 (socket says 172.21.15.49:52800) 2023-10-08T09:24:17.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.837+0000 7f906259c700 1 -- 172.21.15.49:0/3518197328 learned_addr learned my addr 172.21.15.49:0/3518197328 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:17.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f906259c700 1 -- 172.21.15.49:0/3518197328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672143430 msgr2=0x562672145820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:24:17.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f906259c700 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672143430 0x562672145820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f9062d9d700 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672134060 0x56267213dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f906259c700 1 -- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672134060 msgr2=0x56267213dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:17.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f906259c700 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672134060 0x56267213dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f906259c700 1 -- 172.21.15.49:0/3518197328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c00c460 con 0x562672135800 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f9062d9d700 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672134060 0x56267213dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f906259c700 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x56267213e400 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9058096530 tx=0x7f9058099550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e4bc162c60f9c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.838+0000 7f9046ff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.839+0000 7f9046ff5700 1 -- 172.21.15.49:0/3518197328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905809dc10 con 0x562672135800 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.839+0000 7f9046ff5700 1 -- 172.21.15.49:0/3518197328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f90580a1e50 con 0x562672135800 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.839+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56267204eeb0 con 0x562672135800 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.839+0000 7f9046ff5700 1 -- 172.21.15.49:0/3518197328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9058010380 con 0x562672135800 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.839+0000 7f9075374b80 10 client.16716 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:17.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.839+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562671e9ac60 con 0x562672135800 2023-10-08T09:24:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.839+0000 7f9046ff5700 1 -- 172.21.15.49:0/3518197328 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56267204eeb0 con 0x562672135800 2023-10-08T09:24:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9046ff5700 1 -- 172.21.15.49:0/3518197328 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f90580b2210 con 0x562672135800 2023-10-08T09:24:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9075374b80 10 client.16716 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:17.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9075374b80 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:17.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9075374b80 20 client.16716 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:17.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9075374b80 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267214a140 0x56267214c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9075374b80 4 client.16716 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:24:17.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56267207fe10 con 0x56267214a140 2023-10-08T09:24:17.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.841+0000 7f9062d9d700 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267214a140 0x56267214c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.842+0000 7f9062d9d700 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267214a140 0x56267214c560 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43daed3fa4b15f0e server_cookie=183706b7a85873ae in_seq=0 out_seq=0 2023-10-08T09:24:17.843 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.842+0000 7f9046ff5700 10 client.16716 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:17.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.842+0000 7f9046ff5700 1 -- 172.21.15.49:0/3518197328 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56267207fe10 con 0x56267214a140 2023-10-08T09:24:17.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.842+0000 7f9046ff5700 10 client.16716 handle_command_reply: tid=0 2023-10-08T09:24:17.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.844+0000 7f9075374b80 1 client.16716 shutdown 2023-10-08T09:24:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 msgr2=0x56267213e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x56267213e400 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9058096530 tx=0x7f9058099550 comp rx=0 tx=0).stop 2023-10-08T09:24:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 shutdown_connections 2023-10-08T09:24:17.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267214a140 0x56267214c560 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672143430 0x562672145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672134060 0x56267213dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 --2- 172.21.15.49:0/3518197328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x56267213e400 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.849+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 >> 172.21.15.49:0/3518197328 conn(0x5626720a8260 msgr2=0x56267212e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.850+0000 7f9046ff5700 0 client.16716 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.850+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 shutdown_connections 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.850+0000 7f9075374b80 1 -- 172.21.15.49:0/3518197328 wait complete. 2023-10-08T09:24:17.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.850+0000 7f9075374b80 20 client.16716 trim_cache size 0 max 16384 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.878+0000 7f9075374b80 1 Processor -- start 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.879+0000 7f9075374b80 1 -- start start 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.879+0000 7f9075374b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x562672159d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.879+0000 7f9075374b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x562672135010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.879+0000 7f9075374b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672143430 0x5626721387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.879+0000 7f9075374b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671f07920 con 0x562672135800 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.879+0000 7f9075374b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267204eeb0 con 0x562672143430 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f9075374b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267213da30 con 0x562672134060 2023-10-08T09:24:17.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672143430 0x5626721387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672143430 0x5626721387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42422/0 (socket says 172.21.15.49:42422) 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 -- 172.21.15.49:0/2175598025 learned_addr learned my addr 172.21.15.49:0/2175598025 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f9062d9d700 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x562672159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906259c700 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x562672135010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 -- 172.21.15.49:0/2175598025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 msgr2=0x562672159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x562672159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 -- 172.21.15.49:0/2175598025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 msgr2=0x562672135010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x562672135010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.880+0000 7f906359e700 1 -- 172.21.15.49:0/2175598025 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562671ed8250 con 0x562672143430 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.881+0000 7f906359e700 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672143430 0x5626721387a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f905c010d20 tx=0x7f905c00a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a316779c23b61b92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:17.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.881+0000 7f9049ffb700 1 -- 172.21.15.49:0/2175598025 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c00aa00 con 0x562672143430 2023-10-08T09:24:17.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.881+0000 7f9049ffb700 1 -- 172.21.15.49:0/2175598025 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905c00ce80 con 0x562672143430 2023-10-08T09:24:17.884 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.881+0000 7f9049ffb700 1 -- 172.21.15.49:0/2175598025 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c01c780 con 0x562672143430 2023-10-08T09:24:17.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:17 smithi049 ceph-mon[99485]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.5 MiB/s wr, 589 op/s 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 -- 172.21.15.49:0/2175598025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672143430 msgr2=0x5626721387a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672143430 0x5626721387a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f905c010d20 tx=0x7f905c00a990 comp rx=0 tx=0).stop 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 -- 172.21.15.49:0/2175598025 shutdown_connections 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x562672159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672143430 0x5626721387a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 --2- 172.21.15.49:0/2175598025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672135800 0x562672135010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 -- 172.21.15.49:0/2175598025 >> 172.21.15.49:0/2175598025 conn(0x5626720a8260 msgr2=0x56267212af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 -- 172.21.15.49:0/2175598025 shutdown_connections 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 -- 172.21.15.49:0/2175598025 wait complete. 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.882+0000 7f9075374b80 1 -- start start 2023-10-08T09:24:17.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 1 Processor -- start 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x5626721986b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672135800 0x56267219ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672143430 0x56267219d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267219d8c0 con 0x562672143430 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267219da30 con 0x562672135800 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267219dbf0 con 0x562672134060 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9075374b80 10 client.0 _ll_register_callbacks cb 0x7f9058099960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f906259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672135800 0x56267219ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9062d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x5626721986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f906259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672135800 0x56267219ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42428/0 (socket says 172.21.15.49:42428) 2023-10-08T09:24:17.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f906259c700 1 -- 172.21.15.49:0/2342796715 learned_addr learned my addr 172.21.15.49:0/2342796715 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:17.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f906359e700 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672143430 0x56267219d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9062d9d700 1 -- 172.21.15.49:0/2342796715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672135800 msgr2=0x56267219ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9062d9d700 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672135800 0x56267219ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9062d9d700 1 -- 172.21.15.49:0/2342796715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672143430 msgr2=0x56267219d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9062d9d700 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672143430 0x56267219d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.883+0000 7f9062d9d700 1 -- 172.21.15.49:0/2342796715 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562671eb1de0 con 0x562672134060 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9062d9d700 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x5626721986b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90540098e0 tx=0x7f9054005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f977a848d7b7d221 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9046ff5700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9046ff5700 1 -- 172.21.15.49:0/2342796715 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905401f040 con 0x562672134060 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9046ff5700 1 -- 172.21.15.49:0/2342796715 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f905400b9d0 con 0x562672134060 2023-10-08T09:24:17.891 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f905c016070 con 0x562672134060 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9046ff5700 1 -- 172.21.15.49:0/2342796715 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90540245c0 con 0x562672134060 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9075374b80 10 client.26377 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x562671eb02b0 con 0x562672134060 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.884+0000 7f9046ff5700 1 -- 172.21.15.49:0/2342796715 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905c016070 con 0x562672134060 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9046ff5700 1 -- 172.21.15.49:0/2342796715 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f90540050b0 con 0x562672134060 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9075374b80 10 client.26377 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9075374b80 10 client.26377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9075374b80 20 client.26377 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9075374b80 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267219f700 0x5626721a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:17.892 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9075374b80 4 client.26377 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:24:17.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56267207fb30 con 0x56267219f700 2023-10-08T09:24:17.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f906259c700 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267219f700 0x5626721a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:17.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f906259c700 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267219f700 0x5626721a1b20 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6879c79885ef26 server_cookie=cf582cc70e6a2bb2 in_seq=0 out_seq=0 2023-10-08T09:24:17.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9046ff5700 10 client.26377 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:17.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9046ff5700 1 -- 172.21.15.49:0/2342796715 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56267207fb30 con 0x56267219f700 2023-10-08T09:24:17.893 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9046ff5700 10 client.26377 handle_command_reply: tid=0 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.885+0000 7f9075374b80 1 client.26377 shutdown 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 msgr2=0x5626721986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x5626721986b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90540098e0 tx=0x7f9054005c70 comp rx=0 tx=0).stop 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 shutdown_connections 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56267219f700 0x5626721a1b20 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562672134060 0x5626721986b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9046ff5700 0 client.26377 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x562672143430 0x56267219d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.894 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 --2- 172.21.15.49:0/2342796715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x562672135800 0x56267219ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 >> 172.21.15.49:0/2342796715 conn(0x5626720a8260 msgr2=0x56267212da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:17.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 shutdown_connections 2023-10-08T09:24:17.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 1 -- 172.21.15.49:0/2342796715 wait complete. 2023-10-08T09:24:17.895 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.886+0000 7f9075374b80 20 client.26377 trim_cache size 0 max 16384 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:24:17.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:24:17.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:24:17.897 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:24:17.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f904c0a1600 msgr2=0x7f904c0a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f904c0a1600 0x7f904c0a3ac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f905c010560 tx=0x7f905c010920 comp rx=0 tx=0).stop 2023-10-08T09:24:17.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 msgr2=0x7f9064109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:17.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064109990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f905800b550 tx=0x7f905800b910 comp rx=0 tx=0).stop 2023-10-08T09:24:17.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 shutdown_connections 2023-10-08T09:24:17.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f904c0a1600 0x7f904c0a3ac0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f906410cd90 0x7f90641ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9064073250 0x7f9064109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 --2- 172.21.15.49:0/1506903477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9064074830 0x7f9064109990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:17.899 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 >> 172.21.15.49:0/1506903477 conn(0x7f90640fbd30 msgr2=0x7f9064100130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:17.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 shutdown_connections 2023-10-08T09:24:17.900 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:17.887+0000 7f906918b700 1 -- 172.21.15.49:0/1506903477 wait complete. 2023-10-08T09:24:17.905 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:24:18.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:18 smithi132 ceph-mon[103389]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 MiB/s wr, 535 op/s 2023-10-08T09:24:18.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:18 smithi049 ceph-mon[99485]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 MiB/s wr, 535 op/s 2023-10-08T09:24:18.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:18 smithi184 ceph-mon[103675]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 MiB/s wr, 535 op/s 2023-10-08T09:24:19.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 535.0 s, 524.3 tps, lat 190.620 ms stddev 212.399 2023-10-08T09:24:20.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:20 smithi132 ceph-mon[103389]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 MiB/s wr, 541 op/s 2023-10-08T09:24:20.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:20 smithi049 ceph-mon[99485]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 MiB/s wr, 541 op/s 2023-10-08T09:24:21.001 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:20 smithi184 ceph-mon[103675]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 MiB/s wr, 541 op/s 2023-10-08T09:24:23.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:22 smithi049 ceph-mon[99485]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 581 op/s 2023-10-08T09:24:23.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:22 smithi184 ceph-mon[103675]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 581 op/s 2023-10-08T09:24:23.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:22 smithi132 ceph-mon[103389]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 581 op/s 2023-10-08T09:24:24.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 540.0 s, 504.4 tps, lat 194.425 ms stddev 204.537 2023-10-08T09:24:24.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:24 smithi132 ceph-mon[103389]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2023-10-08T09:24:24.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:24 smithi049 ceph-mon[99485]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2023-10-08T09:24:24.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:24 smithi184 ceph-mon[103675]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.9 MiB/s wr, 551 op/s 2023-10-08T09:24:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.466+0000 7fa12ff1d700 1 -- 172.21.15.49:0/3938141582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810a160 msgr2=0x7fa12810c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.466+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/3938141582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810a160 0x7fa12810c5f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa118009990 tx=0x7fa118009ca0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.471+0000 7fa12ff1d700 1 -- 172.21.15.49:0/3938141582 shutdown_connections 2023-10-08T09:24:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.471+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/3938141582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa12810cb30 0x7fa12810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.471+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/3938141582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1281077f0 0x7fa128109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.471+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/3938141582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810a160 0x7fa12810c5f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.471+0000 7fa12ff1d700 1 -- 172.21.15.49:0/3938141582 >> 172.21.15.49:0/3938141582 conn(0x7fa12806d770 msgr2=0x7fa12806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.476+0000 7fa12ff1d700 1 -- 172.21.15.49:0/3938141582 shutdown_connections 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.477+0000 7fa12ff1d700 1 -- 172.21.15.49:0/3938141582 wait complete. 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 Processor -- start 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 -- start start 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1281077f0 0x7fa128113dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa12810a160 0x7fa128114310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810cb30 0x7fa1281b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281173a0 con 0x7fa1281077f0 2023-10-08T09:24:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa128117510 con 0x7fa12810a160 2023-10-08T09:24:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12ff1d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1281b5ca0 con 0x7fa12810cb30 2023-10-08T09:24:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.478+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa12810a160 0x7fa128114310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.479+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa12810a160 0x7fa128114310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42440/0 (socket says 172.21.15.49:42440) 2023-10-08T09:24:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.479+0000 7fa12d4b8700 1 -- 172.21.15.49:0/1251313028 learned_addr learned my addr 172.21.15.49:0/1251313028 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.479+0000 7fa12e4ba700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810cb30 0x7fa1281b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.479+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1281077f0 0x7fa128113dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.481+0000 7fa12e4ba700 1 -- 172.21.15.49:0/1251313028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa12810a160 msgr2=0x7fa128114310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.481+0000 7fa12e4ba700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa12810a160 0x7fa128114310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.481+0000 7fa12e4ba700 1 -- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1281077f0 msgr2=0x7fa128113dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.481+0000 7fa12e4ba700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1281077f0 0x7fa128113dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.481+0000 7fa12e4ba700 1 -- 172.21.15.49:0/1251313028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180095f0 con 0x7fa12810cb30 2023-10-08T09:24:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.481+0000 7fa12e4ba700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810cb30 0x7fa1281b5760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa12000bb00 tx=0x7fa12000bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.481+0000 7fa11effd700 1 -- 172.21.15.49:0/1251313028 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000cae0 con 0x7fa12810cb30 2023-10-08T09:24:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.482+0000 7fa11effd700 1 -- 172.21.15.49:0/1251313028 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12000cc40 con 0x7fa12810cb30 2023-10-08T09:24:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.482+0000 7fa11effd700 1 -- 172.21.15.49:0/1251313028 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120012630 con 0x7fa12810cb30 2023-10-08T09:24:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.483+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1281b5ea0 con 0x7fa12810cb30 2023-10-08T09:24:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.483+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1281b6350 con 0x7fa12810cb30 2023-10-08T09:24:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 Processor -- start 2023-10-08T09:24:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 -- start start 2023-10-08T09:24:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e75255e40 0x558e75256260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75256cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75257730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e751be400 con 0x558e75255e40 2023-10-08T09:24:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e751afbc0 con 0x558e752567a0 2023-10-08T09:24:27.491 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.488+0000 7fa13c106b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e751ad4e0 con 0x558e752572b0 2023-10-08T09:24:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.493+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75257730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.493+0000 7fa12dcb9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75256cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.493+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75257730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39868/0 (socket says 172.21.15.49:39868) 2023-10-08T09:24:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.493+0000 7fa12d4b8700 1 -- 172.21.15.49:0/2086853067 learned_addr learned my addr 172.21.15.49:0/2086853067 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.494+0000 7fa11effd700 1 -- 172.21.15.49:0/1251313028 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa120014440 con 0x7fa12810cb30 2023-10-08T09:24:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.494+0000 7fa11effd700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1140a15d0 0x7fa1140a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.495+0000 7fa12d4b8700 1 -- 172.21.15.49:0/2086853067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 msgr2=0x558e75256cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.495+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/2086853067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75256cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.495+0000 7fa12d4b8700 1 -- 172.21.15.49:0/2086853067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e75255e40 msgr2=0x558e75256260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.495+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/2086853067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e75255e40 0x558e75256260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.495+0000 7fa12d4b8700 1 -- 172.21.15.49:0/2086853067 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e74fd3d80 con 0x558e752572b0 2023-10-08T09:24:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.495+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/2086853067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75257730 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa118009f40 tx=0x7fa118009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed3cefaac255b5d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.495+0000 7fa11effd700 1 -- 172.21.15.49:0/1251313028 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa1200cc4c0 con 0x7fa12810cb30 2023-10-08T09:24:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.497+0000 7fa113fff700 1 -- 172.21.15.49:0/2086853067 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800b9d0 con 0x558e752572b0 2023-10-08T09:24:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.497+0000 7fa113fff700 1 -- 172.21.15.49:0/2086853067 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa11801e030 con 0x558e752572b0 2023-10-08T09:24:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.497+0000 7fa113fff700 1 -- 172.21.15.49:0/2086853067 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800f460 con 0x558e752572b0 2023-10-08T09:24:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.498+0000 7fa13c106b80 1 -- 172.21.15.49:0/2086853067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 msgr2=0x558e75257730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.498+0000 7fa13c106b80 1 --2- 172.21.15.49:0/2086853067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75257730 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa118009f40 tx=0x7fa118009f70 comp rx=0 tx=0).stop 2023-10-08T09:24:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.500+0000 7fa12e4ba700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1140a15d0 0x7fa1140a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.500+0000 7fa13c106b80 1 -- 172.21.15.49:0/2086853067 shutdown_connections 2023-10-08T09:24:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.500+0000 7fa13c106b80 1 --2- 172.21.15.49:0/2086853067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75257730 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.500+0000 7fa13c106b80 1 --2- 172.21.15.49:0/2086853067 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e75255e40 0x558e75256260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.500+0000 7fa13c106b80 1 --2- 172.21.15.49:0/2086853067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75256cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.500+0000 7fa13c106b80 1 -- 172.21.15.49:0/2086853067 >> 172.21.15.49:0/2086853067 conn(0x558e751ca200 msgr2=0x558e752508a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.501+0000 7fa13c106b80 1 -- 172.21.15.49:0/2086853067 shutdown_connections 2023-10-08T09:24:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 -- 172.21.15.49:0/2086853067 wait complete. 2023-10-08T09:24:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa12e4ba700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1140a15d0 0x7fa1140a3a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c5e0 tx=0x7fa12000c550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 -- start start 2023-10-08T09:24:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 Processor -- start 2023-10-08T09:24:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e75255e40 0x558e75281700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75281c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e75288c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e751a8dd0 con 0x558e752572b0 2023-10-08T09:24:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e751456b0 con 0x558e752567a0 2023-10-08T09:24:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.502+0000 7fa13c106b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e75145350 con 0x558e75255e40 2023-10-08T09:24:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.503+0000 7fa13c106b80 10 client.0 _ll_register_callbacks cb 0x558e74fa27a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.503+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e75255e40 0x558e75281700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.503+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e75255e40 0x558e75281700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39874/0 (socket says 172.21.15.49:39874) 2023-10-08T09:24:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.503+0000 7fa12d4b8700 1 -- 172.21.15.49:0/1187596301 learned_addr learned my addr 172.21.15.49:0/1187596301 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.503+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e75288c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.503+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75281c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.504+0000 7fa12d4b8700 1 -- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 msgr2=0x558e75281c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.504+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75281c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.504+0000 7fa12d4b8700 1 -- 172.21.15.49:0/1187596301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 msgr2=0x558e75288c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.504+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e75288c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.504+0000 7fa12d4b8700 1 -- 172.21.15.49:0/1187596301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12810da70 con 0x558e75255e40 2023-10-08T09:24:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.517+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e75255e40 0x558e75281700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1180059d0 tx=0x7fa118009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cea48337eabfb2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.517+0000 7fa1117fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.517+0000 7fa1117fa700 1 -- 172.21.15.49:0/1187596301 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118003b60 con 0x558e75255e40 2023-10-08T09:24:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.517+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e75170e50 con 0x558e75255e40 2023-10-08T09:24:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.517+0000 7fa1117fa700 1 -- 172.21.15.49:0/1187596301 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa118003cc0 con 0x558e75255e40 2023-10-08T09:24:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.522+0000 7fa1117fa700 1 -- 172.21.15.49:0/1187596301 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa118003e80 con 0x558e75255e40 2023-10-08T09:24:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 10 client.26389 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558e74fbcc00 con 0x558e75255e40 2023-10-08T09:24:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa1117fa700 1 -- 172.21.15.49:0/1187596301 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e75170e50 con 0x558e75255e40 2023-10-08T09:24:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa1117fa700 1 -- 172.21.15.49:0/1187596301 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa118024540 con 0x558e75255e40 2023-10-08T09:24:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 10 client.26389 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 10 client.26389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 20 client.26389 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e752aef80 0x558e752b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 4 client.26389 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:24:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.523+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e751a1db0 con 0x558e752aef80 2023-10-08T09:24:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.524+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e752aef80 0x558e752b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.524+0000 7fa1117fa700 10 client.26389 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.524+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e752aef80 0x558e752b13a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b782543cbcda405 server_cookie=c44ff462a92e65d4 in_seq=0 out_seq=0 2023-10-08T09:24:27.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.526+0000 7fa1117fa700 1 -- 172.21.15.49:0/1187596301 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e751a1db0 con 0x558e752aef80 2023-10-08T09:24:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.526+0000 7fa1117fa700 10 client.26389 handle_command_reply: tid=0 2023-10-08T09:24:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.526+0000 7fa13c106b80 1 client.26389 shutdown 2023-10-08T09:24:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.528+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e75255e40 msgr2=0x558e75281700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.528+0000 7fa13c106b80 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e75255e40 0x558e75281700 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1180059d0 tx=0x7fa118009ca0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 shutdown_connections 2023-10-08T09:24:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e752aef80 0x558e752b13a0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e75255e40 0x558e75281700 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e75288c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 --2- 172.21.15.49:0/1187596301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e752567a0 0x558e75281c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 >> 172.21.15.49:0/1187596301 conn(0x558e751ca200 msgr2=0x558e75250010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa1117fa700 0 client.26389 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 shutdown_connections 2023-10-08T09:24:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.529+0000 7fa13c106b80 1 -- 172.21.15.49:0/1187596301 wait complete. 2023-10-08T09:24:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.531+0000 7fa13c106b80 20 client.26389 trim_cache size 0 max 16384 2023-10-08T09:24:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.552+0000 7fa13c106b80 1 Processor -- start 2023-10-08T09:24:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.552+0000 7fa13c106b80 1 -- start start 2023-10-08T09:24:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.552+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e75256220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.552+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752567a0 0x558e75288c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e7525a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa13c106b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e750298c0 con 0x558e752572b0 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa13c106b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e752802a0 con 0x558e75255e40 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa13c106b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e751a1db0 con 0x558e752567a0 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e75256220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12dcb9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752567a0 0x558e75288c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e75256220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42462/0 (socket says 172.21.15.49:42462) 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 -- 172.21.15.49:0/211916779 learned_addr learned my addr 172.21.15.49:0/211916779 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e7525a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 -- 172.21.15.49:0/211916779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752567a0 msgr2=0x558e75288c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752567a0 0x558e75288c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 -- 172.21.15.49:0/211916779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 msgr2=0x558e75256220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e75256220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 -- 172.21.15.49:0/211916779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e74ffa1f0 con 0x558e752572b0 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752567a0 0x558e75288c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e7525a210 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c010 tx=0x7fa118005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac92d0190301a3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.553+0000 7fa11cff9700 1 -- 172.21.15.49:0/211916779 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118024620 con 0x558e752572b0 2023-10-08T09:24:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa11cff9700 1 -- 172.21.15.49:0/211916779 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa118029040 con 0x558e752572b0 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa11cff9700 1 -- 172.21.15.49:0/211916779 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800f6c0 con 0x558e752572b0 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 -- 172.21.15.49:0/211916779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 msgr2=0x558e7525a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e7525a210 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa11800c010 tx=0x7fa118005960 comp rx=0 tx=0).stop 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 -- 172.21.15.49:0/211916779 shutdown_connections 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752572b0 0x558e7525a210 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e75256220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 --2- 172.21.15.49:0/211916779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752567a0 0x558e75288c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 -- 172.21.15.49:0/211916779 >> 172.21.15.49:0/211916779 conn(0x558e751ca200 msgr2=0x558e75280930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 -- 172.21.15.49:0/211916779 shutdown_connections 2023-10-08T09:24:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.554+0000 7fa13c106b80 1 -- 172.21.15.49:0/211916779 wait complete. 2023-10-08T09:24:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 -- start start 2023-10-08T09:24:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 Processor -- start 2023-10-08T09:24:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e752861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752567a0 0x558e75286730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75287600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e75287b40 con 0x558e752567a0 2023-10-08T09:24:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e75287cb0 con 0x558e75255e40 2023-10-08T09:24:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e75287e70 con 0x558e752572b0 2023-10-08T09:24:27.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.555+0000 7fa13c106b80 10 client.0 _ll_register_callbacks cb 0x7fa1240071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752567a0 0x558e75286730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752567a0 0x558e75286730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52858/0 (socket says 172.21.15.49:52858) 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 -- 172.21.15.49:0/3357844107 learned_addr learned my addr 172.21.15.49:0/3357844107 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 -- 172.21.15.49:0/3357844107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 msgr2=0x558e75287600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75287600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 -- 172.21.15.49:0/3357844107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 msgr2=0x558e752861f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e752861f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 -- 172.21.15.49:0/3357844107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12400e040 con 0x558e752567a0 2023-10-08T09:24:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa12dcb9700 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752567a0 0x558e75286730 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x558e752b1c60 tx=0x7fa124008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1b6271f74e1e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.557+0000 7fa1117fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa1117fa700 1 -- 172.21.15.49:0/3357844107 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12401bb30 con 0x558e752567a0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa1117fa700 1 -- 172.21.15.49:0/3357844107 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa12401bc90 con 0x558e752567a0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa11801e030 con 0x558e752567a0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa1117fa700 1 -- 172.21.15.49:0/3357844107 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa12401be50 con 0x558e752567a0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 10 client.16734 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558e74fbcc00 con 0x558e752567a0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa1117fa700 1 -- 172.21.15.49:0/3357844107 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa11801e030 con 0x558e752567a0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa1117fa700 1 -- 172.21.15.49:0/3357844107 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa124014610 con 0x558e752567a0 2023-10-08T09:24:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 10 client.16734 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 20 client.16734 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e753035b0 0x558e75305a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 4 client.16734 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.558+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558e751a1ad0 con 0x558e753035b0 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.559+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e753035b0 0x558e75305a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.559+0000 7fa12d4b8700 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e753035b0 0x558e75305a70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c83451d4a727cd server_cookie=7a4de06e345d3124 in_seq=0 out_seq=0 2023-10-08T09:24:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.559+0000 7fa1117fa700 10 client.16734 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.573+0000 7fa1117fa700 1 -- 172.21.15.49:0/3357844107 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558e751a1ad0 con 0x558e753035b0 2023-10-08T09:24:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.573+0000 7fa1117fa700 10 client.16734 handle_command_reply: tid=0 2023-10-08T09:24:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.573+0000 7fa13c106b80 1 client.16734 shutdown 2023-10-08T09:24:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.573+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752567a0 msgr2=0x558e75286730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.573+0000 7fa13c106b80 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752567a0 0x558e75286730 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x558e752b1c60 tx=0x7fa124008040 comp rx=0 tx=0).stop 2023-10-08T09:24:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.573+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 shutdown_connections 2023-10-08T09:24:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.573+0000 7fa13c106b80 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558e753035b0 0x558e75305a70 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa13c106b80 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558e752572b0 0x558e75287600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa13c106b80 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558e75255e40 0x558e752861f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa13c106b80 1 --2- 172.21.15.49:0/3357844107 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558e752567a0 0x558e75286730 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 >> 172.21.15.49:0/3357844107 conn(0x558e751ca200 msgr2=0x558e7525a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa1117fa700 0 client.16734 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 shutdown_connections 2023-10-08T09:24:27.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa13c106b80 1 -- 172.21.15.49:0/3357844107 wait complete. 2023-10-08T09:24:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.574+0000 7fa13c106b80 20 client.16734 trim_cache size 0 max 16384 2023-10-08T09:24:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:24:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:24:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:24:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:24:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1140a15d0 msgr2=0x7fa1140a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1140a15d0 0x7fa1140a3a90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c5e0 tx=0x7fa12000c550 comp rx=0 tx=0).stop 2023-10-08T09:24:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810cb30 msgr2=0x7fa1281b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810cb30 0x7fa1281b5760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa12000bb00 tx=0x7fa12000bec0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 shutdown_connections 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa1140a15d0 0x7fa1140a3a90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa12810cb30 0x7fa1281b5760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1281077f0 0x7fa128113dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 --2- 172.21.15.49:0/1251313028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa12810a160 0x7fa128114310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 >> 172.21.15.49:0/1251313028 conn(0x7fa12806d770 msgr2=0x7fa128108de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 shutdown_connections 2023-10-08T09:24:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:27.575+0000 7fa12ff1d700 1 -- 172.21.15.49:0/1251313028 wait complete. 2023-10-08T09:24:27.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:27 smithi184 ceph-mon[103675]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 604 op/s 2023-10-08T09:24:27.828 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:27 smithi132 ceph-mon[103389]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 604 op/s 2023-10-08T09:24:27.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:27 smithi049 ceph-mon[99485]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.7 MiB/s wr, 604 op/s 2023-10-08T09:24:28.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:28 smithi132 ceph-mon[103389]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 521 op/s 2023-10-08T09:24:28.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:28 smithi049 ceph-mon[99485]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 521 op/s 2023-10-08T09:24:28.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:28 smithi184 ceph-mon[103675]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.7 MiB/s wr, 521 op/s 2023-10-08T09:24:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 545.0 s, 634.8 tps, lat 159.968 ms stddev 187.930 2023-10-08T09:24:31.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:31 smithi184 ceph-mon[103675]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.3 MiB/s wr, 502 op/s 2023-10-08T09:24:31.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:31 smithi132 ceph-mon[103389]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.3 MiB/s wr, 502 op/s 2023-10-08T09:24:31.886 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:31 smithi049 ceph-mon[99485]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.3 MiB/s wr, 502 op/s 2023-10-08T09:24:33.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:32 smithi049 ceph-mon[99485]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.5 MiB/s wr, 553 op/s 2023-10-08T09:24:33.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:32 smithi184 ceph-mon[103675]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.5 MiB/s wr, 553 op/s 2023-10-08T09:24:33.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:32 smithi132 ceph-mon[103389]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.5 MiB/s wr, 553 op/s 2023-10-08T09:24:34.301 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:34 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:24:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 550.0 s, 509.1 tps, lat 192.439 ms stddev 224.081 2023-10-08T09:24:34.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:34 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:24:34.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:34 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:24:35.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:35 smithi132 ceph-mon[103389]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2023-10-08T09:24:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:35 smithi049 ceph-mon[99485]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2023-10-08T09:24:35.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:35 smithi184 ceph-mon[103675]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 5.6 MiB/s wr, 527 op/s 2023-10-08T09:24:36.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:24:36.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:24:36.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:36 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:24:36.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:24:36.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:24:36.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:36 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:24:36.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:24:36.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:24:36.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:36 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:24:37.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:37 smithi132 ceph-mon[103389]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 6.2 MiB/s wr, 557 op/s 2023-10-08T09:24:37.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:37 smithi049 ceph-mon[99485]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 6.2 MiB/s wr, 557 op/s 2023-10-08T09:24:37.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:37 smithi184 ceph-mon[103675]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 6.2 MiB/s wr, 557 op/s 2023-10-08T09:24:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 555.0 s, 532.7 tps, lat 186.727 ms stddev 211.763 2023-10-08T09:24:39.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:39 smithi184 ceph-mon[103675]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2023-10-08T09:24:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:39 smithi132 ceph-mon[103389]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2023-10-08T09:24:39.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:39 smithi049 ceph-mon[99485]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2023-10-08T09:24:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.804+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1156070931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010cb30 msgr2=0x7f3de010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.806 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.804+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1156070931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010cb30 0x7f3de010cfb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3dd801c0d0 tx=0x7f3dd801c3e0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.815+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1156070931 shutdown_connections 2023-10-08T09:24:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.815+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1156070931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3de01077f0 0x7f3de0109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.815+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1156070931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010cb30 0x7f3de010cfb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.815+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1156070931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de010a160 0x7f3de010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.815+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1156070931 >> 172.21.15.49:0/1156070931 conn(0x7f3de006d770 msgr2=0x7f3de006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.826+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1156070931 shutdown_connections 2023-10-08T09:24:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.828+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1156070931 wait complete. 2023-10-08T09:24:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.829+0000 7f3de7cbd700 1 Processor -- start 2023-10-08T09:24:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.830+0000 7f3de7cbd700 1 -- start start 2023-10-08T09:24:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.830+0000 7f3de7cbd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de01077f0 0x7f3de019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.830+0000 7f3de7cbd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010a160 0x7f3de019d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.830+0000 7f3de7cbd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3de010cb30 0x7f3de01c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.830+0000 7f3de7cbd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de01c6840 con 0x7f3de010a160 2023-10-08T09:24:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.830+0000 7f3de7cbd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de01c69b0 con 0x7f3de01077f0 2023-10-08T09:24:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.830+0000 7f3de7cbd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de01c6b70 con 0x7f3de010cb30 2023-10-08T09:24:40.833 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.831+0000 7f3de5258700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010a160 0x7f3de019d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.831+0000 7f3de5258700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010a160 0x7f3de019d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52872/0 (socket says 172.21.15.49:52872) 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.831+0000 7f3de5258700 1 -- 172.21.15.49:0/1021553331 learned_addr learned my addr 172.21.15.49:0/1021553331 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.832+0000 7f3de5258700 1 -- 172.21.15.49:0/1021553331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3de010cb30 msgr2=0x7f3de01c6270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.832+0000 7f3de5258700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3de010cb30 0x7f3de01c6270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.832+0000 7f3de5258700 1 -- 172.21.15.49:0/1021553331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de01077f0 msgr2=0x7f3de019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.832+0000 7f3de5258700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de01077f0 0x7f3de019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.832+0000 7f3de5258700 1 -- 172.21.15.49:0/1021553331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd801ad10 con 0x7f3de010a160 2023-10-08T09:24:40.834 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.832+0000 7f3de5258700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010a160 0x7f3de019d220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b530 tx=0x7f3ddc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.833+0000 7f3dd6ffd700 1 -- 172.21.15.49:0/1021553331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc011830 con 0x7f3de010a160 2023-10-08T09:24:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.833+0000 7f3dd6ffd700 1 -- 172.21.15.49:0/1021553331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ddc011e50 con 0x7f3de010a160 2023-10-08T09:24:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.833+0000 7f3dd6ffd700 1 -- 172.21.15.49:0/1021553331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00f2e0 con 0x7f3de010a160 2023-10-08T09:24:40.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.834+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de01c6e50 con 0x7f3de010a160 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.834+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de01c7280 con 0x7f3de010a160 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 Processor -- start 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 -- start start 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557466340e40 0x557466341260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574663417a0 0x557466341cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663422b0 0x557466342730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574662a9400 con 0x557466340e40 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746629abc0 con 0x5574663417a0 2023-10-08T09:24:40.841 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.839+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574662984e0 con 0x5574663422b0 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663422b0 0x557466342730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663422b0 0x557466342730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39910/0 (socket says 172.21.15.49:39910) 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 -- 172.21.15.49:0/922454747 learned_addr learned my addr 172.21.15.49:0/922454747 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 -- 172.21.15.49:0/922454747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574663417a0 msgr2=0x557466341cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 --2- 172.21.15.49:0/922454747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574663417a0 0x557466341cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 -- 172.21.15.49:0/922454747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557466340e40 msgr2=0x557466341260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 --2- 172.21.15.49:0/922454747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557466340e40 0x557466341260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.840+0000 7f3de5258700 1 -- 172.21.15.49:0/922454747 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574660bed80 con 0x5574663422b0 2023-10-08T09:24:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.841+0000 7f3de5258700 1 --2- 172.21.15.49:0/922454747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663422b0 0x557466342730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc010a20 tx=0x7f3ddc011c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5ca5231348e3089 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.843+0000 7f3dd6ffd700 1 -- 172.21.15.49:0/1021553331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3ddc024930 con 0x7f3de010a160 2023-10-08T09:24:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.843+0000 7f3dd6ffd700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3dcc0a1600 0x7f3dcc0a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.844+0000 7f3dd6ffd700 1 -- 172.21.15.49:0/1021553331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ddc0ce970 con 0x7f3de010a160 2023-10-08T09:24:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.844+0000 7f3dcbfff700 1 -- 172.21.15.49:0/922454747 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0d5330 con 0x5574663422b0 2023-10-08T09:24:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.844+0000 7f3dcbfff700 1 -- 172.21.15.49:0/922454747 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ddc0d3020 con 0x5574663422b0 2023-10-08T09:24:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.845+0000 7f3de625a700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3dcc0a1600 0x7f3dcc0a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.848 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.845+0000 7f3de625a700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3dcc0a1600 0x7f3dcc0a3ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3dd801e010 tx=0x7f3dd801c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.850+0000 7f3dcbfff700 1 -- 172.21.15.49:0/922454747 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0d6bb0 con 0x5574663422b0 2023-10-08T09:24:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.851+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/922454747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663422b0 msgr2=0x557466342730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.851+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/922454747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663422b0 0x557466342730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc010a20 tx=0x7f3ddc011c70 comp rx=0 tx=0).stop 2023-10-08T09:24:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.860+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/922454747 shutdown_connections 2023-10-08T09:24:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.860+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/922454747 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663422b0 0x557466342730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.860+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/922454747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557466340e40 0x557466341260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.860+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/922454747 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5574663417a0 0x557466341cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.860+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/922454747 >> 172.21.15.49:0/922454747 conn(0x5574662b5200 msgr2=0x55746633b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.865+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/922454747 shutdown_connections 2023-10-08T09:24:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.865+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/922454747 wait complete. 2023-10-08T09:24:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.865+0000 7f3df3ea6b80 1 -- start start 2023-10-08T09:24:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.865+0000 7f3df3ea6b80 1 Processor -- start 2023-10-08T09:24:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.866+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.866+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.866+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466351da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.866+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557466293dd0 con 0x5574663422b0 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.866+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574662306b0 con 0x557466340e40 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.866+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557466230350 con 0x5574663417a0 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.866+0000 7f3df3ea6b80 10 client.0 _ll_register_callbacks cb 0x55746608d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.871+0000 7f3de5a59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.872+0000 7f3de5a59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42496/0 (socket says 172.21.15.49:42496) 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.872+0000 7f3de5a59700 1 -- 172.21.15.49:0/3433031320 learned_addr learned my addr 172.21.15.49:0/3433031320 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.873+0000 7f3de5a59700 1 -- 172.21.15.49:0/3433031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 msgr2=0x55746634ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.873+0000 7f3de5a59700 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.873+0000 7f3de5a59700 1 -- 172.21.15.49:0/3433031320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 msgr2=0x557466351da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.873+0000 7f3de5a59700 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466351da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.873+0000 7f3de5a59700 1 -- 172.21.15.49:0/3433031320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de01c7280 con 0x557466340e40 2023-10-08T09:24:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.873+0000 7f3de5a59700 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634a820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009f00 tx=0x7f3dd000b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=934911f4f751f271 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.873+0000 7f3dc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:24:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.874+0000 7f3dc97fa700 1 -- 172.21.15.49:0/3433031320 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0011420 con 0x557466340e40 2023-10-08T09:24:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.874+0000 7f3dc97fa700 1 -- 172.21.15.49:0/3433031320 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3dd0011580 con 0x557466340e40 2023-10-08T09:24:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.874+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55746625be50 con 0x557466340e40 2023-10-08T09:24:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.879+0000 7f3dc97fa700 1 -- 172.21.15.49:0/3433031320 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd0011740 con 0x557466340e40 2023-10-08T09:24:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.879+0000 7f3df3ea6b80 10 client.26336 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.879+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5574660a7c00 con 0x557466340e40 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3dc97fa700 1 -- 172.21.15.49:0/3433031320 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55746625be50 con 0x557466340e40 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3dc97fa700 1 -- 172.21.15.49:0/3433031320 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3dd001aad0 con 0x557466340e40 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3df3ea6b80 10 client.26336 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3df3ea6b80 10 client.26336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3df3ea6b80 20 client.26336 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557466356730 0x557466358b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3df3ea6b80 4 client.26336 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:24:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.883+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55746628cdb0 con 0x557466356730 2023-10-08T09:24:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.888+0000 7f3de5258700 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557466356730 0x557466358b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.894+0000 7f3de5258700 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557466356730 0x557466358b50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f38a81864677f09 server_cookie=2c00149c42927986 in_seq=0 out_seq=0 2023-10-08T09:24:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.894+0000 7f3dc97fa700 10 client.26336 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.895+0000 7f3dc97fa700 1 -- 172.21.15.49:0/3433031320 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55746628cdb0 con 0x557466356730 2023-10-08T09:24:40.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.895+0000 7f3dc97fa700 10 client.26336 handle_command_reply: tid=0 2023-10-08T09:24:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 client.26336 shutdown 2023-10-08T09:24:40.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 msgr2=0x55746634a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634a820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009f00 tx=0x7f3dd000b3c0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 shutdown_connections 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557466356730 0x557466358b50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466351da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634a820 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3433031320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 >> 172.21.15.49:0/3433031320 conn(0x5574662b5200 msgr2=0x55746633b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3dc97fa700 0 client.26336 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 shutdown_connections 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3433031320 wait complete. 2023-10-08T09:24:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.896+0000 7f3df3ea6b80 20 client.26336 trim_cache size 0 max 16384 2023-10-08T09:24:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 Processor -- start 2023-10-08T09:24:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 -- start start 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x557466352450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x557466351cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466349560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574661148c0 con 0x5574663422b0 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557466353b30 con 0x557466340e40 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55746633c6e0 con 0x5574663417a0 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3de625a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466349560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3de5258700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x557466351cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3de5a59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x557466352450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3de5258700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x557466351cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39922/0 (socket says 172.21.15.49:39922) 2023-10-08T09:24:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3de625a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466349560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52892/0 (socket says 172.21.15.49:52892) 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3de5a59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x557466352450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42502/0 (socket says 172.21.15.49:42502) 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.921+0000 7f3de5258700 1 -- 172.21.15.49:0/3456155129 learned_addr learned my addr 172.21.15.49:0/3456155129 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3de5258700 1 -- 172.21.15.49:0/3456155129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 msgr2=0x557466352450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3de5258700 1 --2- 172.21.15.49:0/3456155129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x557466352450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3de5258700 1 -- 172.21.15.49:0/3456155129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 msgr2=0x557466349560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3de5258700 1 --2- 172.21.15.49:0/3456155129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466349560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3de5258700 1 -- 172.21.15.49:0/3456155129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574660e51f0 con 0x5574663417a0 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3de5258700 1 --2- 172.21.15.49:0/3456155129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x557466351cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0048a0 tx=0x7f3ddc097e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836f3e166aef86b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3dd4d82700 1 -- 172.21.15.49:0/3456155129 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0d2040 con 0x5574663417a0 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3dd4d82700 1 -- 172.21.15.49:0/3456155129 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ddc09b460 con 0x5574663417a0 2023-10-08T09:24:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.922+0000 7f3dd4d82700 1 -- 172.21.15.49:0/3456155129 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0a3e90 con 0x5574663417a0 2023-10-08T09:24:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3456155129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 msgr2=0x557466351cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3456155129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x557466351cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0048a0 tx=0x7f3ddc097e40 comp rx=0 tx=0).stop 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3456155129 shutdown_connections 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3456155129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466349560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3456155129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x557466352450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/3456155129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x557466351cc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3456155129 >> 172.21.15.49:0/3456155129 conn(0x5574662b5200 msgr2=0x55746635d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3456155129 shutdown_connections 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/3456155129 wait complete. 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 -- start start 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 Processor -- start 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.923+0000 7f3df3ea6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466350a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557466350f60 con 0x5574663422b0 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574663510d0 con 0x557466340e40 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3df3ea6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557466351290 con 0x5574663417a0 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3df3ea6b80 10 client.0 _ll_register_callbacks cb 0x7f3dd0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5a59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39930/0 (socket says 172.21.15.49:39930) 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5a59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42508/0 (socket says 172.21.15.49:42508) 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 -- 172.21.15.49:0/1534111792 learned_addr learned my addr 172.21.15.49:0/1534111792 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de625a700 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466350a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 -- 172.21.15.49:0/1534111792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 msgr2=0x55746634d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 -- 172.21.15.49:0/1534111792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 msgr2=0x557466350a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466350a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 -- 172.21.15.49:0/1534111792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574660bed80 con 0x5574663417a0 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de625a700 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466350a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3de5258700 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634da00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc006f60 tx=0x7f3ddc010450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52a9a594b4f8375 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3dc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3dc97fa700 1 -- 172.21.15.49:0/1534111792 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0d2050 con 0x5574663417a0 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ddc0d3020 con 0x5574663417a0 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.924+0000 7f3dc97fa700 1 -- 172.21.15.49:0/1534111792 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ddc0d4030 con 0x5574663417a0 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3dc97fa700 1 -- 172.21.15.49:0/1534111792 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ddc09bb00 con 0x5574663417a0 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 10 client.26401 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:40.931 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5574660bd250 con 0x5574663417a0 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3dc97fa700 1 -- 172.21.15.49:0/1534111792 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ddc0d3020 con 0x5574663417a0 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3dc97fa700 1 -- 172.21.15.49:0/1534111792 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3ddc098210 con 0x5574663417a0 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 10 client.26401 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 10 client.26401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 20 client.26401 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5574663aaa40 0x5574663ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 4 client.26401 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55746628cad0 con 0x5574663aaa40 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.925+0000 7f3de5a59700 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5574663aaa40 0x5574663ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.926+0000 7f3dc97fa700 10 client.26401 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:40.932 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.926+0000 7f3de5a59700 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5574663aaa40 0x5574663ace60 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45abeaff1d2eb3f server_cookie=1765281286bf667f in_seq=0 out_seq=0 2023-10-08T09:24:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.937+0000 7f3dc97fa700 1 -- 172.21.15.49:0/1534111792 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55746628cad0 con 0x5574663aaa40 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.937+0000 7f3dc97fa700 10 client.26401 handle_command_reply: tid=0 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.937+0000 7f3df3ea6b80 1 client.26401 shutdown 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.937+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 msgr2=0x55746634da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.937+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634da00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc006f60 tx=0x7f3ddc010450 comp rx=0 tx=0).stop 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 shutdown_connections 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5574663aaa40 0x5574663ace60 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5574663422b0 0x557466350a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3dc97fa700 0 client.26401 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557466340e40 0x55746634d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 --2- 172.21.15.49:0/1534111792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5574663417a0 0x55746634da00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 >> 172.21.15.49:0/1534111792 conn(0x5574662b5200 msgr2=0x557466349560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 shutdown_connections 2023-10-08T09:24:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 1 -- 172.21.15.49:0/1534111792 wait complete. 2023-10-08T09:24:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3df3ea6b80 20 client.26401 trim_cache size 0 max 16384 2023-10-08T09:24:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:24:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:24:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:24:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:24:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3dcc0a1600 msgr2=0x7f3dcc0a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3dcc0a1600 0x7f3dcc0a3ac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3dd801e010 tx=0x7f3dd801c040 comp rx=0 tx=0).stop 2023-10-08T09:24:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010a160 msgr2=0x7f3de019d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010a160 0x7f3de019d220 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b530 tx=0x7f3ddc00b8f0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 shutdown_connections 2023-10-08T09:24:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3dcc0a1600 0x7f3dcc0a3ac0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3de010cb30 0x7f3de01c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3de01077f0 0x7f3de019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 --2- 172.21.15.49:0/1021553331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3de010a160 0x7f3de019d220 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 >> 172.21.15.49:0/1021553331 conn(0x7f3de006d770 msgr2=0x7f3de010a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 shutdown_connections 2023-10-08T09:24:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:40.938+0000 7f3de7cbd700 1 -- 172.21.15.49:0/1021553331 wait complete. 2023-10-08T09:24:41.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:41 smithi184 ceph-mon[103675]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2023-10-08T09:24:41.804 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:41 smithi132 ceph-mon[103389]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2023-10-08T09:24:41.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:41 smithi049 ceph-mon[99485]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2023-10-08T09:24:43.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:42 smithi049 ceph-mon[99485]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2023-10-08T09:24:43.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:42 smithi184 ceph-mon[103675]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2023-10-08T09:24:43.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:42 smithi132 ceph-mon[103389]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2023-10-08T09:24:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 560.0 s, 501.2 tps, lat 206.938 ms stddev 239.718 2023-10-08T09:24:44.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:44 smithi132 ceph-mon[103389]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.6 MiB/s wr, 503 op/s 2023-10-08T09:24:44.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:44 smithi049 ceph-mon[99485]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.6 MiB/s wr, 503 op/s 2023-10-08T09:24:44.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:44 smithi184 ceph-mon[103675]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.6 MiB/s wr, 503 op/s 2023-10-08T09:24:47.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:47 smithi184 ceph-mon[103675]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 554 op/s 2023-10-08T09:24:47.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:47 smithi132 ceph-mon[103389]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 554 op/s 2023-10-08T09:24:47.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:47 smithi049 ceph-mon[99485]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 554 op/s 2023-10-08T09:24:47.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:24:48.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.150+0000 7f71ffc95700 1 -- 172.21.15.49:0/4110517691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f8100180 msgr2=0x7f71f8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.150+0000 7f71ffc95700 1 --2- 172.21.15.49:0/4110517691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f8100180 0x7f71f8104a20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009990 tx=0x7f71f4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.150+0000 7f71ffc95700 1 -- 172.21.15.49:0/4110517691 shutdown_connections 2023-10-08T09:24:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.150+0000 7f71ffc95700 1 --2- 172.21.15.49:0/4110517691 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f8100180 0x7f71f8104a20 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.150+0000 7f71ffc95700 1 --2- 172.21.15.49:0/4110517691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f80fea60 0x7f71f80fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.150+0000 7f71ffc95700 1 --2- 172.21.15.49:0/4110517691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f80ff7c0 0x7f71f80ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.150+0000 7f71ffc95700 1 -- 172.21.15.49:0/4110517691 >> 172.21.15.49:0/4110517691 conn(0x7f71f80fa390 msgr2=0x7f71f80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:48.168 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.164+0000 7f71ffc95700 1 -- 172.21.15.49:0/4110517691 shutdown_connections 2023-10-08T09:24:48.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.183+0000 7f71ffc95700 1 -- 172.21.15.49:0/4110517691 wait complete. 2023-10-08T09:24:48.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.183+0000 7f71ffc95700 1 Processor -- start 2023-10-08T09:24:48.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71ffc95700 1 -- start start 2023-10-08T09:24:48.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71ffc95700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f80fea60 0x7f71f810ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71ffc95700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f8100180 0x7f71f810b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71ffc95700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f8199f60 0x7f71f819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71ffc95700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f810b970 con 0x7f71f80fea60 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71ffc95700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f810bae0 con 0x7f71f8100180 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71ffc95700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f819c960 con 0x7f71f8199f60 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71fe232700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f8199f60 0x7f71f819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71fe232700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f8199f60 0x7f71f819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39940/0 (socket says 172.21.15.49:39940) 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.184+0000 7f71fe232700 1 -- 172.21.15.49:0/1686265129 learned_addr learned my addr 172.21.15.49:0/1686265129 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.185+0000 7f71fe232700 1 -- 172.21.15.49:0/1686265129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f8100180 msgr2=0x7f71f810b370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.185+0000 7f71fe232700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f8100180 0x7f71f810b370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.185+0000 7f71fe232700 1 -- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f80fea60 msgr2=0x7f71f810ae30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.185+0000 7f71fe232700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f80fea60 0x7f71f810ae30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.185+0000 7f71fe232700 1 -- 172.21.15.49:0/1686265129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f40095f0 con 0x7f71f8199f60 2023-10-08T09:24:48.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.185+0000 7f71fe232700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f8199f60 0x7f71f819c420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71f40092e0 tx=0x7f71f4003d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:48.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.192+0000 7f71eeffd700 1 -- 172.21.15.49:0/1686265129 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f401d070 con 0x7f71f8199f60 2023-10-08T09:24:48.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.192+0000 7f71eeffd700 1 -- 172.21.15.49:0/1686265129 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71f400f460 con 0x7f71f8199f60 2023-10-08T09:24:48.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.192+0000 7f71eeffd700 1 -- 172.21.15.49:0/1686265129 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4022680 con 0x7f71f8199f60 2023-10-08T09:24:48.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.192+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f819cbe0 con 0x7f71f8199f60 2023-10-08T09:24:48.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.192+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f819d040 con 0x7f71f8199f60 2023-10-08T09:24:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.207+0000 7f720be7eb80 1 Processor -- start 2023-10-08T09:24:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.208+0000 7f720be7eb80 1 -- start start 2023-10-08T09:24:48.213 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.208+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a60070 0x564852a60490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.208+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.208+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a61810 0x564852a61ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.208+0000 7f720be7eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648529b7460 con 0x564852a60070 2023-10-08T09:24:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.208+0000 7f720be7eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648529a8c20 con 0x564852a61810 2023-10-08T09:24:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.208+0000 7f720be7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648529a6540 con 0x564852a609d0 2023-10-08T09:24:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.213+0000 7f71fd230700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a61810 0x564852a61ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.214 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.213+0000 7f71fd230700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a61810 0x564852a61ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42526/0 (socket says 172.21.15.49:42526) 2023-10-08T09:24:48.215 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.213+0000 7f71fd230700 1 -- 172.21.15.49:0/789306411 learned_addr learned my addr 172.21.15.49:0/789306411 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:48.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.224+0000 7f71fd230700 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a60070 0x564852a60490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.228 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.227+0000 7f71eeffd700 1 -- 172.21.15.49:0/1686265129 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f71f40052d0 con 0x7f71f8199f60 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.227+0000 7f71eeffd700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f71e40a1650 0x7f71e40a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.228+0000 7f71fda31700 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.228+0000 7f71eeffd700 1 -- 172.21.15.49:0/1686265129 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f71f40cd2c0 con 0x7f71f8199f60 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.228+0000 7f71fd230700 1 -- 172.21.15.49:0/789306411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 msgr2=0x564852a61280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.228+0000 7f71fd230700 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a61280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.228+0000 7f71fd230700 1 -- 172.21.15.49:0/789306411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a60070 msgr2=0x564852a60490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.228+0000 7f71fd230700 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a60070 0x564852a60490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.229 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.228+0000 7f71fd230700 1 -- 172.21.15.49:0/789306411 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648527ccde0 con 0x564852a61810 2023-10-08T09:24:48.239 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.239+0000 7f71fd230700 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a61810 0x564852a61ca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f71e800bf70 tx=0x7f71e800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd5e8ccddf65911b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:48.246 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.246+0000 7f71fe232700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f71e40a1650 0x7f71e40a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.250 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.250+0000 7f71e3fff700 1 -- 172.21.15.49:0/789306411 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e800d9a0 con 0x564852a61810 2023-10-08T09:24:48.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.250+0000 7f71e3fff700 1 -- 172.21.15.49:0/789306411 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e80060d0 con 0x564852a61810 2023-10-08T09:24:48.251 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.250+0000 7f71e3fff700 1 -- 172.21.15.49:0/789306411 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004aa0 con 0x564852a61810 2023-10-08T09:24:48.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.259+0000 7f720be7eb80 1 -- 172.21.15.49:0/789306411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a61810 msgr2=0x564852a61ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:48.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.259+0000 7f720be7eb80 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a61810 0x564852a61ca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f71e800bf70 tx=0x7f71e800b260 comp rx=0 tx=0).stop 2023-10-08T09:24:48.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.272+0000 7f720be7eb80 1 -- 172.21.15.49:0/789306411 shutdown_connections 2023-10-08T09:24:48.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.272+0000 7f720be7eb80 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a61810 0x564852a61ca0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.272+0000 7f720be7eb80 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a60070 0x564852a60490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.272+0000 7f720be7eb80 1 --2- 172.21.15.49:0/789306411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a61280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.272+0000 7f720be7eb80 1 -- 172.21.15.49:0/789306411 >> 172.21.15.49:0/789306411 conn(0x5648529c3260 msgr2=0x564852a49e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:48.276 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.276+0000 7f71fe232700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f71e40a1650 0x7f71e40a3b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f71f4096de0 tx=0x7f71f4009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:48.287 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.281+0000 7f720be7eb80 1 -- 172.21.15.49:0/789306411 shutdown_connections 2023-10-08T09:24:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.287+0000 7f720be7eb80 1 -- 172.21.15.49:0/789306411 wait complete. 2023-10-08T09:24:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.287+0000 7f720be7eb80 1 -- start start 2023-10-08T09:24:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.287+0000 7f720be7eb80 1 Processor -- start 2023-10-08T09:24:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.288+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a69eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.288+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a609d0 0x564852a6a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.288 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.288+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a61810 0x564852a71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.288+0000 7f720be7eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648529a1e30 con 0x564852a609d0 2023-10-08T09:24:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.288+0000 7f720be7eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485293e710 con 0x564852a60070 2023-10-08T09:24:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.288+0000 7f720be7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485293e3b0 con 0x564852a61810 2023-10-08T09:24:48.289 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.288+0000 7f720be7eb80 10 client.0 _ll_register_callbacks cb 0x56485279b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a61810 0x564852a71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a61810 0x564852a71430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39954/0 (socket says 172.21.15.49:39954) 2023-10-08T09:24:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 -- 172.21.15.49:0/3040376874 learned_addr learned my addr 172.21.15.49:0/3040376874 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a69eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 -- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 msgr2=0x564852a69eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a69eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 -- 172.21.15.49:0/3040376874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a609d0 msgr2=0x564852a6a3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:24:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a609d0 0x564852a6a3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.292+0000 7f71fd230700 1 -- 172.21.15.49:0/3040376874 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e8010040 con 0x564852a61810 2023-10-08T09:24:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.293+0000 7f71fd230700 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a61810 0x564852a71430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d0b0 tx=0x7f71e80066d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c5e6ccee4ff952 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:48.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.297+0000 7f71e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:48.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.297+0000 7f71e17fa700 1 -- 172.21.15.49:0/3040376874 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e801c070 con 0x564852a61810 2023-10-08T09:24:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.297+0000 7f71e17fa700 1 -- 172.21.15.49:0/3040376874 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f71e800ddc0 con 0x564852a61810 2023-10-08T09:24:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.297+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564852969eb0 con 0x564852a61810 2023-10-08T09:24:48.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.308+0000 7f71e17fa700 1 -- 172.21.15.49:0/3040376874 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e8007b50 con 0x564852a61810 2023-10-08T09:24:48.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.308+0000 7f720be7eb80 10 client.26410 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:48.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.308+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5648527b5c60 con 0x564852a61810 2023-10-08T09:24:48.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.317+0000 7f71e17fa700 1 -- 172.21.15.49:0/3040376874 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e802b030 con 0x564852a61810 2023-10-08T09:24:48.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.317+0000 7f71e17fa700 1 -- 172.21.15.49:0/3040376874 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71e80036f0 con 0x564852a61810 2023-10-08T09:24:48.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.318+0000 7f720be7eb80 10 client.26410 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:48.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.318+0000 7f720be7eb80 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:48.318 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.318+0000 7f720be7eb80 20 client.26410 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:48.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.318+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852a75b80 0x564852a77fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.318+0000 7f720be7eb80 4 client.26410 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:24:48.319 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.318+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56485299ae10 con 0x564852a75b80 2023-10-08T09:24:48.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.323+0000 7f71fda31700 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852a75b80 0x564852a77fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.336+0000 7f71fda31700 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852a75b80 0x564852a77fa0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49d0d00f31101ed server_cookie=a2ee2c5af98e179f in_seq=0 out_seq=0 2023-10-08T09:24:48.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.342+0000 7f71e17fa700 10 client.26410 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:48.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.344+0000 7f71e17fa700 1 -- 172.21.15.49:0/3040376874 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56485299ae10 con 0x564852a75b80 2023-10-08T09:24:48.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.344+0000 7f71e17fa700 10 client.26410 handle_command_reply: tid=0 2023-10-08T09:24:48.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.344+0000 7f720be7eb80 1 client.26410 shutdown 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a61810 msgr2=0x564852a71430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a61810 0x564852a71430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d0b0 tx=0x7f71e80066d0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 shutdown_connections 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852a75b80 0x564852a77fa0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a61810 0x564852a71430 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71e800d0b0 tx=0x7f71e80066d0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a69eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3040376874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a609d0 0x564852a6a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 >> 172.21.15.49:0/3040376874 conn(0x5648529c3260 msgr2=0x564852a494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:48.347 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.346+0000 7f71e17fa700 0 client.26410 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:48.355 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.354+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 shutdown_connections 2023-10-08T09:24:48.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.356+0000 7f720be7eb80 1 -- 172.21.15.49:0/3040376874 wait complete. 2023-10-08T09:24:48.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.356+0000 7f720be7eb80 20 client.26410 trim_cache size 0 max 16384 2023-10-08T09:24:48.389 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 Processor -- start 2023-10-08T09:24:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 -- start start 2023-10-08T09:24:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a64800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a64d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564852822920 con 0x564852a7c900 2023-10-08T09:24:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564852a725b0 con 0x564852a60070 2023-10-08T09:24:48.390 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.389+0000 7f720be7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564852a69a20 con 0x564852a609d0 2023-10-08T09:24:48.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a64800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fda31700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a64800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42538/0 (socket says 172.21.15.49:42538) 2023-10-08T09:24:48.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 -- 172.21.15.49:0/3115304400 learned_addr learned my addr 172.21.15.49:0/3115304400 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:48.398 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a64d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 -- 172.21.15.49:0/3115304400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 msgr2=0x564852a713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:48.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 -- 172.21.15.49:0/3115304400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 msgr2=0x564852a64d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:48.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a64d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.399 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 -- 172.21.15.49:0/3115304400 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648527f3250 con 0x564852a60070 2023-10-08T09:24:48.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.398+0000 7f71fd230700 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a64800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71e8004780 tx=0x7f71e80066d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b342fe9b19b059c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:48.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.405+0000 7f71ecd82700 1 -- 172.21.15.49:0/3115304400 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e801c040 con 0x564852a60070 2023-10-08T09:24:48.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.405+0000 7f71ecd82700 1 -- 172.21.15.49:0/3115304400 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e802b070 con 0x564852a60070 2023-10-08T09:24:48.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.405+0000 7f71ecd82700 1 -- 172.21.15.49:0/3115304400 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8006740 con 0x564852a60070 2023-10-08T09:24:48.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.406+0000 7f720be7eb80 1 -- 172.21.15.49:0/3115304400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 msgr2=0x564852a64800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:48.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.406+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a64800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71e8004780 tx=0x7f71e80066d0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.406+0000 7f720be7eb80 1 -- 172.21.15.49:0/3115304400 shutdown_connections 2023-10-08T09:24:48.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.406+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a64d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.406+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a64800 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.406+0000 7f720be7eb80 1 --2- 172.21.15.49:0/3115304400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.406+0000 7f720be7eb80 1 -- 172.21.15.49:0/3115304400 >> 172.21.15.49:0/3115304400 conn(0x5648529c3260 msgr2=0x564852a69090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:48.432 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.430+0000 7f720be7eb80 1 -- 172.21.15.49:0/3115304400 shutdown_connections 2023-10-08T09:24:48.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.436+0000 7f720be7eb80 1 -- 172.21.15.49:0/3115304400 wait complete. 2023-10-08T09:24:48.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.436+0000 7f720be7eb80 1 -- start start 2023-10-08T09:24:48.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.436+0000 7f720be7eb80 1 Processor -- start 2023-10-08T09:24:48.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a6fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f720be7eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a70650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f720be7eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564852a71760 con 0x564852a7c900 2023-10-08T09:24:48.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f720be7eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564852ac99a0 con 0x564852a60070 2023-10-08T09:24:48.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f720be7eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564852ac9ae0 con 0x564852a609d0 2023-10-08T09:24:48.438 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f720be7eb80 10 client.0 _ll_register_callbacks cb 0x7f71e8006500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:48.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f71fda31700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f71fda31700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a70110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39964/0 (socket says 172.21.15.49:39964) 2023-10-08T09:24:48.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f71fda31700 1 -- 172.21.15.49:0/1258309738 learned_addr learned my addr 172.21.15.49:0/1258309738 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:48.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.437+0000 7f71fd230700 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a70650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.439+0000 7f71fda31700 1 -- 172.21.15.49:0/1258309738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 msgr2=0x564852a6fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.439+0000 7f71fda31700 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a6fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.439+0000 7f71fda31700 1 -- 172.21.15.49:0/1258309738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 msgr2=0x564852a70650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.439+0000 7f71fda31700 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a70650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.439+0000 7f71fda31700 1 -- 172.21.15.49:0/1258309738 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648527b5c60 con 0x564852a609d0 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.440+0000 7f71fda31700 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a70110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x564852a78880 tx=0x7f71f0016970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ac9f7f6384644e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.440+0000 7f71e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.441+0000 7f71e17fa700 1 -- 172.21.15.49:0/1258309738 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011040 con 0x564852a609d0 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.442+0000 7f71e17fa700 1 -- 172.21.15.49:0/1258309738 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f71f0006e40 con 0x564852a609d0 2023-10-08T09:24:48.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.442+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e80060d0 con 0x564852a609d0 2023-10-08T09:24:48.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.453+0000 7f71e17fa700 1 -- 172.21.15.49:0/1258309738 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f000a600 con 0x564852a609d0 2023-10-08T09:24:48.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.458+0000 7f720be7eb80 10 client.26413 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:48.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.458+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5648527cb2b0 con 0x564852a609d0 2023-10-08T09:24:48.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f71e17fa700 1 -- 172.21.15.49:0/1258309738 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71e80060d0 con 0x564852a609d0 2023-10-08T09:24:48.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f71e17fa700 1 -- 172.21.15.49:0/1258309738 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f71f001f870 con 0x564852a609d0 2023-10-08T09:24:48.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f720be7eb80 10 client.26413 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:48.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f720be7eb80 10 client.26413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:48.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f720be7eb80 20 client.26413 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:48.463 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f720be7eb80 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852acb670 0x564852acda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:48.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f720be7eb80 4 client.26413 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:24:48.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.462+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56485299ab30 con 0x564852acb670 2023-10-08T09:24:48.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.463+0000 7f71fd230700 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852acb670 0x564852acda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:48.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.463+0000 7f71e17fa700 10 client.26413 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:48.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.463+0000 7f71fd230700 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852acb670 0x564852acda90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748fa816348bd9d0 server_cookie=4fa741919ce7c0dc in_seq=0 out_seq=0 2023-10-08T09:24:48.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.463+0000 7f71e17fa700 1 -- 172.21.15.49:0/1258309738 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56485299ab30 con 0x564852acb670 2023-10-08T09:24:48.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.463+0000 7f71e17fa700 10 client.26413 handle_command_reply: tid=0 2023-10-08T09:24:48.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.463+0000 7f720be7eb80 1 client.26413 shutdown 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 msgr2=0x564852a70110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a70110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x564852a78880 tx=0x7f71f0016970 comp rx=0 tx=0).stop 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 shutdown_connections 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564852acb670 0x564852acda90 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564852a7c900 0x564852a70650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f71e17fa700 0 client.26413 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564852a60070 0x564852a6fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 --2- 172.21.15.49:0/1258309738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564852a609d0 0x564852a70110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 >> 172.21.15.49:0/1258309738 conn(0x5648529c3260 msgr2=0x564852a64800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:48.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 shutdown_connections 2023-10-08T09:24:48.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 1 -- 172.21.15.49:0/1258309738 wait complete. 2023-10-08T09:24:48.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f720be7eb80 20 client.26413 trim_cache size 0 max 16384 2023-10-08T09:24:48.467 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:24:48.467 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:24:48.467 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:24:48.467 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:24:48.467 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:24:48.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:24:48.468 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:24:48.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:24:48.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:24:48.468 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f71e40a1650 msgr2=0x7f71e40a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f71ffc95700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f71e40a1650 0x7f71e40a3b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f71f4096de0 tx=0x7f71f4009ca0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f8199f60 msgr2=0x7f71f819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.464+0000 7f71ffc95700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f8199f60 0x7f71f819c420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71f40092e0 tx=0x7f71f4003d80 comp rx=0 tx=0).stop 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 shutdown_connections 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f71e40a1650 0x7f71e40a3b10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71f8199f60 0x7f71f819c420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.469 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71f80fea60 0x7f71f810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 --2- 172.21.15.49:0/1686265129 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f71f8100180 0x7f71f810b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:48.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 >> 172.21.15.49:0/1686265129 conn(0x7f71f80fa390 msgr2=0x7f71f8100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:48.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 shutdown_connections 2023-10-08T09:24:48.470 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:24:48.465+0000 7f71ffc95700 1 -- 172.21.15.49:0/1686265129 wait complete. 2023-10-08T09:24:48.484 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:24:48.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:48 smithi184 ceph-mon[103675]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 522 op/s 2023-10-08T09:24:48.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:48 smithi132 ceph-mon[103389]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 522 op/s 2023-10-08T09:24:48.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:48 smithi049 ceph-mon[99485]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 522 op/s 2023-10-08T09:24:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 565.0 s, 539.2 tps, lat 182.766 ms stddev 188.857 2023-10-08T09:24:51.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:51 smithi184 ceph-mon[103675]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 511 op/s 2023-10-08T09:24:51.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:51 smithi132 ceph-mon[103389]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 511 op/s 2023-10-08T09:24:51.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:51 smithi049 ceph-mon[99485]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 511 op/s 2023-10-08T09:24:53.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:52 smithi049 ceph-mon[99485]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2023-10-08T09:24:53.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:52 smithi184 ceph-mon[103675]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2023-10-08T09:24:53.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:52 smithi132 ceph-mon[103389]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.5 MiB/s wr, 574 op/s 2023-10-08T09:24:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.210+0000 7f1f58819700 1 -- 172.21.15.49:0/3528394131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f54075510 msgr2=0x7f1f540779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.210+0000 7f1f58819700 1 --2- 172.21.15.49:0/3528394131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f54075510 0x7f1f540779a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005ce0 tx=0x7f1f44009a60 comp rx=0 tx=0).stop 2023-10-08T09:24:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.216+0000 7f1f58819700 1 -- 172.21.15.49:0/3528394131 shutdown_connections 2023-10-08T09:24:54.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.216+0000 7f1f58819700 1 --2- 172.21.15.49:0/3528394131 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f540725e0 0x7f1f54072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.216+0000 7f1f58819700 1 --2- 172.21.15.49:0/3528394131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f54077ee0 0x7f1f5407a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.216+0000 7f1f58819700 1 --2- 172.21.15.49:0/3528394131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f54075510 0x7f1f540779a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.216+0000 7f1f58819700 1 -- 172.21.15.49:0/3528394131 >> 172.21.15.49:0/3528394131 conn(0x7f1f5406d770 msgr2=0x7f1f5406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.223+0000 7f1f58819700 1 -- 172.21.15.49:0/3528394131 shutdown_connections 2023-10-08T09:24:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.226+0000 7f1f58819700 1 -- 172.21.15.49:0/3528394131 wait complete. 2023-10-08T09:24:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 Processor -- start 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 -- start start 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f540725e0 0x7f1f5412b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f54075510 0x7f1f5412dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f54077ee0 0x7f1f54130350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54130b20 con 0x7f1f540725e0 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54130c90 con 0x7f1f54075510 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f58819700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f54130e50 con 0x7f1f54077ee0 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f5259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f540725e0 0x7f1f5412b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f5259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f540725e0 0x7f1f5412b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52944/0 (socket says 172.21.15.49:52944) 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f5259c700 1 -- 172.21.15.49:0/1972470847 learned_addr learned my addr 172.21.15.49:0/1972470847 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f52d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f54077ee0 0x7f1f54130350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.227+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f54075510 0x7f1f5412dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.228+0000 7f1f5259c700 1 -- 172.21.15.49:0/1972470847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f54077ee0 msgr2=0x7f1f54130350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.228+0000 7f1f5259c700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f54077ee0 0x7f1f54130350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.228+0000 7f1f5259c700 1 -- 172.21.15.49:0/1972470847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f54075510 msgr2=0x7f1f5412dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.228+0000 7f1f5259c700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f54075510 0x7f1f5412dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.228+0000 7f1f5259c700 1 -- 172.21.15.49:0/1972470847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48009520 con 0x7f1f540725e0 2023-10-08T09:24:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.228+0000 7f1f5259c700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f540725e0 0x7f1f5412b810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b840 tx=0x7f1f4c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.229+0000 7f1f437fe700 1 -- 172.21.15.49:0/1972470847 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00f840 con 0x7f1f540725e0 2023-10-08T09:24:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.229+0000 7f1f437fe700 1 -- 172.21.15.49:0/1972470847 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f4c00fe60 con 0x7f1f540725e0 2023-10-08T09:24:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.229+0000 7f1f437fe700 1 -- 172.21.15.49:0/1972470847 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00e3b0 con 0x7f1f540725e0 2023-10-08T09:24:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.229+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f440095f0 con 0x7f1f540725e0 2023-10-08T09:24:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.229+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f54131460 con 0x7f1f540725e0 2023-10-08T09:24:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.232+0000 7f1f437fe700 1 -- 172.21.15.49:0/1972470847 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1f4c00f9a0 con 0x7f1f540725e0 2023-10-08T09:24:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f437fe700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1f3c0a15c0 0x7f1f3c0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 Processor -- start 2023-10-08T09:24:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f437fe700 1 -- 172.21.15.49:0/1972470847 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1f4c0ccc60 con 0x7f1f540725e0 2023-10-08T09:24:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 -- start start 2023-10-08T09:24:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b3f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b3fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b40730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5aa7400 con 0x55ddc5b3ee40 2023-10-08T09:24:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5a98bc0 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.233+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5a964e0 con 0x55ddc5b402b0 2023-10-08T09:24:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.242+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1f3c0a15c0 0x7f1f3c0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.254+0000 7f1f51d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b3fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.254+0000 7f1f51d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b3fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42566/0 (socket says 172.21.15.49:42566) 2023-10-08T09:24:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.254+0000 7f1f51d9b700 1 -- 172.21.15.49:0/235608874 learned_addr learned my addr 172.21.15.49:0/235608874 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.262+0000 7f1f52d9d700 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b40730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.262+0000 7f1f5259c700 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b3f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.265+0000 7f1f51d9b700 1 -- 172.21.15.49:0/235608874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 msgr2=0x55ddc5b40730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.265+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b40730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.265+0000 7f1f51d9b700 1 -- 172.21.15.49:0/235608874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 msgr2=0x55ddc5b3f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.265+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b3f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.265+0000 7f1f51d9b700 1 -- 172.21.15.49:0/235608874 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc58bcd80 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.285+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b3fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009db0 tx=0x7f1f4402a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd288f7e8f2a98ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.285+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1f3c0a15c0 0x7f1f3c0a3a80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005850 tx=0x7f1f4400b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.288+0000 7f1f40ff9700 1 -- 172.21.15.49:0/235608874 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4402a070 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.288+0000 7f1f40ff9700 1 -- 172.21.15.49:0/235608874 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f44004040 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.289 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.288+0000 7f1f40ff9700 1 -- 172.21.15.49:0/235608874 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4400e9f0 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.293+0000 7f1f64a02b80 1 -- 172.21.15.49:0/235608874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3f7a0 msgr2=0x55ddc5b3fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.293+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b3fcd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009db0 tx=0x7f1f4402a000 comp rx=0 tx=0).stop 2023-10-08T09:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 -- 172.21.15.49:0/235608874 shutdown_connections 2023-10-08T09:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b40730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b3f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/235608874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b3fcd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 -- 172.21.15.49:0/235608874 >> 172.21.15.49:0/235608874 conn(0x55ddc5ab3200 msgr2=0x55ddc5b398a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 -- 172.21.15.49:0/235608874 shutdown_connections 2023-10-08T09:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 -- 172.21.15.49:0/235608874 wait complete. 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.295+0000 7f1f64a02b80 1 -- start start 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 1 Processor -- start 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b48df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5a91dd0 con 0x55ddc5b3ee40 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5a2e6b0 con 0x55ddc5b402b0 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5a2e350 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f64a02b80 10 client.0 _ll_register_callbacks cb 0x55ddc588b7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f51d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b48df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f52d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f51d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b48df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:39996/0 (socket says 172.21.15.49:39996) 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f51d9b700 1 -- 172.21.15.49:0/2899813940 learned_addr learned my addr 172.21.15.49:0/2899813940 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.296+0000 7f1f52d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42574/0 (socket says 172.21.15.49:42574) 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f51d9b700 1 -- 172.21.15.49:0/2899813940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 msgr2=0x55ddc5b4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f51d9b700 1 -- 172.21.15.49:0/2899813940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 msgr2=0x55ddc5b488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f51d9b700 1 -- 172.21.15.49:0/2899813940 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4401a040 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b48df0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f44006010 tx=0x7f1f44005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3de6f5af947cd7ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2899813940 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44005240 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2899813940 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1f44012070 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddc5a59e50 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2899813940 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f440041b0 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f64a02b80 10 client.26422 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.297+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ddc58a5c00 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.298+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2899813940 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ddc5a59e50 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.300+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2899813940 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1f4400e970 con 0x55ddc5b3f7a0 2023-10-08T09:24:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.300+0000 7f1f64a02b80 10 client.26422 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.300+0000 7f1f64a02b80 10 client.26422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.300+0000 7f1f64a02b80 20 client.26422 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.300+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5b542a0 0x55ddc5b56690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.300+0000 7f1f64a02b80 4 client.26422 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:24:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.300+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddc5a8adb0 con 0x55ddc5b542a0 2023-10-08T09:24:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.304+0000 7f1f5259c700 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5b542a0 0x55ddc5b56690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.314+0000 7f1f39ffb700 10 client.26422 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.314+0000 7f1f5259c700 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5b542a0 0x55ddc5b56690 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9baacd0b6a603b server_cookie=687b0873a8e53f4d in_seq=0 out_seq=0 2023-10-08T09:24:54.315 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 570.0 s, 530.2 tps, lat 182.615 ms stddev 200.367 2023-10-08T09:24:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.315+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2899813940 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ddc5a8adb0 con 0x55ddc5b542a0 2023-10-08T09:24:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.315+0000 7f1f39ffb700 10 client.26422 handle_command_reply: tid=0 2023-10-08T09:24:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.315+0000 7f1f64a02b80 1 client.26422 shutdown 2023-10-08T09:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.317+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3f7a0 msgr2=0x55ddc5b48df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.317+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b48df0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f44006010 tx=0x7f1f44005a00 comp rx=0 tx=0).stop 2023-10-08T09:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.318+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 shutdown_connections 2023-10-08T09:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.318+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5b542a0 0x55ddc5b56690 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.318+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.318+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.318+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2899813940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3f7a0 0x55ddc5b48df0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1f44006010 tx=0x7f1f44005a00 comp rx=0 tx=0).stop 2023-10-08T09:24:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.318+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 >> 172.21.15.49:0/2899813940 conn(0x55ddc5ab3200 msgr2=0x55ddc5b39010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.323+0000 7f1f39ffb700 0 client.26422 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.323+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 shutdown_connections 2023-10-08T09:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.323+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2899813940 wait complete. 2023-10-08T09:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.323+0000 7f1f64a02b80 20 client.26422 trim_cache size 0 max 16384 2023-10-08T09:24:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.347+0000 7f1f64a02b80 1 Processor -- start 2023-10-08T09:24:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f64a02b80 1 -- start start 2023-10-08T09:24:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b47500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b46970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc59128c0 con 0x55ddc5b47e30 2023-10-08T09:24:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5a59e50 con 0x55ddc5b402b0 2023-10-08T09:24:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5b50f10 con 0x55ddc5b3ee40 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f52d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b46970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f52d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b46970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52968/0 (socket says 172.21.15.49:52968) 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f52d9d700 1 -- 172.21.15.49:0/799046207 learned_addr learned my addr 172.21.15.49:0/799046207 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.348+0000 7f1f52d9d700 1 -- 172.21.15.49:0/799046207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3ee40 msgr2=0x55ddc5b47500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f52d9d700 1 --2- 172.21.15.49:0/799046207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b47500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f52d9d700 1 -- 172.21.15.49:0/799046207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 msgr2=0x55ddc5b4fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f52d9d700 1 --2- 172.21.15.49:0/799046207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f52d9d700 1 -- 172.21.15.49:0/799046207 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddc58e31f0 con 0x55ddc5b47e30 2023-10-08T09:24:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f52d9d700 1 --2- 172.21.15.49:0/799046207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b46970 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1f48010460 tx=0x7f1f48010770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ad38a87610ecf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f417fa700 1 -- 172.21.15.49:0/799046207 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48004220 con 0x55ddc5b47e30 2023-10-08T09:24:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f417fa700 1 -- 172.21.15.49:0/799046207 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f48018040 con 0x55ddc5b47e30 2023-10-08T09:24:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.349+0000 7f1f417fa700 1 -- 172.21.15.49:0/799046207 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4801c790 con 0x55ddc5b47e30 2023-10-08T09:24:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.352+0000 7f1f64a02b80 1 -- 172.21.15.49:0/799046207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 msgr2=0x55ddc5b46970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.352+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/799046207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b46970 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1f48010460 tx=0x7f1f48010770 comp rx=0 tx=0).stop 2023-10-08T09:24:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 -- 172.21.15.49:0/799046207 shutdown_connections 2023-10-08T09:24:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/799046207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b47500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/799046207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b46970 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/799046207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 -- 172.21.15.49:0/799046207 >> 172.21.15.49:0/799046207 conn(0x55ddc5ab3200 msgr2=0x55ddc5b5b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 -- 172.21.15.49:0/799046207 shutdown_connections 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 -- 172.21.15.49:0/799046207 wait complete. 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.355+0000 7f1f64a02b80 1 -- start start 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 1 Processor -- start 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b4f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b4c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5b4c820 con 0x55ddc5b47e30 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5b4c960 con 0x55ddc5b3ee40 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddc5b4cb20 con 0x55ddc5b402b0 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f64a02b80 10 client.0 _ll_register_callbacks cb 0x7f1f44005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:24:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f5259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b4f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f5259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b4f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42586/0 (socket says 172.21.15.49:42586) 2023-10-08T09:24:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f5259c700 1 -- 172.21.15.49:0/2582408880 learned_addr learned my addr 172.21.15.49:0/2582408880 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:24:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.356+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f51d9b700 1 -- 172.21.15.49:0/2582408880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3ee40 msgr2=0x55ddc5b4f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b4f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f51d9b700 1 -- 172.21.15.49:0/2582408880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 msgr2=0x55ddc5b4c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b4c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f51d9b700 1 -- 172.21.15.49:0/2582408880 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f48009cd0 con 0x55ddc5b402b0 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f51d9b700 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fbe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55ddc5b3b7b0 tx=0x7f1f440456b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3125efb89ea36a8b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2582408880 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4402a040 con 0x55ddc5b402b0 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f44004040 con 0x55ddc5b402b0 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2582408880 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1f44012040 con 0x55ddc5b402b0 2023-10-08T09:24:54.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2582408880 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f4400eb90 con 0x55ddc5b402b0 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f64a02b80 10 client.26425 fetch_fsmap learned FSMap version 40 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.357+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ddc58bb250 con 0x55ddc5b402b0 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2582408880 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f44004040 con 0x55ddc5b402b0 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2582408880 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1f44004500 con 0x55ddc5b402b0 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f64a02b80 10 client.26425 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f64a02b80 10 client.26425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f64a02b80 20 client.26425 populate_metadata read hostname 'smithi049' 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5ba9310 0x55ddc5bab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f64a02b80 4 client.26425 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ddc5a8aad0 con 0x55ddc5ba9310 2023-10-08T09:24:54.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.358+0000 7f1f5259c700 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5ba9310 0x55ddc5bab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:24:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.359+0000 7f1f39ffb700 10 client.26425 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.359+0000 7f1f5259c700 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5ba9310 0x55ddc5bab700 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfee0627881a12b server_cookie=6fc6974aeed4d794 in_seq=0 out_seq=0 2023-10-08T09:24:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f39ffb700 1 -- 172.21.15.49:0/2582408880 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ddc5a8aad0 con 0x55ddc5ba9310 2023-10-08T09:24:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f39ffb700 10 client.26425 handle_command_reply: tid=0 2023-10-08T09:24:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 client.26425 shutdown 2023-10-08T09:24:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 msgr2=0x55ddc5b4fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fbe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55ddc5b3b7b0 tx=0x7f1f440456b0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 shutdown_connections 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ddc5ba9310 0x55ddc5bab700 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ddc5b47e30 0x55ddc5b4c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f39ffb700 0 client.26425 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ddc5b3ee40 0x55ddc5b4f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 --2- 172.21.15.49:0/2582408880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ddc5b402b0 0x55ddc5b4fbe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.371+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 >> 172.21.15.49:0/2582408880 conn(0x55ddc5ab3200 msgr2=0x55ddc5b46930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 shutdown_connections 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f64a02b80 1 -- 172.21.15.49:0/2582408880 wait complete. 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f64a02b80 20 client.26425 trim_cache size 0 max 16384 2023-10-08T09:24:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:24:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:24:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:24:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1f3c0a15c0 msgr2=0x7f1f3c0a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1f3c0a15c0 0x7f1f3c0a3a80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005850 tx=0x7f1f4400b790 comp rx=0 tx=0).stop 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f540725e0 msgr2=0x7f1f5412b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f540725e0 0x7f1f5412b810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b840 tx=0x7f1f4c00bb50 comp rx=0 tx=0).stop 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 shutdown_connections 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1f3c0a15c0 0x7f1f3c0a3a80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f54077ee0 0x7f1f54130350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f540725e0 0x7f1f5412b810 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 --2- 172.21.15.49:0/1972470847 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f54075510 0x7f1f5412dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 >> 172.21.15.49:0/1972470847 conn(0x7f1f5406d770 msgr2=0x7f1f54079560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.372+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 shutdown_connections 2023-10-08T09:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:24:54.373+0000 7f1f58819700 1 -- 172.21.15.49:0/1972470847 wait complete. 2023-10-08T09:24:55.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:55 smithi132 ceph-mon[103389]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 556 op/s 2023-10-08T09:24:55.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:55 smithi049 ceph-mon[99485]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 556 op/s 2023-10-08T09:24:55.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:55 smithi184 ceph-mon[103675]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.5 MiB/s wr, 556 op/s 2023-10-08T09:24:56.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:56 smithi049 ceph-mon[99485]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.5 MiB/s wr, 548 op/s 2023-10-08T09:24:56.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:56 smithi184 ceph-mon[103675]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.5 MiB/s wr, 548 op/s 2023-10-08T09:24:57.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:56 smithi132 ceph-mon[103389]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.5 MiB/s wr, 548 op/s 2023-10-08T09:24:59.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 575.0 s, 521.5 tps, lat 196.684 ms stddev 214.579 2023-10-08T09:24:59.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:24:59 smithi184 ceph-mon[103675]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 542 op/s 2023-10-08T09:24:59.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:24:59 smithi132 ceph-mon[103389]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 542 op/s 2023-10-08T09:24:59.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:24:59 smithi049 ceph-mon[99485]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 542 op/s 2023-10-08T09:25:01.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:01 smithi184 ceph-mon[103675]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.1 MiB/s wr, 527 op/s 2023-10-08T09:25:01.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:01 smithi132 ceph-mon[103389]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.1 MiB/s wr, 527 op/s 2023-10-08T09:25:01.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:01 smithi049 ceph-mon[99485]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.1 MiB/s wr, 527 op/s 2023-10-08T09:25:03.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:02 smithi049 ceph-mon[99485]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.5 MiB/s wr, 540 op/s 2023-10-08T09:25:03.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:02 smithi184 ceph-mon[103675]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.5 MiB/s wr, 540 op/s 2023-10-08T09:25:03.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:02 smithi132 ceph-mon[103389]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.5 MiB/s wr, 540 op/s 2023-10-08T09:25:04.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 580.0 s, 581.6 tps, lat 170.507 ms stddev 201.253 2023-10-08T09:25:05.752 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:05 smithi184 ceph-mon[103675]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2023-10-08T09:25:05.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:05 smithi132 ceph-mon[103389]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2023-10-08T09:25:05.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:05 smithi049 ceph-mon[99485]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2023-10-08T09:25:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.616+0000 7f1773da3700 1 -- 172.21.15.49:0/641684460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c072630 msgr2=0x7f176c109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.616+0000 7f1773da3700 1 --2- 172.21.15.49:0/641684460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c072630 0x7f176c109800 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f176400b0d0 tx=0x7f176400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.623+0000 7f1773da3700 1 -- 172.21.15.49:0/641684460 shutdown_connections 2023-10-08T09:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.623+0000 7f1773da3700 1 --2- 172.21.15.49:0/641684460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c072630 0x7f176c109800 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.623+0000 7f1773da3700 1 --2- 172.21.15.49:0/641684460 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c10c8f0 0x7f176c117730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.623+0000 7f1773da3700 1 --2- 172.21.15.49:0/641684460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176c109dd0 0x7f176c10c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.623+0000 7f1773da3700 1 -- 172.21.15.49:0/641684460 >> 172.21.15.49:0/641684460 conn(0x7f176c06d6e0 msgr2=0x7f176c06fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.630+0000 7f1773da3700 1 -- 172.21.15.49:0/641684460 shutdown_connections 2023-10-08T09:25:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.631+0000 7f1773da3700 1 -- 172.21.15.49:0/641684460 wait complete. 2023-10-08T09:25:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.631+0000 7f1773da3700 1 Processor -- start 2023-10-08T09:25:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.631+0000 7f1773da3700 1 -- start start 2023-10-08T09:25:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1773da3700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c109dd0 0x7f176c1a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1773da3700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c10c8f0 0x7f176c1a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1773da3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176c1aa980 0x7f176c1acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1773da3700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c1ad340 con 0x7f176c109dd0 2023-10-08T09:25:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1773da3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c1ad480 con 0x7f176c1aa980 2023-10-08T09:25:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1773da3700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c1ad610 con 0x7f176c10c8f0 2023-10-08T09:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c109dd0 0x7f176c1a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c109dd0 0x7f176c1a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52986/0 (socket says 172.21.15.49:52986) 2023-10-08T09:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1771b3f700 1 -- 172.21.15.49:0/3405432936 learned_addr learned my addr 172.21.15.49:0/3405432936 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f177133e700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c10c8f0 0x7f176c1a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1771b3f700 1 -- 172.21.15.49:0/3405432936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c10c8f0 msgr2=0x7f176c1a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1771b3f700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c10c8f0 0x7f176c1a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1771b3f700 1 -- 172.21.15.49:0/3405432936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176c1aa980 msgr2=0x7f176c1acd70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:25:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.632+0000 7f1771b3f700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176c1aa980 0x7f176c1acd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.633+0000 7f1771b3f700 1 -- 172.21.15.49:0/3405432936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764009d00 con 0x7f176c109dd0 2023-10-08T09:25:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.633+0000 7f1771b3f700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c109dd0 0x7f176c1a5480 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1764007850 tx=0x7f1764007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.633+0000 7f1762ffd700 1 -- 172.21.15.49:0/3405432936 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400e040 con 0x7f176c109dd0 2023-10-08T09:25:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.633+0000 7f1762ffd700 1 -- 172.21.15.49:0/3405432936 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1764003e10 con 0x7f176c109dd0 2023-10-08T09:25:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.633+0000 7f1762ffd700 1 -- 172.21.15.49:0/3405432936 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17640054b0 con 0x7f176c109dd0 2023-10-08T09:25:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.633+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c10ef20 con 0x7f176c109dd0 2023-10-08T09:25:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.633+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c10f460 con 0x7f176c109dd0 2023-10-08T09:25:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.638+0000 7f177ff8cb80 1 Processor -- start 2023-10-08T09:25:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.644+0000 7f1762ffd700 1 -- 172.21.15.49:0/3405432936 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1764005610 con 0x7f176c109dd0 2023-10-08T09:25:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.645+0000 7f177ff8cb80 1 -- start start 2023-10-08T09:25:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.645+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea13e50 0x557a2ea14270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.645+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea147b0 0x557a2ea14ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.645+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea152c0 0x557a2ea15740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.645+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e95a400 con 0x557a2ea152c0 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.645+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e94bbc0 con 0x557a2ea13e50 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.645+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e9494e0 con 0x557a2ea147b0 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea147b0 0x557a2ea14ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1762ffd700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f17580a1570 0x7f17580a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea147b0 0x557a2ea14ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40030/0 (socket says 172.21.15.49:40030) 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 -- 172.21.15.49:0/546596040 learned_addr learned my addr 172.21.15.49:0/546596040 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 -- 172.21.15.49:0/546596040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea13e50 msgr2=0x557a2ea14270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:25:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 --2- 172.21.15.49:0/546596040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea13e50 0x557a2ea14270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 -- 172.21.15.49:0/546596040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea152c0 msgr2=0x557a2ea15740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 --2- 172.21.15.49:0/546596040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea152c0 0x557a2ea15740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.646+0000 7f1771b3f700 1 -- 172.21.15.49:0/546596040 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a2e76fd80 con 0x557a2ea147b0 2023-10-08T09:25:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.647+0000 7f1762ffd700 1 -- 172.21.15.49:0/3405432936 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f17640ce4b0 con 0x7f176c109dd0 2023-10-08T09:25:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.647+0000 7f1771b3f700 1 --2- 172.21.15.49:0/546596040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea147b0 0x557a2ea14ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1764023fd0 tx=0x7f1764090c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f935dfca9eb029 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.647+0000 7f1757fff700 1 -- 172.21.15.49:0/546596040 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176409b2a0 con 0x557a2ea147b0 2023-10-08T09:25:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.647+0000 7f1757fff700 1 -- 172.21.15.49:0/546596040 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f176409b010 con 0x557a2ea147b0 2023-10-08T09:25:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.647+0000 7f1757fff700 1 -- 172.21.15.49:0/546596040 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176401aea0 con 0x557a2ea147b0 2023-10-08T09:25:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.648+0000 7f177ff8cb80 1 -- 172.21.15.49:0/546596040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea147b0 msgr2=0x557a2ea14ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.648+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/546596040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea147b0 0x557a2ea14ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1764023fd0 tx=0x7f1764090c90 comp rx=0 tx=0).stop 2023-10-08T09:25:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.660+0000 7f177ff8cb80 1 -- 172.21.15.49:0/546596040 shutdown_connections 2023-10-08T09:25:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.660+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/546596040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea152c0 0x557a2ea15740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.660+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/546596040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea13e50 0x557a2ea14270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.660+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/546596040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea147b0 0x557a2ea14ce0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.660+0000 7f177ff8cb80 1 -- 172.21.15.49:0/546596040 >> 172.21.15.49:0/546596040 conn(0x557a2e966200 msgr2=0x557a2e9ec8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.663+0000 7f1772340700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f17580a1570 0x7f17580a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.669+0000 7f177ff8cb80 1 -- 172.21.15.49:0/546596040 shutdown_connections 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.670+0000 7f177ff8cb80 1 -- 172.21.15.49:0/546596040 wait complete. 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.670+0000 7f1772340700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f17580a1570 0x7f17580a3a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f175c001560 tx=0x7f175c007b90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.670+0000 7f177ff8cb80 1 -- start start 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.670+0000 7f177ff8cb80 1 Processor -- start 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea13e50 0x557a2ea1d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea147b0 0x557a2ea1daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea152c0 0x557a2ea24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e944dd0 con 0x557a2ea13e50 2023-10-08T09:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e8e16b0 con 0x557a2ea147b0 2023-10-08T09:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e8e1350 con 0x557a2ea152c0 2023-10-08T09:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177ff8cb80 10 client.0 _ll_register_callbacks cb 0x557a2e73e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177133e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea147b0 0x557a2ea1daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177133e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea147b0 0x557a2ea1daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42614/0 (socket says 172.21.15.49:42614) 2023-10-08T09:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f177133e700 1 -- 172.21.15.49:0/245417383 learned_addr learned my addr 172.21.15.49:0/245417383 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.671+0000 7f1771b3f700 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea13e50 0x557a2ea1d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f177133e700 1 -- 172.21.15.49:0/245417383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea152c0 msgr2=0x557a2ea24ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f177133e700 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea152c0 0x557a2ea24ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f177133e700 1 -- 172.21.15.49:0/245417383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea13e50 msgr2=0x557a2ea1d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f177133e700 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea13e50 0x557a2ea1d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f177133e700 1 -- 172.21.15.49:0/245417383 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c10f460 con 0x557a2ea147b0 2023-10-08T09:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f177133e700 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea147b0 0x557a2ea1daa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1768004790 tx=0x7f17680106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c12127174a99683d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f17557fa700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f17557fa700 1 -- 172.21.15.49:0/245417383 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176801c680 con 0x557a2ea147b0 2023-10-08T09:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f17557fa700 1 -- 172.21.15.49:0/245417383 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1768017070 con 0x557a2ea147b0 2023-10-08T09:25:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.672+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a2e90ce50 con 0x557a2ea147b0 2023-10-08T09:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.674+0000 7f17557fa700 1 -- 172.21.15.49:0/245417383 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1768010e50 con 0x557a2ea147b0 2023-10-08T09:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.679+0000 7f177ff8cb80 10 client.26372 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.679+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557a2e758c00 con 0x557a2ea147b0 2023-10-08T09:25:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.684+0000 7f17557fa700 1 -- 172.21.15.49:0/245417383 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a2e90ce50 con 0x557a2ea147b0 2023-10-08T09:25:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.684+0000 7f17557fa700 1 -- 172.21.15.49:0/245417383 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1768011a10 con 0x557a2ea147b0 2023-10-08T09:25:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.686+0000 7f177ff8cb80 10 client.26372 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.686+0000 7f177ff8cb80 10 client.26372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.686+0000 7f177ff8cb80 20 client.26372 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.686+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea29420 0x557a2ea2b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.686+0000 7f177ff8cb80 4 client.26372 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:25:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.686+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a2e93ddb0 con 0x557a2ea29420 2023-10-08T09:25:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.686+0000 7f1771b3f700 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea29420 0x557a2ea2b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.687+0000 7f17557fa700 10 client.26372 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.687+0000 7f1771b3f700 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea29420 0x557a2ea2b840 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c87ff26f6e26dd8 server_cookie=b9da29035c0f7157 in_seq=0 out_seq=0 2023-10-08T09:25:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.688+0000 7f17557fa700 1 -- 172.21.15.49:0/245417383 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a2e93ddb0 con 0x557a2ea29420 2023-10-08T09:25:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.688+0000 7f17557fa700 10 client.26372 handle_command_reply: tid=0 2023-10-08T09:25:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.688+0000 7f177ff8cb80 1 client.26372 shutdown 2023-10-08T09:25:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.688+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea147b0 msgr2=0x557a2ea1daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.688+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea147b0 0x557a2ea1daa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1768004790 tx=0x7f17680106b0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 shutdown_connections 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea29420 0x557a2ea2b840 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea152c0 0x557a2ea24ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea13e50 0x557a2ea1d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/245417383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea147b0 0x557a2ea1daa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 >> 172.21.15.49:0/245417383 conn(0x557a2e966200 msgr2=0x557a2e9ec010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f17557fa700 0 client.26372 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.689+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 shutdown_connections 2023-10-08T09:25:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.690+0000 7f177ff8cb80 1 -- 172.21.15.49:0/245417383 wait complete. 2023-10-08T09:25:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.690+0000 7f177ff8cb80 20 client.26372 trim_cache size 0 max 16384 2023-10-08T09:25:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.716+0000 7f177ff8cb80 1 Processor -- start 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.716+0000 7f177ff8cb80 1 -- start start 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea2b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea1ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea24a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e7c58c0 con 0x557a2ea147b0 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2e90ce50 con 0x557a2ea152c0 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2ea14ef0 con 0x557a2ea13e50 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea2b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea2b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40042/0 (socket says 172.21.15.49:40042) 2023-10-08T09:25:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f1771b3f700 1 -- 172.21.15.49:0/425037812 learned_addr learned my addr 172.21.15.49:0/425037812 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f1772340700 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea24a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.717+0000 7f177133e700 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea1ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1771b3f700 1 -- 172.21.15.49:0/425037812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 msgr2=0x557a2ea24a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1771b3f700 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea24a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1771b3f700 1 -- 172.21.15.49:0/425037812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 msgr2=0x557a2ea1ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1771b3f700 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea1ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1771b3f700 1 -- 172.21.15.49:0/425037812 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a2e7961f0 con 0x557a2ea13e50 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1771b3f700 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea2b810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f176400b820 tx=0x7f176400b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6453f171f50d065 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1760ff9700 1 -- 172.21.15.49:0/425037812 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176409ae30 con 0x557a2ea13e50 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1760ff9700 1 -- 172.21.15.49:0/425037812 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f176401aea0 con 0x557a2ea13e50 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.718+0000 7f1760ff9700 1 -- 172.21.15.49:0/425037812 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17640a37d0 con 0x557a2ea13e50 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.719+0000 7f177ff8cb80 1 -- 172.21.15.49:0/425037812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 msgr2=0x557a2ea2b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.719+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea2b810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f176400b820 tx=0x7f176400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.723+0000 7f177ff8cb80 1 -- 172.21.15.49:0/425037812 shutdown_connections 2023-10-08T09:25:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.723+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea2b810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.723+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea24a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.723+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/425037812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea1ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.723+0000 7f177ff8cb80 1 -- 172.21.15.49:0/425037812 >> 172.21.15.49:0/425037812 conn(0x557a2e966200 msgr2=0x557a2ea301a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.724+0000 7f177ff8cb80 1 -- 172.21.15.49:0/425037812 shutdown_connections 2023-10-08T09:25:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.724+0000 7f177ff8cb80 1 -- 172.21.15.49:0/425037812 wait complete. 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.724+0000 7f177ff8cb80 1 -- start start 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.724+0000 7f177ff8cb80 1 Processor -- start 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea20260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177ff8cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2ea23d00 con 0x557a2ea147b0 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2ea23e70 con 0x557a2ea152c0 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177ff8cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a2ea24030 con 0x557a2ea13e50 2023-10-08T09:25:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177ff8cb80 10 client.0 _ll_register_callbacks cb 0x7f176800b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea20260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea20260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40048/0 (socket says 172.21.15.49:40048) 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 -- 172.21.15.49:0/2797865069 learned_addr learned my addr 172.21.15.49:0/2797865069 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1772340700 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f177133e700 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 -- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 msgr2=0x557a2ea237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 -- 172.21.15.49:0/2797865069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 msgr2=0x557a2ea207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 -- 172.21.15.49:0/2797865069 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a2e76fd80 con 0x557a2ea13e50 2023-10-08T09:25:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.725+0000 7f1771b3f700 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea20260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17640079a0 tx=0x7f1764011f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69a50463fb21060 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f17557fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:25:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f17557fa700 1 -- 172.21.15.49:0/2797865069 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176409b4d0 con 0x557a2ea13e50 2023-10-08T09:25:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f17557fa700 1 -- 172.21.15.49:0/2797865069 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f176409bbd0 con 0x557a2ea13e50 2023-10-08T09:25:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f176409b010 con 0x557a2ea13e50 2023-10-08T09:25:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f17557fa700 1 -- 172.21.15.49:0/2797865069 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17640a37d0 con 0x557a2ea13e50 2023-10-08T09:25:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f177ff8cb80 10 client.26440 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557a2e76e250 con 0x557a2ea13e50 2023-10-08T09:25:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.728+0000 7f17557fa700 1 -- 172.21.15.49:0/2797865069 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f176409b010 con 0x557a2ea13e50 2023-10-08T09:25:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.729+0000 7f17557fa700 1 -- 172.21.15.49:0/2797865069 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f17640119a0 con 0x557a2ea13e50 2023-10-08T09:25:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.729+0000 7f177ff8cb80 10 client.26440 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.729+0000 7f177ff8cb80 10 client.26440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.729+0000 7f177ff8cb80 20 client.26440 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.729+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea7d740 0x557a2ea7fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.729+0000 7f177ff8cb80 4 client.26440 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.729+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a2e93dad0 con 0x557a2ea7d740 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.730+0000 7f177133e700 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea7d740 0x557a2ea7fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.730+0000 7f17557fa700 10 client.26440 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:07.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.730+0000 7f177133e700 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea7d740 0x557a2ea7fb30 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4225fe141000f83b server_cookie=e48a1004b621bf3 in_seq=0 out_seq=0 2023-10-08T09:25:07.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:07 smithi184 ceph-mon[103675]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2023-10-08T09:25:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.759+0000 7f17557fa700 1 -- 172.21.15.49:0/2797865069 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a2e93dad0 con 0x557a2ea7d740 2023-10-08T09:25:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.759+0000 7f17557fa700 10 client.26440 handle_command_reply: tid=0 2023-10-08T09:25:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.770+0000 7f177ff8cb80 1 client.26440 shutdown 2023-10-08T09:25:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.783+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 msgr2=0x557a2ea20260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.783+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea20260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f17640079a0 tx=0x7f1764011f30 comp rx=0 tx=0).stop 2023-10-08T09:25:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.784+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 shutdown_connections 2023-10-08T09:25:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.784+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557a2ea7d740 0x557a2ea7fb30 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.784+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557a2ea13e50 0x557a2ea20260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.784+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557a2ea152c0 0x557a2ea237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.784+0000 7f177ff8cb80 1 --2- 172.21.15.49:0/2797865069 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557a2ea147b0 0x557a2ea207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.784+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 >> 172.21.15.49:0/2797865069 conn(0x557a2e966200 msgr2=0x557a2ea1ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.792+0000 7f17557fa700 0 client.26440 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:07.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:07 smithi132 ceph-mon[103389]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2023-10-08T09:25:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.795+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 shutdown_connections 2023-10-08T09:25:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.795+0000 7f177ff8cb80 1 -- 172.21.15.49:0/2797865069 wait complete. 2023-10-08T09:25:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.813+0000 7f177ff8cb80 20 client.26440 trim_cache size 0 max 16384 2023-10-08T09:25:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:25:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:25:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:25:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:25:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f17580a1570 msgr2=0x7f17580a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f17580a1570 0x7f17580a3a30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f175c001560 tx=0x7f175c007b90 comp rx=0 tx=0).stop 2023-10-08T09:25:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c109dd0 msgr2=0x7f176c1a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c109dd0 0x7f176c1a5480 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f1764007850 tx=0x7f1764007880 comp rx=0 tx=0).stop 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 shutdown_connections 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f17580a1570 0x7f17580a3a30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f176c1aa980 0x7f176c1acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f176c109dd0 0x7f176c1a5480 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 --2- 172.21.15.49:0/3405432936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f176c10c8f0 0x7f176c1a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 >> 172.21.15.49:0/3405432936 conn(0x7f176c06d6e0 msgr2=0x7f176c108980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.814+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 shutdown_connections 2023-10-08T09:25:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:07.815+0000 7f1773da3700 1 -- 172.21.15.49:0/3405432936 wait complete. 2023-10-08T09:25:07.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:07 smithi049 ceph-mon[99485]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 5.2 MiB/s wr, 513 op/s 2023-10-08T09:25:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 585.0 s, 635.5 tps, lat 160.895 ms stddev 178.668 2023-10-08T09:25:09.802 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:09 smithi132 ceph-mon[103389]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.8 MiB/s wr, 573 op/s 2023-10-08T09:25:09.886 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:09 smithi049 ceph-mon[99485]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.8 MiB/s wr, 573 op/s 2023-10-08T09:25:10.008 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:09 smithi184 ceph-mon[103675]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.8 MiB/s wr, 573 op/s 2023-10-08T09:25:10.888 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:10 smithi049 ceph-mon[99485]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.1 MiB/s wr, 519 op/s 2023-10-08T09:25:10.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:10 smithi184 ceph-mon[103675]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.1 MiB/s wr, 519 op/s 2023-10-08T09:25:11.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:10 smithi132 ceph-mon[103389]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.1 MiB/s wr, 519 op/s 2023-10-08T09:25:13.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:12 smithi184 ceph-mon[103675]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 530 op/s 2023-10-08T09:25:13.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:12 smithi132 ceph-mon[103389]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 530 op/s 2023-10-08T09:25:13.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:12 smithi049 ceph-mon[99485]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 530 op/s 2023-10-08T09:25:14.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 590.0 s, 552.7 tps, lat 179.038 ms stddev 194.341 2023-10-08T09:25:15.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:15 smithi132 ceph-mon[103389]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.5 MiB/s wr, 571 op/s 2023-10-08T09:25:15.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:15 smithi049 ceph-mon[99485]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.5 MiB/s wr, 571 op/s 2023-10-08T09:25:15.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:15 smithi184 ceph-mon[103675]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.5 MiB/s wr, 571 op/s 2023-10-08T09:25:17.805 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:17 smithi132 ceph-mon[103389]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 544 op/s 2023-10-08T09:25:17.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:17 smithi049 ceph-mon[99485]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 544 op/s 2023-10-08T09:25:18.003 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:17 smithi184 ceph-mon[103675]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.2 MiB/s wr, 544 op/s 2023-10-08T09:25:18.485 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:25:18.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 -- 172.21.15.49:0/3726342728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e40ff6d0 msgr2=0x7fb2e40ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/3726342728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e40ffaf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009920 tx=0x7fb2d4009c30 comp rx=0 tx=0).stop 2023-10-08T09:25:18.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 -- 172.21.15.49:0/3726342728 shutdown_connections 2023-10-08T09:25:18.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/3726342728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2e4101260 0x7fb2e4107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/3726342728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e40ffaf0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.778 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/3726342728 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e41008c0 0x7fb2e4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 -- 172.21.15.49:0/3726342728 >> 172.21.15.49:0/3726342728 conn(0x7fb2e40fac40 msgr2=0x7fb2e40fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 -- 172.21.15.49:0/3726342728 shutdown_connections 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.774+0000 7fb2e9cff700 1 -- 172.21.15.49:0/3726342728 wait complete. 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 Processor -- start 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 -- start start 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e41058a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2e41008c0 0x7fb2e4105de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e4101260 0x7fb2e4106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41073a0 con 0x7fb2e4101260 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4107510 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e9cff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e41076d0 con 0x7fb2e41008c0 2023-10-08T09:25:18.779 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e41058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2e41008c0 0x7fb2e4105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e41058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42642/0 (socket says 172.21.15.49:42642) 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e37fe700 1 -- 172.21.15.49:0/694089301 learned_addr learned my addr 172.21.15.49:0/694089301 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.775+0000 7fb2e3fff700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e4101260 0x7fb2e4106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e37fe700 1 -- 172.21.15.49:0/694089301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2e41008c0 msgr2=0x7fb2e4105de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2e41008c0 0x7fb2e4105de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e37fe700 1 -- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e4101260 msgr2=0x7fb2e4106dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e4101260 0x7fb2e4106dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e37fe700 1 -- 172.21.15.49:0/694089301 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4009520 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2e41008c0 0x7fb2e4105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e41058a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009230 tx=0x7fb2d400bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:18.780 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e0ff9700 1 -- 172.21.15.49:0/694089301 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d401c070 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e41078f0 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e0ff9700 1 -- 172.21.15.49:0/694089301 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d400bba0 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e0ff9700 1 -- 172.21.15.49:0/694089301 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4004930 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.776+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4107da0 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.779+0000 7fb2e0ff9700 1 -- 172.21.15.49:0/694089301 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb2d4020070 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.781 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.780+0000 7fb2e0ff9700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.780+0000 7fb2e0ff9700 1 -- 172.21.15.49:0/694089301 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb2d40cdc00 con 0x7fb2e40ff6d0 2023-10-08T09:25:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.780+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.782 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 Processor -- start 2023-10-08T09:25:18.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 -- start start 2023-10-08T09:25:18.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e0e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005fd0 tx=0x7fb2d8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:18.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e0f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96d76460 con 0x564c96e0e060 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96d67c20 con 0x564c96e0f800 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96d65540 con 0x564c96e0e9c0 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e0f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e0f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40072/0 (socket says 172.21.15.49:40072) 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/2193925700 learned_addr learned my addr 172.21.15.49:0/2193925700 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e0fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.781+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/2193925700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 msgr2=0x564c96e0fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/2193925700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e0fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/2193925700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 msgr2=0x564c96e0e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:18.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/2193925700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e0e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/2193925700 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c96b8bde0 con 0x564c96e0e9c0 2023-10-08T09:25:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/2193925700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e0f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80043c0 tx=0x7fb2d8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c374804f1d6c5a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2c9ffb700 1 -- 172.21.15.49:0/2193925700 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8003d10 con 0x564c96e0e9c0 2023-10-08T09:25:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2c9ffb700 1 -- 172.21.15.49:0/2193925700 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d8003e70 con 0x564c96e0e9c0 2023-10-08T09:25:18.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.782+0000 7fb2c9ffb700 1 -- 172.21.15.49:0/2193925700 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800e780 con 0x564c96e0e9c0 2023-10-08T09:25:18.787 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.783+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2193925700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 msgr2=0x564c96e0f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.783+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2193925700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e0f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80043c0 tx=0x7fb2d8014000 comp rx=0 tx=0).stop 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2193925700 shutdown_connections 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2193925700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e0fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2193925700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e0e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2193925700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e0f270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2193925700 >> 172.21.15.49:0/2193925700 conn(0x564c96d82260 msgr2=0x564c96e08e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2193925700 shutdown_connections 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2193925700 wait complete. 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- start start 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 Processor -- start 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0e060 0x564c96e28ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.788 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e29410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0f800 0x564c96e30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96d60e30 con 0x564c96e0e9c0 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96cfd710 con 0x564c96e0e060 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.784+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96cfd3b0 con 0x564c96e0f800 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2f5ee8b80 10 client.0 _ll_register_callbacks cb 0x564c96b5a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e29410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0e060 0x564c96e28ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e29410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53044/0 (socket says 172.21.15.49:53044) 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/4003791252 learned_addr learned my addr 172.21.15.49:0/4003791252 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e3fff700 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0f800 0x564c96e30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/4003791252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0f800 msgr2=0x564c96e30450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:18.789 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0f800 0x564c96e30450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0e060 msgr2=0x564c96e28ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0e060 0x564c96e28ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 -- 172.21.15.49:0/4003791252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d8017070 con 0x564c96e0e9c0 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0e060 0x564c96e28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e29410 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8003960 tx=0x7fb2d80052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e14368029f7306 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.785+0000 7fb2c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4003791252 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d802c070 con 0x564c96e0e9c0 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4003791252 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb2d8005690 con 0x564c96e0e9c0 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c96d28eb0 con 0x564c96e0e9c0 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4003791252 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2d8004710 con 0x564c96e0e9c0 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 10 client.16791 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:18.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564c96b74c60 con 0x564c96e0e9c0 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4003791252 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c96d28eb0 con 0x564c96e0e9c0 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4003791252 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb2d8024020 con 0x564c96e0e9c0 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 10 client.16791 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 20 client.16791 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e34b70 0x564c96e36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 4 client.16791 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:25:18.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.786+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c96d59e10 con 0x564c96e34b70 2023-10-08T09:25:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.787+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e34b70 0x564c96e36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.791+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e34b70 0x564c96e36f90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4c4231bfed32ae server_cookie=6a3f1db68274af7b in_seq=0 out_seq=0 2023-10-08T09:25:18.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.791+0000 7fb2c2ffd700 10 client.16791 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:18.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.791+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4003791252 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c96d59e10 con 0x564c96e34b70 2023-10-08T09:25:18.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.791+0000 7fb2c2ffd700 10 client.16791 handle_command_reply: tid=0 2023-10-08T09:25:18.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.791+0000 7fb2f5ee8b80 1 client.16791 shutdown 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 msgr2=0x564c96e29410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e29410 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8003960 tx=0x7fb2d80052e0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 shutdown_connections 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e34b70 0x564c96e36f90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0f800 0x564c96e30450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2c2ffd700 0 client.16791 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0e060 0x564c96e28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4003791252 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e29410 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 >> 172.21.15.49:0/4003791252 conn(0x564c96d82260 msgr2=0x564c96e084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 shutdown_connections 2023-10-08T09:25:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4003791252 wait complete. 2023-10-08T09:25:18.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.792+0000 7fb2f5ee8b80 20 client.16791 trim_cache size 0 max 16384 2023-10-08T09:25:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.822+0000 7fb2f5ee8b80 1 Processor -- start 2023-10-08T09:25:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.823+0000 7fb2f5ee8b80 1 -- start start 2023-10-08T09:25:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.823+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.823+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e30410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.823+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e12d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.823+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96be1920 con 0x564c96e0e060 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.823+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96e27720 con 0x564c96e0f800 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.823+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96e29dd0 con 0x564c96e0e9c0 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.824+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e12800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53050/0 (socket says 172.21.15.49:53050) 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 -- 172.21.15.49:0/2033010286 learned_addr learned my addr 172.21.15.49:0/2033010286 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e3fff700 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e12d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 -- 172.21.15.49:0/2033010286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 msgr2=0x564c96e30410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e30410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 -- 172.21.15.49:0/2033010286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 msgr2=0x564c96e12d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e12d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 -- 172.21.15.49:0/2033010286 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c96bb2250 con 0x564c96e0e060 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.825+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e12800 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40976f0 tx=0x7fb2d4005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7508c8fd54ba20 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.828+0000 7fb2ca7fc700 1 -- 172.21.15.49:0/2033010286 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d409ac80 con 0x564c96e0e060 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.828+0000 7fb2ca7fc700 1 -- 172.21.15.49:0/2033010286 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d409edd0 con 0x564c96e0e060 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.828+0000 7fb2ca7fc700 1 -- 172.21.15.49:0/2033010286 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d409a170 con 0x564c96e0e060 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.829+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2033010286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 msgr2=0x564c96e12800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.835 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.829+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e12800 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40976f0 tx=0x7fb2d4005570 comp rx=0 tx=0).stop 2023-10-08T09:25:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.835+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2033010286 shutdown_connections 2023-10-08T09:25:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.835+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e12d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.835+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e060 0x564c96e12800 secure :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40976f0 tx=0x7fb2d4005570 comp rx=0 tx=0).stop 2023-10-08T09:25:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.835+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/2033010286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e0e9c0 0x564c96e30410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.844 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.835+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2033010286 >> 172.21.15.49:0/2033010286 conn(0x564c96d82260 msgr2=0x564c96e283e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:18.868 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.859+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2033010286 shutdown_connections 2023-10-08T09:25:18.879 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.869+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/2033010286 wait complete. 2023-10-08T09:25:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.869+0000 7fb2f5ee8b80 1 -- start start 2023-10-08T09:25:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.869+0000 7fb2f5ee8b80 1 Processor -- start 2023-10-08T09:25:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.870+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e30030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.870+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e2e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.880 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.870+0000 7fb2f5ee8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e2ec20 0x564c96e2f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.870+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96e306a0 con 0x564c96e0e9c0 2023-10-08T09:25:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.870+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96e30810 con 0x564c96e0f800 2023-10-08T09:25:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.870+0000 7fb2f5ee8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c96e889d0 con 0x564c96e2ec20 2023-10-08T09:25:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.870+0000 7fb2f5ee8b80 10 client.0 _ll_register_callbacks cb 0x7fb2dc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e30030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e30030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53054/0 (socket says 172.21.15.49:53054) 2023-10-08T09:25:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 -- 172.21.15.49:0/4237787081 learned_addr learned my addr 172.21.15.49:0/4237787081 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 -- 172.21.15.49:0/4237787081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e2ec20 msgr2=0x564c96e2f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e2ec20 0x564c96e2f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 -- 172.21.15.49:0/4237787081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 msgr2=0x564c96e2e6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:25:18.882 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e2e6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 -- 172.21.15.49:0/4237787081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4107da0 con 0x564c96e0e9c0 2023-10-08T09:25:18.883 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2e37fe700 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e30030 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4099270 tx=0x7fb2d4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6b6eaec4b1ec71 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.873+0000 7fb2c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:25:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4237787081 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d409b170 con 0x564c96e0e9c0 2023-10-08T09:25:18.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4237787081 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb2d40b32b0 con 0x564c96e0e9c0 2023-10-08T09:25:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2d8003e70 con 0x564c96e0e9c0 2023-10-08T09:25:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4237787081 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2d409ad00 con 0x564c96e0e9c0 2023-10-08T09:25:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 10 client.16797 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564c96b8a2b0 con 0x564c96e0e9c0 2023-10-08T09:25:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4237787081 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2d407d080 con 0x564c96e0e9c0 2023-10-08T09:25:18.888 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4237787081 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb2d4079b90 con 0x564c96e0e9c0 2023-10-08T09:25:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 20 client.16797 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e16a50 0x564c96e18e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 4 client.16797 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:25:18.889 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.874+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c96d59b30 con 0x564c96e16a50 2023-10-08T09:25:18.890 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.889+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e16a50 0x564c96e18e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:18.898 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:18 smithi049 ceph-mon[99485]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.0 MiB/s wr, 614 op/s 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.923+0000 7fb2e2ffd700 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e16a50 0x564c96e18e70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8e5d26eed3ac55 server_cookie=26ff49d06891b312 in_seq=0 out_seq=0 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.924+0000 7fb2c2ffd700 10 client.16797 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.928+0000 7fb2c2ffd700 1 -- 172.21.15.49:0/4237787081 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c96d59b30 con 0x564c96e16a50 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.928+0000 7fb2c2ffd700 10 client.16797 handle_command_reply: tid=0 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.928+0000 7fb2f5ee8b80 1 client.16797 shutdown 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 msgr2=0x564c96e30030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e30030 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4099270 tx=0x7fb2d4009c30 comp rx=0 tx=0).stop 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 shutdown_connections 2023-10-08T09:25:18.933 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564c96e16a50 0x564c96e18e70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c96e2ec20 0x564c96e2f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564c96e0e9c0 0x564c96e30030 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 --2- 172.21.15.49:0/4237787081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564c96e0f800 0x564c96e2e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.930+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 >> 172.21.15.49:0/4237787081 conn(0x564c96d82260 msgr2=0x564c96e12800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:18.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.931+0000 7fb2c2ffd700 0 client.16797 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:18.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.931+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 shutdown_connections 2023-10-08T09:25:18.934 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.931+0000 7fb2f5ee8b80 1 -- 172.21.15.49:0/4237787081 wait complete. 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:25:18.944 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:25:18.945 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:25:18.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.942+0000 7fb2f5ee8b80 20 client.16797 trim_cache size 0 max 16384 2023-10-08T09:25:18.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.942+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb2cc0a1480 msgr2=0x7fb2cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.942+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8005fd0 tx=0x7fb2d8009310 comp rx=0 tx=0).stop 2023-10-08T09:25:18.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.942+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e40ff6d0 msgr2=0x7fb2e41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:18.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.942+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e41058a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009230 tx=0x7fb2d400bef0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.949+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 shutdown_connections 2023-10-08T09:25:18.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.949+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.949+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb2e4101260 0x7fb2e4106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.949+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2e40ff6d0 0x7fb2e41058a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.949+0000 7fb2e9cff700 1 --2- 172.21.15.49:0/694089301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2e41008c0 0x7fb2e4105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.949+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 >> 172.21.15.49:0/694089301 conn(0x7fb2e40fac40 msgr2=0x7fb2e40fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:18.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.952+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 shutdown_connections 2023-10-08T09:25:18.966 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:18.957+0000 7fb2e9cff700 1 -- 172.21.15.49:0/694089301 wait complete. 2023-10-08T09:25:18.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:18 smithi184 ceph-mon[103675]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.0 MiB/s wr, 614 op/s 2023-10-08T09:25:19.001 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:25:19.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:18 smithi132 ceph-mon[103389]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.0 MiB/s wr, 614 op/s 2023-10-08T09:25:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 595.0 s, 511.5 tps, lat 194.589 ms stddev 230.870 2023-10-08T09:25:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.298+0000 7fe5a8c62700 1 -- 172.21.15.49:0/4230647479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410a160 msgr2=0x7fe5a410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.298+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/4230647479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410a160 0x7fe5a410c5f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe594009990 tx=0x7fe594009ca0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.303+0000 7fe5a8c62700 1 -- 172.21.15.49:0/4230647479 shutdown_connections 2023-10-08T09:25:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.303+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/4230647479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5a410cb30 0x7fe5a410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.303+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/4230647479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5a41077f0 0x7fe5a4109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.303+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/4230647479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410a160 0x7fe5a410c5f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.303+0000 7fe5a8c62700 1 -- 172.21.15.49:0/4230647479 >> 172.21.15.49:0/4230647479 conn(0x7fe5a406d770 msgr2=0x7fe5a406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.309+0000 7fe5a8c62700 1 -- 172.21.15.49:0/4230647479 shutdown_connections 2023-10-08T09:25:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.310+0000 7fe5a8c62700 1 -- 172.21.15.49:0/4230647479 wait complete. 2023-10-08T09:25:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.311+0000 7fe5a8c62700 1 Processor -- start 2023-10-08T09:25:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.313+0000 7fe5a8c62700 1 -- start start 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.313+0000 7fe5a8c62700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5a41077f0 0x7fe5a4118950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.313+0000 7fe5a8c62700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410cb30 0x7fe5a4118e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.313+0000 7fe5a8c62700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5a4117020 0x7fe5a41174d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.313+0000 7fe5a8c62700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a4117a10 con 0x7fe5a41077f0 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.313+0000 7fe5a8c62700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a4117b80 con 0x7fe5a4117020 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.313+0000 7fe5a8c62700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a4117d40 con 0x7fe5a410cb30 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.314+0000 7fe5a1d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410cb30 0x7fe5a4118e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.314+0000 7fe5a1d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410cb30 0x7fe5a4118e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40104/0 (socket says 172.21.15.49:40104) 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.314+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/325862448 learned_addr learned my addr 172.21.15.49:0/325862448 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.316+0000 7fe5a259c700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5a41077f0 0x7fe5a4118950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.317+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5a4117020 0x7fe5a41174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.317+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/325862448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5a4117020 msgr2=0x7fe5a41174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.317+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5a4117020 0x7fe5a41174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.317+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5a41077f0 msgr2=0x7fe5a4118950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.317+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5a41077f0 0x7fe5a4118950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.317+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/325862448 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940095f0 con 0x7fe5a410cb30 2023-10-08T09:25:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.320+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410cb30 0x7fe5a4118e90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe5940098e0 tx=0x7fe5940057d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.320+0000 7fe5937fe700 1 -- 172.21.15.49:0/325862448 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59401d070 con 0x7fe5a410cb30 2023-10-08T09:25:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.320+0000 7fe5937fe700 1 -- 172.21.15.49:0/325862448 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59400f460 con 0x7fe5a410cb30 2023-10-08T09:25:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.320+0000 7fe5937fe700 1 -- 172.21.15.49:0/325862448 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594017650 con 0x7fe5a410cb30 2023-10-08T09:25:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.320+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a4078a50 con 0x7fe5a410cb30 2023-10-08T09:25:21.322 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.320+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a4078eb0 con 0x7fe5a410cb30 2023-10-08T09:25:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.324+0000 7fe5937fe700 1 -- 172.21.15.49:0/325862448 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe59400f5d0 con 0x7fe5a410cb30 2023-10-08T09:25:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.332+0000 7fe5b4e4bb80 1 Processor -- start 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5b4e4bb80 1 -- start start 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5937fe700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe58c0a15d0 0x7fe58c0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1c9e40 0x55685a1ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1ca7a0 0x55685a1cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a132400 con 0x55685a1ca7a0 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a123bc0 con 0x55685a1c9e40 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a1214e0 con 0x55685a1cb2b0 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.333+0000 7fe5937fe700 1 -- 172.21.15.49:0/325862448 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe5940ce6b0 con 0x7fe5a410cb30 2023-10-08T09:25:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40110/0 (socket says 172.21.15.49:40110) 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 -- 172.21.15.49:0/760320903 learned_addr learned my addr 172.21.15.49:0/760320903 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe58c0a15d0 0x7fe58c0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 -- 172.21.15.49:0/760320903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1c9e40 msgr2=0x55685a1ca260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/760320903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1c9e40 0x55685a1ca260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 -- 172.21.15.49:0/760320903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1ca7a0 msgr2=0x55685a1cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/760320903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1ca7a0 0x55685a1cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 -- 172.21.15.49:0/760320903 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556859f47d80 con 0x55685a1cb2b0 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/760320903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1cb730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe598034ae0 tx=0x7fe598034df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307d40e31c699fa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.334+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe58c0a15d0 0x7fe58c0a3a90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5980326c0 tx=0x7fe598008000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.335+0000 7fe590ff9700 1 -- 172.21.15.49:0/760320903 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598032df0 con 0x55685a1cb2b0 2023-10-08T09:25:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.335+0000 7fe590ff9700 1 -- 172.21.15.49:0/760320903 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe598009aa0 con 0x55685a1cb2b0 2023-10-08T09:25:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.335+0000 7fe590ff9700 1 -- 172.21.15.49:0/760320903 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598003680 con 0x55685a1cb2b0 2023-10-08T09:25:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.336+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/760320903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 msgr2=0x55685a1cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.336+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/760320903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1cb730 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe598034ae0 tx=0x7fe598034df0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.336+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/760320903 shutdown_connections 2023-10-08T09:25:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.336+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/760320903 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1cb730 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.336+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/760320903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1c9e40 0x55685a1ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.336+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/760320903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1ca7a0 0x55685a1cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.336+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/760320903 >> 172.21.15.49:0/760320903 conn(0x55685a13e200 msgr2=0x55685a1c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.337+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/760320903 shutdown_connections 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.337+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/760320903 wait complete. 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 -- start start 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 Processor -- start 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1db330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a11cdd0 con 0x55685a1d8f40 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a0b96b0 con 0x55685a1ca7a0 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a0b9350 con 0x55685a1cb2b0 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5b4e4bb80 10 client.0 _ll_register_callbacks cb 0x556859f167a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5a259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5a259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1d3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42694/0 (socket says 172.21.15.49:42694) 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5a259c700 1 -- 172.21.15.49:0/2849453931 learned_addr learned my addr 172.21.15.49:0/2849453931 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.338+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/2849453931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 msgr2=0x55685a1d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/2849453931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 msgr2=0x55685a1db330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1db330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/2849453931 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59803c040 con 0x55685a1cb2b0 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1d3f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe59409b660 tx=0x7fe59409b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd831d27dfd5bbbd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe589ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:25:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe589ffb700 1 -- 172.21.15.49:0/2849453931 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59409bdd0 con 0x55685a1cb2b0 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe589ffb700 1 -- 172.21.15.49:0/2849453931 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe59406be70 con 0x55685a1cb2b0 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.339+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55685a0e4e50 con 0x55685a1cb2b0 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe589ffb700 1 -- 172.21.15.49:0/2849453931 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59409b6d0 con 0x55685a1cb2b0 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 10 client.26464 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556859f30c00 con 0x55685a1cb2b0 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe589ffb700 1 -- 172.21.15.49:0/2849453931 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55685a0e4e50 con 0x55685a1cb2b0 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe589ffb700 1 -- 172.21.15.49:0/2849453931 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe59406bb30 con 0x55685a1cb2b0 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 10 client.26464 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 20 client.26464 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a1df8a0 0x55685a1e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 4 client.26464 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.340+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55685a115db0 con 0x55685a1df8a0 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.341+0000 7fe5a259c700 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a1df8a0 0x55685a1e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.342+0000 7fe5a259c700 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a1df8a0 0x55685a1e1c90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61f5e98327d302e server_cookie=8638419315d382b6 in_seq=0 out_seq=0 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.342+0000 7fe589ffb700 10 client.26464 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.342+0000 7fe589ffb700 1 -- 172.21.15.49:0/2849453931 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55685a115db0 con 0x55685a1df8a0 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.342+0000 7fe589ffb700 10 client.26464 handle_command_reply: tid=0 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.342+0000 7fe5b4e4bb80 1 client.26464 shutdown 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 msgr2=0x55685a1d3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1d3f60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe59409b660 tx=0x7fe59409b520 comp rx=0 tx=0).stop 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 shutdown_connections 2023-10-08T09:25:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a1df8a0 0x55685a1e1c90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2849453931 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1cb2b0 0x55685a1d3f60 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe59409b660 tx=0x7fe59409b520 comp rx=0 tx=0).stop 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 >> 172.21.15.49:0/2849453931 conn(0x55685a13e200 msgr2=0x55685a1c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe589ffb700 0 client.26464 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 shutdown_connections 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2849453931 wait complete. 2023-10-08T09:25:21.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.343+0000 7fe5b4e4bb80 20 client.26464 trim_cache size 0 max 16384 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 Processor -- start 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 -- start start 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1c0c80 0x55685a1d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556859f9d8c0 con 0x55685a1d8f40 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a1c5550 con 0x55685a1ca7a0 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.362+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a1d3590 con 0x55685a1c0c80 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53090/0 (socket says 172.21.15.49:53090) 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/2318266681 learned_addr learned my addr 172.21.15.49:0/2318266681 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a259c700 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1c0c80 0x55685a1d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/2318266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1c0c80 msgr2=0x55685a1d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1c0c80 0x55685a1d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/2318266681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 msgr2=0x55685a1cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 -- 172.21.15.49:0/2318266681 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556859f6e1f0 con 0x55685a1d8f40 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1ef470 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe594097bd0 tx=0x7fe59400fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72eac499da7960a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5917fa700 1 -- 172.21.15.49:0/2318266681 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59409b210 con 0x55685a1d8f40 2023-10-08T09:25:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5917fa700 1 -- 172.21.15.49:0/2318266681 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59409bdd0 con 0x55685a1d8f40 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.363+0000 7fe5917fa700 1 -- 172.21.15.49:0/2318266681 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5940ce4a0 con 0x55685a1d8f40 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2318266681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 msgr2=0x55685a1ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1ef470 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe594097bd0 tx=0x7fe59400fbd0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2318266681 shutdown_connections 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1c0c80 0x55685a1d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1ca7a0 0x55685a1cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/2318266681 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1d8f40 0x55685a1ef470 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2318266681 >> 172.21.15.49:0/2318266681 conn(0x55685a13e200 msgr2=0x55685a1e65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2318266681 shutdown_connections 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/2318266681 wait complete. 2023-10-08T09:25:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.364+0000 7fe5b4e4bb80 1 -- start start 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 1 Processor -- start 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1c0c80 0x55685a1eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1ca7a0 0x55685a1ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1d8f40 0x55685a1edec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a1ee400 con 0x55685a1c0c80 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a1ee570 con 0x55685a1d8f40 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685a1ee730 con 0x55685a1ca7a0 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5b4e4bb80 10 client.0 _ll_register_callbacks cb 0x556859f167a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1c0c80 0x55685a1eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5a259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1c0c80 0x55685a1eca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53096/0 (socket says 172.21.15.49:53096) 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5a259c700 1 -- 172.21.15.49:0/1927116139 learned_addr learned my addr 172.21.15.49:0/1927116139 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1d8f40 0x55685a1edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.365+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1ca7a0 0x55685a1ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5a259c700 1 -- 172.21.15.49:0/1927116139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1ca7a0 msgr2=0x55685a1ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5a259c700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1ca7a0 0x55685a1ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5a259c700 1 -- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1d8f40 msgr2=0x55685a1edec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5a259c700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1d8f40 0x55685a1edec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5a259c700 1 -- 172.21.15.49:0/1927116139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556859f30c00 con 0x55685a1c0c80 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5a2d9d700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1d8f40 0x55685a1edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5a259c700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1c0c80 0x55685a1eca60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55685a1e2540 tx=0x7fe59c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b9cb337ab2ce92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe589ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe589ffb700 1 -- 172.21.15.49:0/1927116139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c01d070 con 0x55685a1c0c80 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe589ffb700 1 -- 172.21.15.49:0/1927116139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe59c003a00 con 0x55685a1c0c80 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe598009aa0 con 0x55685a1c0c80 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe589ffb700 1 -- 172.21.15.49:0/1927116139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe59c003c00 con 0x55685a1c0c80 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5b4e4bb80 10 client.16812 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.366+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x556859f46250 con 0x55685a1c0c80 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe589ffb700 1 -- 172.21.15.49:0/1927116139 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe598009aa0 con 0x55685a1c0c80 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe589ffb700 1 -- 172.21.15.49:0/1927116139 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe59c022530 con 0x55685a1c0c80 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe5b4e4bb80 10 client.16812 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe5b4e4bb80 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe5b4e4bb80 20 client.16812 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a234360 0x55685a236780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe5b4e4bb80 4 client.16812 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:25:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55685a115ad0 con 0x55685a234360 2023-10-08T09:25:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.367+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a234360 0x55685a236780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.368+0000 7fe589ffb700 10 client.16812 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.368+0000 7fe5a1d9b700 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a234360 0x55685a236780 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ce48ee30f4e394 server_cookie=c9c1016a958747b9 in_seq=0 out_seq=0 2023-10-08T09:25:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.381+0000 7fe589ffb700 1 -- 172.21.15.49:0/1927116139 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55685a115ad0 con 0x55685a234360 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.381+0000 7fe589ffb700 10 client.16812 handle_command_reply: tid=0 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.381+0000 7fe5b4e4bb80 1 client.16812 shutdown 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1c0c80 msgr2=0x55685a1eca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1c0c80 0x55685a1eca60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55685a1e2540 tx=0x7fe59c0078e0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 shutdown_connections 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55685a234360 0x55685a236780 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55685a1d8f40 0x55685a1edec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55685a1c0c80 0x55685a1eca60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 --2- 172.21.15.49:0/1927116139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55685a1ca7a0 0x55685a1ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 >> 172.21.15.49:0/1927116139 conn(0x55685a13e200 msgr2=0x55685a1ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe589ffb700 0 client.16812 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 shutdown_connections 2023-10-08T09:25:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 1 -- 172.21.15.49:0/1927116139 wait complete. 2023-10-08T09:25:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.382+0000 7fe5b4e4bb80 20 client.16812 trim_cache size 0 max 16384 2023-10-08T09:25:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:25:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:25:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:25:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:25:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe58c0a15d0 msgr2=0x7fe58c0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe58c0a15d0 0x7fe58c0a3a90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5980326c0 tx=0x7fe598008000 comp rx=0 tx=0).stop 2023-10-08T09:25:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410cb30 msgr2=0x7fe5a4118e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410cb30 0x7fe5a4118e90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe5940098e0 tx=0x7fe5940057d0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 shutdown_connections 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe58c0a15d0 0x7fe58c0a3a90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe5a4117020 0x7fe5a41174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5a41077f0 0x7fe5a4118950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 --2- 172.21.15.49:0/325862448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5a410cb30 0x7fe5a4118e90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 >> 172.21.15.49:0/325862448 conn(0x7fe5a406d770 msgr2=0x7fe5a410a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 shutdown_connections 2023-10-08T09:25:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:21.383+0000 7fe5a8c62700 1 -- 172.21.15.49:0/325862448 wait complete. 2023-10-08T09:25:21.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:21 smithi049 ceph-mon[99485]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.6 MiB/s wr, 569 op/s 2023-10-08T09:25:21.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:21 smithi184 ceph-mon[103675]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.6 MiB/s wr, 569 op/s 2023-10-08T09:25:22.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:21 smithi132 ceph-mon[103389]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.6 MiB/s wr, 569 op/s 2023-10-08T09:25:23.251 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:22 smithi184 ceph-mon[103675]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.5 MiB/s wr, 571 op/s 2023-10-08T09:25:23.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:22 smithi132 ceph-mon[103389]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.5 MiB/s wr, 571 op/s 2023-10-08T09:25:23.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:22 smithi049 ceph-mon[99485]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.5 MiB/s wr, 571 op/s 2023-10-08T09:25:24.313 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 600.0 s, 629.4 tps, lat 160.362 ms stddev 194.245 2023-10-08T09:25:25.889 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:25 smithi049 ceph-mon[99485]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.7 MiB/s wr, 612 op/s 2023-10-08T09:25:25.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:25 smithi184 ceph-mon[103675]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.7 MiB/s wr, 612 op/s 2023-10-08T09:25:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:25 smithi132 ceph-mon[103389]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.7 MiB/s wr, 612 op/s 2023-10-08T09:25:26.887 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:26 smithi049 ceph-mon[99485]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2023-10-08T09:25:26.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:26 smithi184 ceph-mon[103675]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2023-10-08T09:25:27.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:26 smithi132 ceph-mon[103389]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2023-10-08T09:25:29.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 605.0 s, 608.2 tps, lat 167.900 ms stddev 179.001 2023-10-08T09:25:29.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:29 smithi049 ceph-mon[99485]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.1 MiB/s wr, 627 op/s 2023-10-08T09:25:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:29 smithi184 ceph-mon[103675]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.1 MiB/s wr, 627 op/s 2023-10-08T09:25:30.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:29 smithi132 ceph-mon[103389]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.1 MiB/s wr, 627 op/s 2023-10-08T09:25:30.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:30 smithi049 ceph-mon[99485]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.2 MiB/s wr, 545 op/s 2023-10-08T09:25:30.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:30 smithi184 ceph-mon[103675]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.2 MiB/s wr, 545 op/s 2023-10-08T09:25:31.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:30 smithi132 ceph-mon[103389]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.2 MiB/s wr, 545 op/s 2023-10-08T09:25:33.256 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:32 smithi184 ceph-mon[103675]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2023-10-08T09:25:33.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:32 smithi132 ceph-mon[103389]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2023-10-08T09:25:33.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:32 smithi049 ceph-mon[99485]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.8 MiB/s wr, 535 op/s 2023-10-08T09:25:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 610.0 s, 624.2 tps, lat 158.909 ms stddev 165.977 2023-10-08T09:25:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.630+0000 7f60bd994700 1 -- 172.21.15.49:0/3814942299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b810cb30 msgr2=0x7f60b810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.630+0000 7f60bd994700 1 --2- 172.21.15.49:0/3814942299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b810cb30 0x7f60b810cfb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0098f0 tx=0x7f60ac009c00 comp rx=0 tx=0).stop 2023-10-08T09:25:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.655+0000 7f60bd994700 1 -- 172.21.15.49:0/3814942299 shutdown_connections 2023-10-08T09:25:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.655+0000 7f60bd994700 1 --2- 172.21.15.49:0/3814942299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b81077f0 0x7f60b8109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.655+0000 7f60bd994700 1 --2- 172.21.15.49:0/3814942299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b810cb30 0x7f60b810cfb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.655+0000 7f60bd994700 1 --2- 172.21.15.49:0/3814942299 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b810a160 0x7f60b810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.655+0000 7f60bd994700 1 -- 172.21.15.49:0/3814942299 >> 172.21.15.49:0/3814942299 conn(0x7f60b806d770 msgr2=0x7f60b806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.669+0000 7f60bd994700 1 -- 172.21.15.49:0/3814942299 shutdown_connections 2023-10-08T09:25:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.670+0000 7f60bd994700 1 -- 172.21.15.49:0/3814942299 wait complete. 2023-10-08T09:25:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.670+0000 7f60bd994700 1 Processor -- start 2023-10-08T09:25:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.677+0000 7f60bd994700 1 -- start start 2023-10-08T09:25:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60bd994700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81077f0 0x7f60b81a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60bd994700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b810a160 0x7f60b81a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60bd994700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b810cb30 0x7f60b81ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60bd994700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b81acfc0 con 0x7f60b810cb30 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60bd994700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b810ef20 con 0x7f60b81077f0 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60bd994700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b810f0b0 con 0x7f60b810a160 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b810a160 0x7f60b81a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b810a160 0x7f60b81a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40142/0 (socket says 172.21.15.49:40142) 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 -- 172.21.15.49:0/1555937417 learned_addr learned my addr 172.21.15.49:0/1555937417 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 -- 172.21.15.49:0/1555937417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81077f0 msgr2=0x7f60b81a5490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81077f0 0x7f60b81a5490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 -- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b810cb30 msgr2=0x7f60b81ac9f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b810cb30 0x7f60b81ac9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.678+0000 7f60b67fc700 1 -- 172.21.15.49:0/1555937417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac009520 con 0x7f60b810a160 2023-10-08T09:25:34.680 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.679+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b810a160 0x7f60b81a59d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d980 tx=0x7f60a800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.679+0000 7f60bc992700 1 -- 172.21.15.49:0/1555937417 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8009b40 con 0x7f60b810a160 2023-10-08T09:25:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.679+0000 7f60bc992700 1 -- 172.21.15.49:0/1555937417 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a800de60 con 0x7f60b810a160 2023-10-08T09:25:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.679+0000 7f60bc992700 1 -- 172.21.15.49:0/1555937417 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a800f620 con 0x7f60b810a160 2023-10-08T09:25:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.679+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b810f390 con 0x7f60b810a160 2023-10-08T09:25:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.679+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b810f850 con 0x7f60b810a160 2023-10-08T09:25:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.685+0000 7f60bc992700 1 -- 172.21.15.49:0/1555937417 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f60a800f780 con 0x7f60b810a160 2023-10-08T09:25:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.686+0000 7f60bc992700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60a00a15d0 0x7f60a00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.686+0000 7f60bc992700 1 -- 172.21.15.49:0/1555937417 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f60a80ccf00 con 0x7f60b810a160 2023-10-08T09:25:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.686+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60a00a15d0 0x7f60a00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.686+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60a00a15d0 0x7f60a00a3a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f60b0007fa0 tx=0x7f60b00074f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.687+0000 7f60c9b7db80 1 Processor -- start 2023-10-08T09:25:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.687+0000 7f60c9b7db80 1 -- start start 2023-10-08T09:25:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f4e40 0x55bad77f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad77f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad775d400 con 0x55bad77f62b0 2023-10-08T09:25:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad774ebc0 con 0x55bad77f4e40 2023-10-08T09:25:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad774c4e0 con 0x55bad77f57a0 2023-10-08T09:25:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40150/0 (socket says 172.21.15.49:40150) 2023-10-08T09:25:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 -- 172.21.15.49:0/1285782353 learned_addr learned my addr 172.21.15.49:0/1285782353 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f4e40 0x55bad77f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b77fe700 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad77f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 -- 172.21.15.49:0/1285782353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f4e40 msgr2=0x55bad77f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f4e40 0x55bad77f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 -- 172.21.15.49:0/1285782353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 msgr2=0x55bad77f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad77f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.688+0000 7f60b67fc700 1 -- 172.21.15.49:0/1285782353 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad7572d80 con 0x55bad77f57a0 2023-10-08T09:25:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.695+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77f5cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f60a8099b90 tx=0x7f60a800dfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9ca33762cd54fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.697+0000 7f609dffb700 1 -- 172.21.15.49:0/1285782353 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a809dfe0 con 0x55bad77f57a0 2023-10-08T09:25:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.697+0000 7f609dffb700 1 -- 172.21.15.49:0/1285782353 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a80a2220 con 0x55bad77f57a0 2023-10-08T09:25:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.697+0000 7f609dffb700 1 -- 172.21.15.49:0/1285782353 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a80a1d90 con 0x55bad77f57a0 2023-10-08T09:25:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.708+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1285782353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 msgr2=0x55bad77f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.708+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77f5cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f60a8099b90 tx=0x7f60a800dfd0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.712+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1285782353 shutdown_connections 2023-10-08T09:25:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.712+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad77f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.712+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f4e40 0x55bad77f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.712+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1285782353 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77f5cd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.712+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1285782353 >> 172.21.15.49:0/1285782353 conn(0x55bad7769200 msgr2=0x55bad77ef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.721+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1285782353 shutdown_connections 2023-10-08T09:25:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.721+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1285782353 wait complete. 2023-10-08T09:25:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.722+0000 7f60c9b7db80 1 -- start start 2023-10-08T09:25:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.722+0000 7f60c9b7db80 1 Processor -- start 2023-10-08T09:25:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f4e40 0x55bad77fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad77feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad7805f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad7747dd0 con 0x55bad77f62b0 2023-10-08T09:25:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad76e46b0 con 0x55bad77f57a0 2023-10-08T09:25:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad76e4350 con 0x55bad77f4e40 2023-10-08T09:25:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60c9b7db80 10 client.0 _ll_register_callbacks cb 0x55bad75417a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60b6ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f4e40 0x55bad77fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60b6ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f4e40 0x55bad77fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40154/0 (socket says 172.21.15.49:40154) 2023-10-08T09:25:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.724+0000 7f60b6ffd700 1 -- 172.21.15.49:0/1345558895 learned_addr learned my addr 172.21.15.49:0/1345558895 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f60b6ffd700 1 -- 172.21.15.49:0/1345558895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 msgr2=0x55bad77feec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:25:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad77feec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f60b6ffd700 1 -- 172.21.15.49:0/1345558895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 msgr2=0x55bad7805f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad7805f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f60b6ffd700 1 -- 172.21.15.49:0/1345558895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b810f850 con 0x55bad77f4e40 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f4e40 0x55bad77fe980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f60b0015bf0 tx=0x7f60b0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8ca9a2c779ec9a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f6096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f6096ffd700 1 -- 172.21.15.49:0/1345558895 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0024070 con 0x55bad77f4e40 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bad770fe50 con 0x55bad77f4e40 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.725+0000 7f6096ffd700 1 -- 172.21.15.49:0/1345558895 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f60b0004bb0 con 0x55bad77f4e40 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f6096ffd700 1 -- 172.21.15.49:0/1345558895 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b000f070 con 0x55bad77f4e40 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 10 client.26479 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55bad755bc00 con 0x55bad77f4e40 2023-10-08T09:25:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f6096ffd700 1 -- 172.21.15.49:0/1345558895 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bad770fe50 con 0x55bad77f4e40 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f6096ffd700 1 -- 172.21.15.49:0/1345558895 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f60b0027020 con 0x55bad77f4e40 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 10 client.26479 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 10 client.26479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 20 client.26479 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad780a8c0 0x55bad780cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 4 client.26479 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.726+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bad7740db0 con 0x55bad780a8c0 2023-10-08T09:25:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.727+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad780a8c0 0x55bad780cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.737+0000 7f60b67fc700 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad780a8c0 0x55bad780cd80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fded5bb9dcb19d1 server_cookie=e0b7e4f28f53fe3d in_seq=0 out_seq=0 2023-10-08T09:25:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.737+0000 7f6096ffd700 10 client.26479 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.738+0000 7f6096ffd700 1 -- 172.21.15.49:0/1345558895 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bad7740db0 con 0x55bad780a8c0 2023-10-08T09:25:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.738+0000 7f6096ffd700 10 client.26479 handle_command_reply: tid=0 2023-10-08T09:25:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.738+0000 7f60c9b7db80 1 client.26479 shutdown 2023-10-08T09:25:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.738+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f4e40 msgr2=0x55bad77fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.738+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f4e40 0x55bad77fe980 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f60b0015bf0 tx=0x7f60b0004b40 comp rx=0 tx=0).stop 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.738+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 shutdown_connections 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.738+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad780a8c0 0x55bad780cd80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.739+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f4e40 0x55bad77fe980 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f60b0015bf0 tx=0x7f60b0004b40 comp rx=0 tx=0).stop 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.739+0000 7f6096ffd700 0 client.26479 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.739+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad7805f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.739+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/1345558895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad77feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.739+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 >> 172.21.15.49:0/1345558895 conn(0x55bad7769200 msgr2=0x55bad77ef010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.739+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 shutdown_connections 2023-10-08T09:25:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.740+0000 7f60c9b7db80 1 -- 172.21.15.49:0/1345558895 wait complete. 2023-10-08T09:25:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.740+0000 7f60c9b7db80 20 client.26479 trim_cache size 0 max 16384 2023-10-08T09:25:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.770+0000 7f60c9b7db80 1 Processor -- start 2023-10-08T09:25:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.770+0000 7f60c9b7db80 1 -- start start 2023-10-08T09:25:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.771+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77fefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.771+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f62b0 0x55bad78065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.771+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f6690 0x55bad7805e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.771+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad75c88c0 con 0x55bad77f6690 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.771+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad7807c90 con 0x55bad77f62b0 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.771+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad78070d0 con 0x55bad77f57a0 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.774+0000 7f60b77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f6690 0x55bad7805e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.774+0000 7f60b77fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f6690 0x55bad7805e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53124/0 (socket says 172.21.15.49:53124) 2023-10-08T09:25:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.774+0000 7f60b77fe700 1 -- 172.21.15.49:0/2733773570 learned_addr learned my addr 172.21.15.49:0/2733773570 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.781+0000 7f60b67fc700 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f62b0 0x55bad78065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.782+0000 7f60b77fe700 1 -- 172.21.15.49:0/2733773570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 msgr2=0x55bad77fefc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:25:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.782+0000 7f60b77fe700 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77fefc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.782+0000 7f60b77fe700 1 -- 172.21.15.49:0/2733773570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f62b0 msgr2=0x55bad78065b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.782+0000 7f60b77fe700 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f62b0 0x55bad78065b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.782+0000 7f60b77fe700 1 -- 172.21.15.49:0/2733773570 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bad75991f0 con 0x55bad77f6690 2023-10-08T09:25:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.782+0000 7f60b77fe700 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f6690 0x55bad7805e90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00b2e0 tx=0x7f60ac010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462f20e8e7780cf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.784+0000 7f609e7fc700 1 -- 172.21.15.49:0/2733773570 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00f720 con 0x55bad77f6690 2023-10-08T09:25:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.784+0000 7f609e7fc700 1 -- 172.21.15.49:0/2733773570 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60ac01d040 con 0x55bad77f6690 2023-10-08T09:25:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.784+0000 7f609e7fc700 1 -- 172.21.15.49:0/2733773570 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00bbc0 con 0x55bad77f6690 2023-10-08T09:25:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.785+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2733773570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f6690 msgr2=0x55bad7805e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.785+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f6690 0x55bad7805e90 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00b2e0 tx=0x7f60ac010e70 comp rx=0 tx=0).stop 2023-10-08T09:25:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.785+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2733773570 shutdown_connections 2023-10-08T09:25:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.785+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f57a0 0x55bad77fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.785+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f6690 0x55bad7805e90 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.785+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2733773570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f62b0 0x55bad78065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.785+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2733773570 >> 172.21.15.49:0/2733773570 conn(0x55bad7769200 msgr2=0x55bad7811720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.798+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2733773570 shutdown_connections 2023-10-08T09:25:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.803+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2733773570 wait complete. 2023-10-08T09:25:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.803+0000 7f60c9b7db80 1 -- start start 2023-10-08T09:25:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.803+0000 7f60c9b7db80 1 Processor -- start 2023-10-08T09:25:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.809+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad78058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.809+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad7805df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.809+0000 7f60c9b7db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f6690 0x55bad7803f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.809+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad7804560 con 0x55bad77f62b0 2023-10-08T09:25:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.809+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad78046d0 con 0x55bad77f57a0 2023-10-08T09:25:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.809+0000 7f60c9b7db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bad7804890 con 0x55bad77f6690 2023-10-08T09:25:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.809+0000 7f60c9b7db80 10 client.0 _ll_register_callbacks cb 0x7f60b00157d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.810+0000 7f60b77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f6690 0x55bad7803f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.812+0000 7f60b6ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad78058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.812+0000 7f60b77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f6690 0x55bad7803f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40166/0 (socket says 172.21.15.49:40166) 2023-10-08T09:25:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.812+0000 7f60b77fe700 1 -- 172.21.15.49:0/2590072655 learned_addr learned my addr 172.21.15.49:0/2590072655 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.820+0000 7f60b67fc700 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad7805df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.824+0000 7f60b6ffd700 1 -- 172.21.15.49:0/2590072655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f6690 msgr2=0x55bad7803f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.824+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f6690 0x55bad7803f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.824+0000 7f60b6ffd700 1 -- 172.21.15.49:0/2590072655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 msgr2=0x55bad7805df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.824+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad7805df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.824+0000 7f60b6ffd700 1 -- 172.21.15.49:0/2590072655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b810da70 con 0x55bad77f57a0 2023-10-08T09:25:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.825+0000 7f60b6ffd700 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad78058b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55bad780a240 tx=0x7f60b0020eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3f5ca10aba8d96 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.825+0000 7f6096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:25:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.825+0000 7f6096ffd700 1 -- 172.21.15.49:0/2590072655 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0024040 con 0x55bad77f57a0 2023-10-08T09:25:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.825+0000 7f6096ffd700 1 -- 172.21.15.49:0/2590072655 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f60b0004e40 con 0x55bad77f57a0 2023-10-08T09:25:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.830+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60a80a2220 con 0x55bad77f57a0 2023-10-08T09:25:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.840+0000 7f6096ffd700 1 -- 172.21.15.49:0/2590072655 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b0014a70 con 0x55bad77f57a0 2023-10-08T09:25:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.846+0000 7f60c9b7db80 10 client.26417 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.846+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55bad7571250 con 0x55bad77f57a0 2023-10-08T09:25:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.847+0000 7f6096ffd700 1 -- 172.21.15.49:0/2590072655 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a80a2220 con 0x55bad77f57a0 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.847+0000 7f6096ffd700 1 -- 172.21.15.49:0/2590072655 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f60b0039070 con 0x55bad77f57a0 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.850+0000 7f60c9b7db80 10 client.26417 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.850+0000 7f60c9b7db80 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.850+0000 7f60c9b7db80 20 client.26417 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.850+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad785faa0 0x55bad7861ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.850+0000 7f60c9b7db80 4 client.26417 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.850+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bad7740ad0 con 0x55bad785faa0 2023-10-08T09:25:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.850+0000 7f60b67fc700 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad785faa0 0x55bad7861ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.853+0000 7f60b67fc700 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad785faa0 0x55bad7861ec0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83be160fda3ea1a8 server_cookie=6f9521ee917a9524 in_seq=0 out_seq=0 2023-10-08T09:25:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.853+0000 7f6096ffd700 10 client.26417 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.877+0000 7f6096ffd700 1 -- 172.21.15.49:0/2590072655 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bad7740ad0 con 0x55bad785faa0 2023-10-08T09:25:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.877+0000 7f6096ffd700 10 client.26417 handle_command_reply: tid=0 2023-10-08T09:25:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.877+0000 7f60c9b7db80 1 client.26417 shutdown 2023-10-08T09:25:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 msgr2=0x55bad78058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad78058b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55bad780a240 tx=0x7f60b0020eb0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 shutdown_connections 2023-10-08T09:25:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bad785faa0 0x55bad7861ec0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bad77f6690 0x55bad7803f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bad77f57a0 0x55bad78058b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 --2- 172.21.15.49:0/2590072655 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bad77f62b0 0x55bad7805df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f6096ffd700 0 client.26417 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 >> 172.21.15.49:0/2590072655 conn(0x55bad7769200 msgr2=0x55bad77fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 shutdown_connections 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.885+0000 7f60c9b7db80 1 -- 172.21.15.49:0/2590072655 wait complete. 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60c9b7db80 20 client.26417 trim_cache size 0 max 16384 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:25:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:25:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:25:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60a00a15d0 msgr2=0x7f60a00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60a00a15d0 0x7f60a00a3a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f60b0007fa0 tx=0x7f60b00074f0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b810a160 msgr2=0x7f60b81a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b810a160 0x7f60b81a59d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d980 tx=0x7f60a800dd40 comp rx=0 tx=0).stop 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 shutdown_connections 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f60a00a15d0 0x7f60a00a3a90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b810cb30 0x7f60b81ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60b81077f0 0x7f60b81a5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 --2- 172.21.15.49:0/1555937417 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b810a160 0x7f60b81a59d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 >> 172.21.15.49:0/1555937417 conn(0x7f60b806d770 msgr2=0x7f60b8108df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.886+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 shutdown_connections 2023-10-08T09:25:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:34.887+0000 7f60bd994700 1 -- 172.21.15.49:0/1555937417 wait complete. 2023-10-08T09:25:35.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:35 smithi132 ceph-mon[103389]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.6 MiB/s wr, 589 op/s 2023-10-08T09:25:35.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:35 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:25:35.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:35 smithi049 ceph-mon[99485]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.6 MiB/s wr, 589 op/s 2023-10-08T09:25:35.890 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:35 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:25:36.001 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:35 smithi184 ceph-mon[103675]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.6 MiB/s wr, 589 op/s 2023-10-08T09:25:36.002 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:35 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:25:37.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:37 smithi132 ceph-mon[103389]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.4 MiB/s wr, 562 op/s 2023-10-08T09:25:37.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:37 smithi049 ceph-mon[99485]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.4 MiB/s wr, 562 op/s 2023-10-08T09:25:38.000 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:37 smithi184 ceph-mon[103675]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.4 MiB/s wr, 562 op/s 2023-10-08T09:25:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 615.0 s, 484.4 tps, lat 203.794 ms stddev 228.154 2023-10-08T09:25:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:39 smithi049 ceph-mon[99485]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 5.8 MiB/s wr, 607 op/s 2023-10-08T09:25:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:39.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:39 smithi184 ceph-mon[103675]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 5.8 MiB/s wr, 607 op/s 2023-10-08T09:25:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:39 smithi132 ceph-mon[103389]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 5.8 MiB/s wr, 607 op/s 2023-10-08T09:25:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:41 smithi132 ceph-mon[103389]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.7 MiB/s wr, 512 op/s 2023-10-08T09:25:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.390 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:41 smithi049 ceph-mon[99485]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.7 MiB/s wr, 512 op/s 2023-10-08T09:25:41.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:41.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:41 smithi184 ceph-mon[103675]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.7 MiB/s wr, 512 op/s 2023-10-08T09:25:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:42 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:25:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:42 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:25:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:42 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:42 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:25:42.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:42 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:25:42.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:42 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:42 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:25:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:42 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:25:42.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:42 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:25:43.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:43 smithi049 ceph-mon[99485]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.1 MiB/s wr, 535 op/s 2023-10-08T09:25:43.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:43 smithi184 ceph-mon[103675]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.1 MiB/s wr, 535 op/s 2023-10-08T09:25:43.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:43 smithi132 ceph-mon[103389]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.1 MiB/s wr, 535 op/s 2023-10-08T09:25:44.316 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 620.0 s, 440.4 tps, lat 224.305 ms stddev 229.994 2023-10-08T09:25:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:45 smithi049 ceph-mon[99485]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2023-10-08T09:25:45.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:45 smithi184 ceph-mon[103675]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2023-10-08T09:25:46.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:45 smithi132 ceph-mon[103389]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2023-10-08T09:25:46.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:46 smithi184 ceph-mon[103675]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.1 MiB/s wr, 539 op/s 2023-10-08T09:25:47.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:46 smithi132 ceph-mon[103389]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.1 MiB/s wr, 539 op/s 2023-10-08T09:25:47.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:46 smithi049 ceph-mon[99485]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.1 MiB/s wr, 539 op/s 2023-10-08T09:25:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.213+0000 7fa464993700 1 -- 172.21.15.49:0/149658959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4601077f0 msgr2=0x7fa460109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.213+0000 7fa464993700 1 --2- 172.21.15.49:0/149658959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4601077f0 0x7fa460109c20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa450009990 tx=0x7fa450009ca0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.214+0000 7fa464993700 1 -- 172.21.15.49:0/149658959 shutdown_connections 2023-10-08T09:25:48.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.214+0000 7fa464993700 1 --2- 172.21.15.49:0/149658959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4601077f0 0x7fa460109c20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.214+0000 7fa464993700 1 --2- 172.21.15.49:0/149658959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa46010cb30 0x7fa46010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.214+0000 7fa464993700 1 --2- 172.21.15.49:0/149658959 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa46010a160 0x7fa46010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.214+0000 7fa464993700 1 -- 172.21.15.49:0/149658959 >> 172.21.15.49:0/149658959 conn(0x7fa46006d770 msgr2=0x7fa46006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.218+0000 7fa464993700 1 -- 172.21.15.49:0/149658959 shutdown_connections 2023-10-08T09:25:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.218+0000 7fa464993700 1 -- 172.21.15.49:0/149658959 wait complete. 2023-10-08T09:25:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 Processor -- start 2023-10-08T09:25:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 -- start start 2023-10-08T09:25:48.220 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4601077f0 0x7fa460118190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46010cb30 0x7fa4601186d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4601c4480 0x7fa4601c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4601c6e40 con 0x7fa4601c4480 2023-10-08T09:25:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4601c6f80 con 0x7fa4601077f0 2023-10-08T09:25:48.221 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa464993700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4601c70f0 con 0x7fa46010cb30 2023-10-08T09:25:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa45ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4601c4480 0x7fa4601c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa45ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4601c4480 0x7fa4601c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53146/0 (socket says 172.21.15.49:53146) 2023-10-08T09:25:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa45ed9d700 1 -- 172.21.15.49:0/2483654307 learned_addr learned my addr 172.21.15.49:0/2483654307 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.219+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46010cb30 0x7fa4601186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa45ed9d700 1 -- 172.21.15.49:0/2483654307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46010cb30 msgr2=0x7fa4601186d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa45ed9d700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46010cb30 0x7fa4601186d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa45ed9d700 1 -- 172.21.15.49:0/2483654307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4601077f0 msgr2=0x7fa460118190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa45ed9d700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4601077f0 0x7fa460118190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa45ed9d700 1 -- 172.21.15.49:0/2483654307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45801ac50 con 0x7fa4601c4480 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa45ed9d700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4601c4480 0x7fa4601c6870 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa458007800 tx=0x7fa458007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa44f7fe700 1 -- 172.21.15.49:0/2483654307 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458009250 con 0x7fa4601c4480 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4500095f0 con 0x7fa4601c4480 2023-10-08T09:25:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4601c76f0 con 0x7fa4601c4480 2023-10-08T09:25:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa44f7fe700 1 -- 172.21.15.49:0/2483654307 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45801a3a0 con 0x7fa4601c4480 2023-10-08T09:25:48.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.220+0000 7fa44f7fe700 1 -- 172.21.15.49:0/2483654307 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580047a0 con 0x7fa4601c4480 2023-10-08T09:25:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.223+0000 7fa44f7fe700 1 -- 172.21.15.49:0/2483654307 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa458018510 con 0x7fa4601c4480 2023-10-08T09:25:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.223+0000 7fa44f7fe700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4480a1480 0x7fa4480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.223+0000 7fa44f7fe700 1 -- 172.21.15.49:0/2483654307 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa4580cdbe0 con 0x7fa4601c4480 2023-10-08T09:25:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.225+0000 7fa470b7cb80 1 Processor -- start 2023-10-08T09:25:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.225+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4480a1480 0x7fa4480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa470b7cb80 1 -- start start 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7141260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba70a9400 con 0x55aba7140e40 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba709abc0 con 0x55aba71417a0 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba70984e0 con 0x55aba71422b0 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa45dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa45dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7141cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42766/0 (socket says 172.21.15.49:42766) 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2724565083 learned_addr learned my addr 172.21.15.49:0/2724565083 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4480a1480 0x7fa4480a3940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa4500098e0 tx=0x7fa450005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:48.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.226+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.227+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2724565083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 msgr2=0x55aba7142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.227+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.227+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2724565083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 msgr2=0x55aba7141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.227+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.227+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2724565083 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba6ebed80 con 0x55aba71417a0 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.227+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.229+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7141cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa454009a20 tx=0x7fa454009d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f45b0cc13d67ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.229+0000 7fa44cff9700 1 -- 172.21.15.49:0/2724565083 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45400e8d0 con 0x55aba71417a0 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.229+0000 7fa44cff9700 1 -- 172.21.15.49:0/2724565083 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa454013470 con 0x55aba71417a0 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.229+0000 7fa44cff9700 1 -- 172.21.15.49:0/2724565083 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454012610 con 0x55aba71417a0 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.230+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2724565083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 msgr2=0x55aba7141cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.230+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7141cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa454009a20 tx=0x7fa454009d30 comp rx=0 tx=0).stop 2023-10-08T09:25:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.231+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2724565083 shutdown_connections 2023-10-08T09:25:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.231+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.231+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.231+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2724565083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7141cd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.231+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2724565083 >> 172.21.15.49:0/2724565083 conn(0x55aba70b5200 msgr2=0x55aba713b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.232+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2724565083 shutdown_connections 2023-10-08T09:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.235+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2724565083 wait complete. 2023-10-08T09:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.235+0000 7fa470b7cb80 1 -- start start 2023-10-08T09:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.235+0000 7fa470b7cb80 1 Processor -- start 2023-10-08T09:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba7140e40 0x55aba714a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba714add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba71422b0 0x55aba7151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba7093dd0 con 0x55aba71422b0 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba70306b0 con 0x55aba71417a0 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba7030350 con 0x55aba7140e40 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa470b7cb80 10 client.0 _ll_register_callbacks cb 0x55aba6e8d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa45dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba714add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa45dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba714add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42774/0 (socket says 172.21.15.49:42774) 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.236+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2297579667 learned_addr learned my addr 172.21.15.49:0/2297579667 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2297579667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba7140e40 msgr2=0x55aba714a890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba7140e40 0x55aba714a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2297579667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba71422b0 msgr2=0x55aba7151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba71422b0 0x55aba7151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa45dd9b700 1 -- 172.21.15.49:0/2297579667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa454009650 con 0x55aba71417a0 2023-10-08T09:25:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba714add0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa45400a010 tx=0x7fa45400eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70086124ac237d36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa445ffb700 1 -- 172.21.15.49:0/2297579667 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454013be0 con 0x55aba71417a0 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.237+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aba705be50 con 0x55aba71417a0 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa445ffb700 1 -- 172.21.15.49:0/2297579667 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa45400ee30 con 0x55aba71417a0 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa445ffb700 1 -- 172.21.15.49:0/2297579667 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa454006d10 con 0x55aba71417a0 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 10 client.26426 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55aba6ea7c00 con 0x55aba71417a0 2023-10-08T09:25:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa445ffb700 1 -- 172.21.15.49:0/2297579667 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aba705be50 con 0x55aba71417a0 2023-10-08T09:25:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa445ffb700 1 -- 172.21.15.49:0/2297579667 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa454011070 con 0x55aba71417a0 2023-10-08T09:25:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 10 client.26426 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 10 client.26426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 20 client.26426 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba7156710 0x55aba7158b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 4 client.26426 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:25:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.238+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aba708cdb0 con 0x55aba7156710 2023-10-08T09:25:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.239+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba7156710 0x55aba7158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.239+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba7156710 0x55aba7158b30 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a742e043929fb2 server_cookie=7fcea89eb207e64e in_seq=0 out_seq=0 2023-10-08T09:25:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.239+0000 7fa445ffb700 10 client.26426 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.241+0000 7fa445ffb700 1 -- 172.21.15.49:0/2297579667 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aba708cdb0 con 0x55aba7156710 2023-10-08T09:25:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.241+0000 7fa445ffb700 10 client.26426 handle_command_reply: tid=0 2023-10-08T09:25:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.241+0000 7fa470b7cb80 1 client.26426 shutdown 2023-10-08T09:25:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.242+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 msgr2=0x55aba714add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.242+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba714add0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa45400a010 tx=0x7fa45400eb90 comp rx=0 tx=0).stop 2023-10-08T09:25:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 shutdown_connections 2023-10-08T09:25:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba7156710 0x55aba7158b30 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba7140e40 0x55aba714a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba71422b0 0x55aba7151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2297579667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba714add0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 >> 172.21.15.49:0/2297579667 conn(0x55aba70b5200 msgr2=0x55aba713b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa445ffb700 0 client.26426 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 shutdown_connections 2023-10-08T09:25:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2297579667 wait complete. 2023-10-08T09:25:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.243+0000 7fa470b7cb80 20 client.26426 trim_cache size 0 max 16384 2023-10-08T09:25:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.271+0000 7fa470b7cb80 1 Processor -- start 2023-10-08T09:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.275+0000 7fa470b7cb80 1 -- start start 2023-10-08T09:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.275+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7149a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.275+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.275+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.275+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba6f148c0 con 0x55aba7140e40 2023-10-08T09:25:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.275+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba71493b0 con 0x55aba71417a0 2023-10-08T09:25:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.275+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba713c610 con 0x55aba71422b0 2023-10-08T09:25:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.279+0000 7fa45ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.279+0000 7fa45ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40200/0 (socket says 172.21.15.49:40200) 2023-10-08T09:25:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.279+0000 7fa45ed9d700 1 -- 172.21.15.49:0/2712389745 learned_addr learned my addr 172.21.15.49:0/2712389745 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.285+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.285+0000 7fa45e59c700 1 -- 172.21.15.49:0/2712389745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 msgr2=0x55aba7151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.285+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.285+0000 7fa45e59c700 1 -- 172.21.15.49:0/2712389745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 msgr2=0x55aba7149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.285+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.285+0000 7fa45e59c700 1 -- 172.21.15.49:0/2712389745 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba6ee51f0 con 0x55aba7140e40 2023-10-08T09:25:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.286+0000 7fa45e59c700 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7149a70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa450006010 tx=0x7fa450018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575c824f176d780b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.290+0000 7fa44d7fa700 1 -- 172.21.15.49:0/2712389745 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45001d070 con 0x55aba7140e40 2023-10-08T09:25:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.290+0000 7fa44d7fa700 1 -- 172.21.15.49:0/2712389745 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa45002a040 con 0x55aba7140e40 2023-10-08T09:25:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.290+0000 7fa44d7fa700 1 -- 172.21.15.49:0/2712389745 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45000b7e0 con 0x55aba7140e40 2023-10-08T09:25:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.291+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2712389745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 msgr2=0x55aba7149a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.291+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7149a70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa450006010 tx=0x7fa450018000 comp rx=0 tx=0).stop 2023-10-08T09:25:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.293+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2712389745 shutdown_connections 2023-10-08T09:25:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.293+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71422b0 0x55aba7151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.293+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba7149a70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.293+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/2712389745 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71417a0 0x55aba7149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.293+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2712389745 >> 172.21.15.49:0/2712389745 conn(0x55aba70b5200 msgr2=0x55aba715d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.294+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2712389745 shutdown_connections 2023-10-08T09:25:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.294+0000 7fa470b7cb80 1 -- 172.21.15.49:0/2712389745 wait complete. 2023-10-08T09:25:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.294+0000 7fa470b7cb80 1 -- start start 2023-10-08T09:25:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.294+0000 7fa470b7cb80 1 Processor -- start 2023-10-08T09:25:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.294+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba714f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71417a0 0x55aba714fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa470b7cb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71422b0 0x55aba7150d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba71512b0 con 0x55aba7140e40 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba7151420 con 0x55aba71422b0 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa470b7cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba71515e0 con 0x55aba71417a0 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa470b7cb80 10 client.0 _ll_register_callbacks cb 0x7fa454009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba714f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba714f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53170/0 (socket says 172.21.15.49:53170) 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 -- 172.21.15.49:0/1884568251 learned_addr learned my addr 172.21.15.49:0/1884568251 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45ed9d700 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71422b0 0x55aba7150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71417a0 0x55aba714fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 -- 172.21.15.49:0/1884568251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71417a0 msgr2=0x55aba714fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71417a0 0x55aba714fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 -- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71422b0 msgr2=0x55aba7150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71422b0 0x55aba7150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.295+0000 7fa45e59c700 1 -- 172.21.15.49:0/1884568251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46010da70 con 0x55aba7140e40 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa45e59c700 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba714f820 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa450009350 tx=0x7fa45002f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edebedde81134ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa445ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa445ffb700 1 -- 172.21.15.49:0/1884568251 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45001d040 con 0x55aba7140e40 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa445ffb700 1 -- 172.21.15.49:0/1884568251 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa45000bdf0 con 0x55aba7140e40 2023-10-08T09:25:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa454013470 con 0x55aba7140e40 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa445ffb700 1 -- 172.21.15.49:0/1884568251 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa450003680 con 0x55aba7140e40 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa470b7cb80 10 client.16836 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.296+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55aba6ebd250 con 0x55aba7140e40 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa445ffb700 1 -- 172.21.15.49:0/1884568251 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa454013470 con 0x55aba7140e40 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa445ffb700 1 -- 172.21.15.49:0/1884568251 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa450041070 con 0x55aba7140e40 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa470b7cb80 10 client.16836 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa470b7cb80 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa470b7cb80 20 client.16836 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba71aabc0 0x55aba71acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa470b7cb80 4 client.16836 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aba708cad0 con 0x55aba71aabc0 2023-10-08T09:25:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.297+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba71aabc0 0x55aba71acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.314+0000 7fa45dd9b700 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba71aabc0 0x55aba71acfe0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe89a2d8023a519 server_cookie=2b7e5769b940f0e8 in_seq=0 out_seq=0 2023-10-08T09:25:48.318 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.314+0000 7fa445ffb700 10 client.16836 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.360+0000 7fa445ffb700 1 -- 172.21.15.49:0/1884568251 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aba708cad0 con 0x55aba71aabc0 2023-10-08T09:25:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.360+0000 7fa445ffb700 10 client.16836 handle_command_reply: tid=0 2023-10-08T09:25:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.362+0000 7fa470b7cb80 1 client.16836 shutdown 2023-10-08T09:25:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.367+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 msgr2=0x55aba714f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.367+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba714f820 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa450009350 tx=0x7fa45002f5a0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 shutdown_connections 2023-10-08T09:25:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55aba71aabc0 0x55aba71acfe0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55aba71422b0 0x55aba7150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55aba7140e40 0x55aba714f820 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa445ffb700 0 client.16836 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 --2- 172.21.15.49:0/1884568251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55aba71417a0 0x55aba714fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 >> 172.21.15.49:0/1884568251 conn(0x55aba70b5200 msgr2=0x55aba7149a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 shutdown_connections 2023-10-08T09:25:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 1 -- 172.21.15.49:0/1884568251 wait complete. 2023-10-08T09:25:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.368+0000 7fa470b7cb80 20 client.16836 trim_cache size 0 max 16384 2023-10-08T09:25:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:25:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:25:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:25:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:25:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.379+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4480a1480 msgr2=0x7fa4480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.379+0000 7fa464993700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4480a1480 0x7fa4480a3940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa4500098e0 tx=0x7fa450005c00 comp rx=0 tx=0).stop 2023-10-08T09:25:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.379+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4601c4480 msgr2=0x7fa4601c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.379+0000 7fa464993700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4601c4480 0x7fa4601c6870 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa458007800 tx=0x7fa458007b10 comp rx=0 tx=0).stop 2023-10-08T09:25:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.381+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 shutdown_connections 2023-10-08T09:25:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.381+0000 7fa464993700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa4480a1480 0x7fa4480a3940 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa4500098e0 tx=0x7fa450005c00 comp rx=0 tx=0).stop 2023-10-08T09:25:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.381+0000 7fa464993700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4601c4480 0x7fa4601c6870 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.381+0000 7fa464993700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4601077f0 0x7fa460118190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.381+0000 7fa464993700 1 --2- 172.21.15.49:0/2483654307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa46010cb30 0x7fa4601186d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.381+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 >> 172.21.15.49:0/2483654307 conn(0x7fa46006d770 msgr2=0x7fa460108130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.386+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 shutdown_connections 2023-10-08T09:25:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:25:48.387+0000 7fa464993700 1 -- 172.21.15.49:0/2483654307 wait complete. 2023-10-08T09:25:49.002 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:25:49.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.261+0000 7f200548d700 1 -- 172.21.15.49:0/2627489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010a160 msgr2=0x7f200010c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.261+0000 7f200548d700 1 --2- 172.21.15.49:0/2627489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010a160 0x7f200010c5f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00098f0 tx=0x7f1ff0009c00 comp rx=0 tx=0).stop 2023-10-08T09:25:49.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.267+0000 7f200548d700 1 -- 172.21.15.49:0/2627489 shutdown_connections 2023-10-08T09:25:49.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.267+0000 7f200548d700 1 --2- 172.21.15.49:0/2627489 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f200010cb30 0x7f200010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.267+0000 7f200548d700 1 --2- 172.21.15.49:0/2627489 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f20001077f0 0x7f2000109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.267+0000 7f200548d700 1 --2- 172.21.15.49:0/2627489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010a160 0x7f200010c5f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.269 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.267+0000 7f200548d700 1 -- 172.21.15.49:0/2627489 >> 172.21.15.49:0/2627489 conn(0x7f200006d790 msgr2=0x7f200006fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:49.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.285+0000 7f200548d700 1 -- 172.21.15.49:0/2627489 shutdown_connections 2023-10-08T09:25:49.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.291+0000 7f200548d700 1 -- 172.21.15.49:0/2627489 wait complete. 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 Processor -- start 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 -- start start 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20001077f0 0x7f2000113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200010a160 0x7f2000114360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010cb30 0x7f20001a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2000114930 con 0x7f200010a160 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2000114aa0 con 0x7f20001077f0 2023-10-08T09:25:49.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.292+0000 7f200548d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20001a4c40 con 0x7f200010cb30 2023-10-08T09:25:49.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.299+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010cb30 0x7f20001a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.299+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010cb30 0x7f20001a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40216/0 (socket says 172.21.15.49:40216) 2023-10-08T09:25:49.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.299+0000 7f1fff7fe700 1 -- 172.21.15.49:0/201722578 learned_addr learned my addr 172.21.15.49:0/201722578 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:49.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.303+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200010a160 0x7f2000114360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.303+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20001077f0 0x7f2000113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.303+0000 7f1ffe7fc700 1 -- 172.21.15.49:0/201722578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010cb30 msgr2=0x7f20001a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.303+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010cb30 0x7f20001a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.303+0000 7f1ffe7fc700 1 -- 172.21.15.49:0/201722578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20001077f0 msgr2=0x7f2000113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.303+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20001077f0 0x7f2000113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.303+0000 7f1ffe7fc700 1 -- 172.21.15.49:0/201722578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff0009520 con 0x7f200010a160 2023-10-08T09:25:49.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.304+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200010a160 0x7f2000114360 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009230 tx=0x7f1ff000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:49.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.304+0000 7f1fe7fff700 1 -- 172.21.15.49:0/201722578 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff001c070 con 0x7f200010a160 2023-10-08T09:25:49.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.304+0000 7f1fe7fff700 1 -- 172.21.15.49:0/201722578 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ff000f7d0 con 0x7f200010a160 2023-10-08T09:25:49.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.304+0000 7f1fe7fff700 1 -- 172.21.15.49:0/201722578 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0003aa0 con 0x7f200010a160 2023-10-08T09:25:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 625.0 s, 467.6 tps, lat 208.072 ms stddev 235.045 2023-10-08T09:25:49.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.310+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20001a4ec0 con 0x7f200010a160 2023-10-08T09:25:49.314 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.310+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20001a5320 con 0x7f200010a160 2023-10-08T09:25:49.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.316+0000 7f1fe7fff700 1 -- 172.21.15.49:0/201722578 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1ff0020030 con 0x7f200010a160 2023-10-08T09:25:49.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.316+0000 7f2011676b80 1 Processor -- start 2023-10-08T09:25:49.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f1fe7fff700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1fe80a1650 0x7f1fe80a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f1fe7fff700 1 -- 172.21.15.49:0/201722578 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1ff00cdd80 con 0x7f200010a160 2023-10-08T09:25:49.320 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f2011676b80 1 -- start start 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f2011676b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1d80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f2011676b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1d81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f2011676b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1d81c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f2011676b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1ce8460 con 0x563ef1d81800 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f2011676b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1cd9c20 con 0x563ef1d809c0 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.317+0000 7f2011676b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1cd7540 con 0x563ef1d80060 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.318+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1fe80a1650 0x7f1fe80a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.318+0000 7f1ffe7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1d81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.318+0000 7f1ffe7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1d81270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42804/0 (socket says 172.21.15.49:42804) 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.318+0000 7f1ffe7fc700 1 -- 172.21.15.49:0/3362944348 learned_addr learned my addr 172.21.15.49:0/3362944348 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:49.321 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.318+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1d81c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.319+0000 7f1fff7fe700 1 -- 172.21.15.49:0/3362944348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 msgr2=0x563ef1d80480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.319+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1d80480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.319+0000 7f1fff7fe700 1 -- 172.21.15.49:0/3362944348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 msgr2=0x563ef1d81270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.319+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1d81270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.319+0000 7f1fff7fe700 1 -- 172.21.15.49:0/3362944348 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef1afdde0 con 0x563ef1d81800 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.319+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1fe80a1650 0x7f1fe80a3b10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40156e0 tx=0x7f1ff40099d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.319+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1d81c90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4018690 tx=0x7f1ff401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a47979f3b6e3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.322+0000 7f1fe57fa700 1 -- 172.21.15.49:0/3362944348 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4022650 con 0x563ef1d81800 2023-10-08T09:25:49.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.322+0000 7f1fe57fa700 1 -- 172.21.15.49:0/3362944348 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ff4022c70 con 0x563ef1d81800 2023-10-08T09:25:49.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.322+0000 7f1fe57fa700 1 -- 172.21.15.49:0/3362944348 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4017a20 con 0x563ef1d81800 2023-10-08T09:25:49.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 -- 172.21.15.49:0/3362944348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 msgr2=0x563ef1d81c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1d81c90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4018690 tx=0x7f1ff401a000 comp rx=0 tx=0).stop 2023-10-08T09:25:49.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 -- 172.21.15.49:0/3362944348 shutdown_connections 2023-10-08T09:25:49.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1d80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1d81c90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 --2- 172.21.15.49:0/3362944348 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1d81270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 -- 172.21.15.49:0/3362944348 >> 172.21.15.49:0/3362944348 conn(0x563ef1cf4260 msgr2=0x563ef1d7ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 -- 172.21.15.49:0/3362944348 shutdown_connections 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.323+0000 7f2011676b80 1 -- 172.21.15.49:0/3362944348 wait complete. 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 -- start start 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 Processor -- start 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1dabf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1dac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1cd2e30 con 0x563ef1d81800 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1c6f710 con 0x563ef1d809c0 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1c6f3b0 con 0x563ef1d80060 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f2011676b80 10 client.0 _ll_register_callbacks cb 0x563ef1acc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.324+0000 7f1ffeffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1dabf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1dabf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40234/0 (socket says 172.21.15.49:40234) 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 -- 172.21.15.49:0/178875301 learned_addr learned my addr 172.21.15.49:0/178875301 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 -- 172.21.15.49:0/178875301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 msgr2=0x563ef1dac440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1dac440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 -- 172.21.15.49:0/178875301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 msgr2=0x563ef1db3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.333 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 -- 172.21.15.49:0/178875301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff4018450 con 0x563ef1d80060 2023-10-08T09:25:49.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1dabf00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8004a60 tx=0x7f1ff8007e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=751ee17ba8512094 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:49.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.325+0000 7f1fe2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:25:49.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.330+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/178875301 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800e050 con 0x563ef1d80060 2023-10-08T09:25:49.334 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.330+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/178875301 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1ff8010b00 con 0x563ef1d80060 2023-10-08T09:25:49.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.330+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ef1c9aeb0 con 0x563ef1d80060 2023-10-08T09:25:49.342 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.341+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/178875301 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ff8010cc0 con 0x563ef1d80060 2023-10-08T09:25:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.342+0000 7f2011676b80 10 client.26503 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:49.345 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.342+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563ef1ae6c60 con 0x563ef1d80060 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.346+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/178875301 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ef1c9aeb0 con 0x563ef1d80060 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.346+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/178875301 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ff801f3f0 con 0x563ef1d80060 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.353+0000 7f2011676b80 10 client.26503 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.353+0000 7f2011676b80 10 client.26503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.353+0000 7f2011676b80 20 client.26503 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.353+0000 7f2011676b80 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1db7cd0 0x563ef1dba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.353+0000 7f2011676b80 4 client.26503 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:25:49.356 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.353+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ef1ccbe10 con 0x563ef1db7cd0 2023-10-08T09:25:49.362 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.361+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1db7cd0 0x563ef1dba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.380+0000 7f1fe2ff5700 10 client.26503 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:49.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.380+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1db7cd0 0x563ef1dba0f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b07a1d2180389ab server_cookie=4ed2523068dbeaf8 in_seq=0 out_seq=0 2023-10-08T09:25:49.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.381+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/178875301 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563ef1ccbe10 con 0x563ef1db7cd0 2023-10-08T09:25:49.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.381+0000 7f1fe2ff5700 10 client.26503 handle_command_reply: tid=0 2023-10-08T09:25:49.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.381+0000 7f2011676b80 1 client.26503 shutdown 2023-10-08T09:25:49.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 msgr2=0x563ef1dabf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1dabf00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8004a60 tx=0x7f1ff8007e90 comp rx=0 tx=0).stop 2023-10-08T09:25:49.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 shutdown_connections 2023-10-08T09:25:49.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1db7cd0 0x563ef1dba0f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d80060 0x563ef1dabf00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 --2- 172.21.15.49:0/178875301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d809c0 0x563ef1dac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.388+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 >> 172.21.15.49:0/178875301 conn(0x563ef1cf4260 msgr2=0x563ef1d7a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.392+0000 7f1fe2ff5700 0 client.26503 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.392+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 shutdown_connections 2023-10-08T09:25:49.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.393+0000 7f2011676b80 1 -- 172.21.15.49:0/178875301 wait complete. 2023-10-08T09:25:49.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.405+0000 7f2011676b80 20 client.26503 trim_cache size 0 max 16384 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 Processor -- start 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 -- start start 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d80060 0x563ef1d79a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1db3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1b53920 con 0x563ef1d81800 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1daa8e0 con 0x563ef1d80060 2023-10-08T09:25:49.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.432+0000 7f2011676b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1d810d0 con 0x563ef1d809c0 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1db3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffeffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d80060 0x563ef1d79a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffeffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d80060 0x563ef1d79a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42818/0 (socket says 172.21.15.49:42818) 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53208/0 (socket says 172.21.15.49:53208) 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1db3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40240/0 (socket says 172.21.15.49:40240) 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffeffd700 1 -- 172.21.15.49:0/1431967744 learned_addr learned my addr 172.21.15.49:0/1431967744 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 -- 172.21.15.49:0/1431967744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d80060 msgr2=0x563ef1d79a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d80060 0x563ef1d79a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 -- 172.21.15.49:0/1431967744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 msgr2=0x563ef1db9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:49.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 -- 172.21.15.49:0/1431967744 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef1b24250 con 0x563ef1d809c0 2023-10-08T09:25:49.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1fff7fe700 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:25:49.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d80060 0x563ef1d79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:25:49.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1db3350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ff009ad30 tx=0x7f1ff0019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b60c5292edd3df6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:49.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1fe5ffb700 1 -- 172.21.15.49:0/1431967744 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00b7070 con 0x563ef1d809c0 2023-10-08T09:25:49.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1fe5ffb700 1 -- 172.21.15.49:0/1431967744 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ff009e040 con 0x563ef1d809c0 2023-10-08T09:25:49.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.433+0000 7f1fe5ffb700 1 -- 172.21.15.49:0/1431967744 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0098fc0 con 0x563ef1d809c0 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.434+0000 7f2011676b80 1 -- 172.21.15.49:0/1431967744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 msgr2=0x563ef1db3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.434+0000 7f2011676b80 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1db3350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ff009ad30 tx=0x7f1ff0019040 comp rx=0 tx=0).stop 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.434+0000 7f2011676b80 1 -- 172.21.15.49:0/1431967744 shutdown_connections 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.434+0000 7f2011676b80 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d81800 0x563ef1db9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.434+0000 7f2011676b80 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d80060 0x563ef1d79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.434+0000 7f2011676b80 1 --2- 172.21.15.49:0/1431967744 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1db3350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 -- 172.21.15.49:0/1431967744 >> 172.21.15.49:0/1431967744 conn(0x563ef1cf4260 msgr2=0x563ef1d76f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 -- 172.21.15.49:0/1431967744 shutdown_connections 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 -- 172.21.15.49:0/1431967744 wait complete. 2023-10-08T09:25:49.441 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 -- start start 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 Processor -- start 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d80060 0x563ef1daf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1dafaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d81800 0x563ef1db2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1db3000 con 0x563ef1d80060 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1db3170 con 0x563ef1d81800 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef1db3330 con 0x563ef1d809c0 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f2011676b80 10 client.0 _ll_register_callbacks cb 0x7f1ff8009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f1ffeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d80060 0x563ef1daf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f1ffeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d80060 0x563ef1daf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53210/0 (socket says 172.21.15.49:53210) 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f1ffeffd700 1 -- 172.21.15.49:0/2719479479 learned_addr learned my addr 172.21.15.49:0/2719479479 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:25:49.442 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.435+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1dafaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1ffeffd700 1 -- 172.21.15.49:0/2719479479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 msgr2=0x563ef1dafaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1dafaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1ffeffd700 1 -- 172.21.15.49:0/2719479479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d81800 msgr2=0x563ef1db2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d81800 0x563ef1db2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1ffeffd700 1 -- 172.21.15.49:0/2719479479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ef1ae6c60 con 0x563ef1d80060 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1dafaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1ffeffd700 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d80060 0x563ef1daf560 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8007900 tx=0x7f1ff801fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a46955d16f82b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1fe2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/2719479479 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800e070 con 0x563ef1d80060 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/2719479479 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1ff8010dd0 con 0x563ef1d80060 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ff4022c70 con 0x563ef1d80060 2023-10-08T09:25:49.443 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/2719479479 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ff8031420 con 0x563ef1d80060 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f2011676b80 10 client.16854 fetch_fsmap learned FSMap version 40 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.436+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x563ef1afdde0 con 0x563ef1d80060 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/2719479479 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ff4022c70 con 0x563ef1d80060 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/2719479479 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1ff8007bb0 con 0x563ef1d80060 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f2011676b80 10 client.16854 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f2011676b80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f2011676b80 20 client.16854 populate_metadata read hostname 'smithi049' 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f2011676b80 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1e0d020 0x563ef1e0f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f2011676b80 4 client.16854 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563ef1ccbb30 con 0x563ef1e0d020 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.437+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1e0d020 0x563ef1e0f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:25:49.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f1fe2ff5700 10 client.16854 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f1ffe7fc700 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1e0d020 0x563ef1e0f440 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ca488cef9bd702 server_cookie=d29f39645c67a83 in_seq=0 out_seq=0 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f1fe2ff5700 1 -- 172.21.15.49:0/2719479479 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563ef1ccbb30 con 0x563ef1e0d020 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f1fe2ff5700 10 client.16854 handle_command_reply: tid=0 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 client.16854 shutdown 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d80060 msgr2=0x563ef1daf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d80060 0x563ef1daf560 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8007900 tx=0x7f1ff801fbf0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 shutdown_connections 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x563ef1e0d020 0x563ef1e0f440 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x563ef1d81800 0x563ef1db2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563ef1d80060 0x563ef1daf560 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f1fe2ff5700 0 client.16854 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:25:49.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 --2- 172.21.15.49:0/2719479479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563ef1d809c0 0x563ef1dafaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 >> 172.21.15.49:0/2719479479 conn(0x563ef1cf4260 msgr2=0x563ef1db9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 shutdown_connections 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 1 -- 172.21.15.49:0/2719479479 wait complete. 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.438+0000 7f2011676b80 20 client.16854 trim_cache size 0 max 16384 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:25:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:25:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:25:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:25:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:25:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:25:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:25:49.447 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1fe80a1650 msgr2=0x7f1fe80a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1fe80a1650 0x7f1fe80a3b10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40156e0 tx=0x7f1ff40099d0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200010a160 msgr2=0x7f2000114360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200010a160 0x7f2000114360 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009230 tx=0x7f1ff000b790 comp rx=0 tx=0).stop 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 shutdown_connections 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f1fe80a1650 0x7f1fe80a3b10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f200010cb30 0x7f20001a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20001077f0 0x7f2000113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 --2- 172.21.15.49:0/201722578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f200010a160 0x7f2000114360 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:25:49.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 >> 172.21.15.49:0/201722578 conn(0x7f200006d790 msgr2=0x7f2000107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:25:49.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 shutdown_connections 2023-10-08T09:25:49.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:25:49.439+0000 7f200548d700 1 -- 172.21.15.49:0/201722578 wait complete. 2023-10-08T09:25:49.464 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:25:49.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:49 smithi049 ceph-mon[99485]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 621 op/s 2023-10-08T09:25:49.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:49 smithi184 ceph-mon[103675]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 621 op/s 2023-10-08T09:25:50.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:49 smithi132 ceph-mon[103389]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 621 op/s 2023-10-08T09:25:50.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:50 smithi184 ceph-mon[103675]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.9 MiB/s wr, 524 op/s 2023-10-08T09:25:51.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:50 smithi132 ceph-mon[103389]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.9 MiB/s wr, 524 op/s 2023-10-08T09:25:51.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:50 smithi049 ceph-mon[99485]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.9 MiB/s wr, 524 op/s 2023-10-08T09:25:53.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:52 smithi184 ceph-mon[103675]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 573 op/s 2023-10-08T09:25:53.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:52 smithi132 ceph-mon[103389]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 573 op/s 2023-10-08T09:25:53.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:52 smithi049 ceph-mon[99485]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.0 MiB/s wr, 573 op/s 2023-10-08T09:25:54.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 630.0 s, 516.3 tps, lat 202.899 ms stddev 248.400 2023-10-08T09:25:54.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:54 smithi184 ceph-mon[103675]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 571 op/s 2023-10-08T09:25:55.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:54 smithi132 ceph-mon[103389]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 571 op/s 2023-10-08T09:25:55.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:54 smithi049 ceph-mon[99485]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 571 op/s 2023-10-08T09:25:57.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:57 smithi049 ceph-mon[99485]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 586 op/s 2023-10-08T09:25:57.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:57 smithi184 ceph-mon[103675]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 586 op/s 2023-10-08T09:25:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:57 smithi132 ceph-mon[103389]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.9 MiB/s wr, 586 op/s 2023-10-08T09:25:58.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:25:58 smithi184 ceph-mon[103675]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.7 MiB/s wr, 561 op/s 2023-10-08T09:25:59.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:25:58 smithi132 ceph-mon[103389]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.7 MiB/s wr, 561 op/s 2023-10-08T09:25:59.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:25:58 smithi049 ceph-mon[99485]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.7 MiB/s wr, 561 op/s 2023-10-08T09:25:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 635.0 s, 555.8 tps, lat 177.331 ms stddev 185.425 2023-10-08T09:26:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.609+0000 7f07c582e700 1 -- 172.21.15.49:0/3992542559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 msgr2=0x7f07c00ff430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.609+0000 7f07c582e700 1 --2- 172.21.15.49:0/3992542559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 0x7f07c00ff430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f07b40098f0 tx=0x7f07b4009c00 comp rx=0 tx=0).stop 2023-10-08T09:26:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.612+0000 7f07c582e700 1 -- 172.21.15.49:0/3992542559 shutdown_connections 2023-10-08T09:26:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.612+0000 7f07c582e700 1 --2- 172.21.15.49:0/3992542559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 0x7f07c00ff430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.612+0000 7f07c582e700 1 --2- 172.21.15.49:0/3992542559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07c0100330 0x7f07c01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.612+0000 7f07c582e700 1 --2- 172.21.15.49:0/3992542559 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c00ff970 0x7f07c00ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.612+0000 7f07c582e700 1 -- 172.21.15.49:0/3992542559 >> 172.21.15.49:0/3992542559 conn(0x7f07c00fabb0 msgr2=0x7f07c00fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.625+0000 7f07c582e700 1 -- 172.21.15.49:0/3992542559 shutdown_connections 2023-10-08T09:26:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.626+0000 7f07c582e700 1 -- 172.21.15.49:0/3992542559 wait complete. 2023-10-08T09:26:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.626+0000 7f07c582e700 1 Processor -- start 2023-10-08T09:26:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.627+0000 7f07c582e700 1 -- start start 2023-10-08T09:26:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.627+0000 7f07c582e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 0x7f07c0103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.627+0000 7f07c582e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07c00ff970 0x7f07c0103ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.627+0000 7f07c582e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0100330 0x7f07c010ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.627+0000 7f07c582e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c010b440 con 0x7f07c0100330 2023-10-08T09:26:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.627+0000 7f07c582e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c006a210 con 0x7f07c00ff970 2023-10-08T09:26:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.627+0000 7f07c582e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c006a3a0 con 0x7f07c00ff010 2023-10-08T09:26:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.633+0000 7f07beffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 0x7f07c0103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.633+0000 7f07beffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 0x7f07c0103960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40258/0 (socket says 172.21.15.49:40258) 2023-10-08T09:26:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.633+0000 7f07beffd700 1 -- 172.21.15.49:0/1295606073 learned_addr learned my addr 172.21.15.49:0/1295606073 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.635+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0100330 0x7f07c010ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.635+0000 7f07bf7fe700 1 -- 172.21.15.49:0/1295606073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 msgr2=0x7f07c0103960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.635+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 0x7f07c0103960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.635+0000 7f07bf7fe700 1 -- 172.21.15.49:0/1295606073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07c00ff970 msgr2=0x7f07c0103ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:26:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.635+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07c00ff970 0x7f07c0103ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.635+0000 7f07bf7fe700 1 -- 172.21.15.49:0/1295606073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4009520 con 0x7f07c0100330 2023-10-08T09:26:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.636+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0100330 0x7f07c010ae70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f07b800bb00 tx=0x7f07b800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.637+0000 7f07c482c700 1 -- 172.21.15.49:0/1295606073 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b800ca30 con 0x7f07c0100330 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.637+0000 7f07c482c700 1 -- 172.21.15.49:0/1295606073 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b800cb90 con 0x7f07c0100330 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.637+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c006a680 con 0x7f07c0100330 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.637+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c006ab90 con 0x7f07c0100330 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.642+0000 7f07c482c700 1 -- 172.21.15.49:0/1295606073 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b80125a0 con 0x7f07c0100330 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.642+0000 7f07c482c700 1 -- 172.21.15.49:0/1295606073 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f07b80127c0 con 0x7f07c0100330 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.642+0000 7f07c482c700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07a80a1700 0x7f07a80a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.643+0000 7f07c482c700 1 -- 172.21.15.49:0/1295606073 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07b80cd6d0 con 0x7f07c0100330 2023-10-08T09:26:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.647+0000 7f07d1a17b80 1 Processor -- start 2023-10-08T09:26:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.649+0000 7f07beffd700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07a80a1700 0x7f07a80a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.650+0000 7f07d1a17b80 1 -- start start 2023-10-08T09:26:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.650+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e52e40 0x5636b0e53260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.650+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e537a0 0x5636b0e53cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.650+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.650+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0dbb400 con 0x5636b0e52e40 2023-10-08T09:26:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.650+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0dacbc0 con 0x5636b0e537a0 2023-10-08T09:26:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.650+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0daa4e0 con 0x5636b0e542b0 2023-10-08T09:26:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.651+0000 7f07beffd700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07a80a1700 0x7f07a80a3bc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07b400b2e0 tx=0x7f07b400bfa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40266/0 (socket says 172.21.15.49:40266) 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 -- 172.21.15.49:0/2603825159 learned_addr learned my addr 172.21.15.49:0/2603825159 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 -- 172.21.15.49:0/2603825159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e537a0 msgr2=0x5636b0e53cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/2603825159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e537a0 0x5636b0e53cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 -- 172.21.15.49:0/2603825159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e52e40 msgr2=0x5636b0e53260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/2603825159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e52e40 0x5636b0e53260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 -- 172.21.15.49:0/2603825159 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636b0bd0d80 con 0x5636b0e542b0 2023-10-08T09:26:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.653+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/2603825159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009440 tx=0x7f07b809a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4030c07380954ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.654+0000 7f07a5ffb700 1 -- 172.21.15.49:0/2603825159 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b809a6e0 con 0x5636b0e542b0 2023-10-08T09:26:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.654+0000 7f07a5ffb700 1 -- 172.21.15.49:0/2603825159 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b809e830 con 0x5636b0e542b0 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.654+0000 7f07a5ffb700 1 -- 172.21.15.49:0/2603825159 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b80b8be0 con 0x5636b0e542b0 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.655+0000 7f07d1a17b80 1 -- 172.21.15.49:0/2603825159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 msgr2=0x5636b0e54730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.655+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/2603825159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009440 tx=0x7f07b809a040 comp rx=0 tx=0).stop 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.655+0000 7f07d1a17b80 1 -- 172.21.15.49:0/2603825159 shutdown_connections 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.655+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/2603825159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54730 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.655+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/2603825159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e52e40 0x5636b0e53260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.655+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/2603825159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e537a0 0x5636b0e53cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.655+0000 7f07d1a17b80 1 -- 172.21.15.49:0/2603825159 >> 172.21.15.49:0/2603825159 conn(0x5636b0dc7200 msgr2=0x5636b0e4d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.666+0000 7f07d1a17b80 1 -- 172.21.15.49:0/2603825159 shutdown_connections 2023-10-08T09:26:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.671+0000 7f07d1a17b80 1 -- 172.21.15.49:0/2603825159 wait complete. 2023-10-08T09:26:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.671+0000 7f07d1a17b80 1 -- start start 2023-10-08T09:26:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.671+0000 7f07d1a17b80 1 Processor -- start 2023-10-08T09:26:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.672+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e52e40 0x5636b0e5c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.672+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e542b0 0x5636b0e5cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.672+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e64330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.672+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0da5dd0 con 0x5636b0e61f40 2023-10-08T09:26:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.672+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0d426b0 con 0x5636b0e542b0 2023-10-08T09:26:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.672+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0d42350 con 0x5636b0e52e40 2023-10-08T09:26:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.672+0000 7f07d1a17b80 10 client.0 _ll_register_callbacks cb 0x5636b0b9f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e542b0 0x5636b0e5cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e542b0 0x5636b0e5cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42852/0 (socket says 172.21.15.49:42852) 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 -- 172.21.15.49:0/1443316650 learned_addr learned my addr 172.21.15.49:0/1443316650 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 -- 172.21.15.49:0/1443316650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e52e40 msgr2=0x5636b0e5c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e52e40 0x5636b0e5c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 -- 172.21.15.49:0/1443316650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 msgr2=0x5636b0e64330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e64330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.673+0000 7f07be7fc700 1 -- 172.21.15.49:0/1443316650 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c006ab90 con 0x5636b0e542b0 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.674+0000 7f07be7fc700 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e542b0 0x5636b0e5cf10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f07b000baf0 tx=0x7f07b000beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=638eb4db4dce06ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.674+0000 7f079effd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:26:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.687+0000 7f079effd700 1 -- 172.21.15.49:0/1443316650 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b000dcd0 con 0x5636b0e542b0 2023-10-08T09:26:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.688+0000 7f079effd700 1 -- 172.21.15.49:0/1443316650 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07b000de30 con 0x5636b0e542b0 2023-10-08T09:26:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.688+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636b0d6de50 con 0x5636b0e542b0 2023-10-08T09:26:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.701+0000 7f079effd700 1 -- 172.21.15.49:0/1443316650 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b0004830 con 0x5636b0e542b0 2023-10-08T09:26:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.701+0000 7f07d1a17b80 10 client.26441 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.701+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5636b0bb9c00 con 0x5636b0e542b0 2023-10-08T09:26:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f079effd700 1 -- 172.21.15.49:0/1443316650 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5636b0d6de50 con 0x5636b0e542b0 2023-10-08T09:26:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f079effd700 1 -- 172.21.15.49:0/1443316650 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07b0010460 con 0x5636b0e542b0 2023-10-08T09:26:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f07d1a17b80 10 client.26441 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f07d1a17b80 10 client.26441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f07d1a17b80 20 client.26441 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0e68d80 0x5636b0e6b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f07d1a17b80 4 client.26441 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:26:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636b0d9edb0 con 0x5636b0e68d80 2023-10-08T09:26:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.707+0000 7f07beffd700 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0e68d80 0x5636b0e6b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.708+0000 7f07beffd700 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0e68d80 0x5636b0e6b1a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8635d1e3d5f86b78 server_cookie=f2c072fd267c7f45 in_seq=0 out_seq=0 2023-10-08T09:26:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.708+0000 7f079effd700 10 client.26441 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.711+0000 7f079effd700 1 -- 172.21.15.49:0/1443316650 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5636b0d9edb0 con 0x5636b0e68d80 2023-10-08T09:26:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.711+0000 7f079effd700 10 client.26441 handle_command_reply: tid=0 2023-10-08T09:26:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.711+0000 7f07d1a17b80 1 client.26441 shutdown 2023-10-08T09:26:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.722+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e542b0 msgr2=0x5636b0e5cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.722+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e542b0 0x5636b0e5cf10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f07b000baf0 tx=0x7f07b000beb0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.737+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 shutdown_connections 2023-10-08T09:26:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.737+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0e68d80 0x5636b0e6b1a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.737+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e52e40 0x5636b0e5c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.737+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e64330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.737+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1443316650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e542b0 0x5636b0e5cf10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.737+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 >> 172.21.15.49:0/1443316650 conn(0x5636b0dc7200 msgr2=0x5636b0e4d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.737+0000 7f079effd700 0 client.26441 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.741+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 shutdown_connections 2023-10-08T09:26:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.745+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1443316650 wait complete. 2023-10-08T09:26:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.753+0000 7f07d1a17b80 20 client.26441 trim_cache size 0 max 16384 2023-10-08T09:26:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 Processor -- start 2023-10-08T09:26:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 -- start start 2023-10-08T09:26:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 0x5636b0e64330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0c268c0 con 0x5636b0e61f40 2023-10-08T09:26:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0e54120 con 0x5636b0e52e40 2023-10-08T09:26:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.780+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0e4f300 con 0x5636b0e542b0 2023-10-08T09:26:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.786+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.786+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53244/0 (socket says 172.21.15.49:53244) 2023-10-08T09:26:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.786+0000 7f07bf7fe700 1 -- 172.21.15.49:0/571128170 learned_addr learned my addr 172.21.15.49:0/571128170 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.793+0000 7f07beffd700 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 0x5636b0e64330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.793+0000 7f07be7fc700 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.799+0000 7f07bf7fe700 1 -- 172.21.15.49:0/571128170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 msgr2=0x5636b0e54690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.799+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.799+0000 7f07bf7fe700 1 -- 172.21.15.49:0/571128170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 msgr2=0x5636b0e64330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.799+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 0x5636b0e64330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.800 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.799+0000 7f07bf7fe700 1 -- 172.21.15.49:0/571128170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636b0bf71f0 con 0x5636b0e61f40 2023-10-08T09:26:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.804+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e53b80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f07b809a3e0 tx=0x7f07b80ba1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7628b576fca21ad8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.805+0000 7f07a67fc700 1 -- 172.21.15.49:0/571128170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b80a2a70 con 0x5636b0e61f40 2023-10-08T09:26:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.805+0000 7f07a67fc700 1 -- 172.21.15.49:0/571128170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b8098a80 con 0x5636b0e61f40 2023-10-08T09:26:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.805+0000 7f07a67fc700 1 -- 172.21.15.49:0/571128170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b80b8e40 con 0x5636b0e61f40 2023-10-08T09:26:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.814+0000 7f07d1a17b80 1 -- 172.21.15.49:0/571128170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 msgr2=0x5636b0e53b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.814+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e53b80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f07b809a3e0 tx=0x7f07b80ba1d0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.815+0000 7f07d1a17b80 1 -- 172.21.15.49:0/571128170 shutdown_connections 2023-10-08T09:26:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.815+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e53b80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.815+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 0x5636b0e64330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.815+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/571128170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e54690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.815+0000 7f07d1a17b80 1 -- 172.21.15.49:0/571128170 >> 172.21.15.49:0/571128170 conn(0x5636b0dc7200 msgr2=0x5636b0e6fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.817+0000 7f07d1a17b80 1 -- 172.21.15.49:0/571128170 shutdown_connections 2023-10-08T09:26:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.818+0000 7f07d1a17b80 1 -- 172.21.15.49:0/571128170 wait complete. 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.818+0000 7f07d1a17b80 1 -- start start 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 1 Processor -- start 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 0x5636b0e75f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0e77a10 con 0x5636b0e61f40 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0e77b80 con 0x5636b0e52e40 2023-10-08T09:26:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636b0e77d40 con 0x5636b0e542b0 2023-10-08T09:26:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.819+0000 7f07d1a17b80 10 client.0 _ll_register_callbacks cb 0x7f07b000b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.820+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.820+0000 7f07bf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e774d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53252/0 (socket says 172.21.15.49:53252) 2023-10-08T09:26:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.820+0000 7f07bf7fe700 1 -- 172.21.15.49:0/1279580638 learned_addr learned my addr 172.21.15.49:0/1279580638 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:01.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.820+0000 7f07bf7fe700 1 -- 172.21.15.49:0/1279580638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 msgr2=0x5636b0e764c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f07bf7fe700 1 -- 172.21.15.49:0/1279580638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 msgr2=0x5636b0e75f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 0x5636b0e75f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f07bf7fe700 1 -- 172.21.15.49:0/1279580638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636b0bd0d80 con 0x5636b0e61f40 2023-10-08T09:26:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f07bf7fe700 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e774d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f07b8097100 tx=0x7f07b8004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0f797865a43508 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f079effd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:26:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f079effd700 1 -- 172.21.15.49:0/1279580638 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8099dd0 con 0x5636b0e61f40 2023-10-08T09:26:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.821+0000 7f079effd700 1 -- 172.21.15.49:0/1279580638 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07b809a940 con 0x5636b0e61f40 2023-10-08T09:26:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.827+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07b809e830 con 0x5636b0e61f40 2023-10-08T09:26:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.832+0000 7f079effd700 1 -- 172.21.15.49:0/1279580638 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07b809ab00 con 0x5636b0e61f40 2023-10-08T09:26:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.836+0000 7f07d1a17b80 10 client.16866 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.836+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5636b0bcf250 con 0x5636b0e61f40 2023-10-08T09:26:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f079effd700 1 -- 172.21.15.49:0/1279580638 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07b809e830 con 0x5636b0e61f40 2023-10-08T09:26:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f079effd700 1 -- 172.21.15.49:0/1279580638 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07b8099710 con 0x5636b0e61f40 2023-10-08T09:26:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f07d1a17b80 10 client.16866 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f07d1a17b80 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f07d1a17b80 20 client.16866 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0ebd560 0x5636b0ebf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f07d1a17b80 4 client.16866 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:26:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.841+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5636b0d9ead0 con 0x5636b0ebd560 2023-10-08T09:26:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.843+0000 7f07beffd700 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0ebd560 0x5636b0ebf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.844+0000 7f07beffd700 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0ebd560 0x5636b0ebf950 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3014691458a616 server_cookie=709d585111aeb62c in_seq=0 out_seq=0 2023-10-08T09:26:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.846+0000 7f079effd700 10 client.16866 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:01.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:01 smithi049 ceph-mon[99485]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.1 MiB/s wr, 436 op/s 2023-10-08T09:26:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.896+0000 7f079effd700 1 -- 172.21.15.49:0/1279580638 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5636b0d9ead0 con 0x5636b0ebd560 2023-10-08T09:26:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.896+0000 7f079effd700 10 client.16866 handle_command_reply: tid=0 2023-10-08T09:26:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.896+0000 7f07d1a17b80 1 client.16866 shutdown 2023-10-08T09:26:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.901+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 msgr2=0x5636b0e774d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.901+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e774d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f07b8097100 tx=0x7f07b8004b60 comp rx=0 tx=0).stop 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 shutdown_connections 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5636b0ebd560 0x5636b0ebf950 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5636b0e61f40 0x5636b0e774d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5636b0e52e40 0x5636b0e75f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 --2- 172.21.15.49:0/1279580638 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5636b0e542b0 0x5636b0e764c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 >> 172.21.15.49:0/1279580638 conn(0x5636b0dc7200 msgr2=0x5636b0e5a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f079effd700 0 client.16866 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 shutdown_connections 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 1 -- 172.21.15.49:0/1279580638 wait complete. 2023-10-08T09:26:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.903+0000 7f07d1a17b80 20 client.16866 trim_cache size 0 max 16384 2023-10-08T09:26:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:26:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:26:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:26:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07a80a1700 msgr2=0x7f07a80a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07a80a1700 0x7f07a80a3bc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f07b400b2e0 tx=0x7f07b400bfa0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0100330 msgr2=0x7f07c010ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0100330 0x7f07c010ae70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f07b800bb00 tx=0x7f07b800be10 comp rx=0 tx=0).stop 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 shutdown_connections 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07a80a1700 0x7f07a80a3bc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07c00ff010 0x7f07c0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07c0100330 0x7f07c010ae70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 --2- 172.21.15.49:0/1295606073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07c00ff970 0x7f07c0103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.904+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 >> 172.21.15.49:0/1295606073 conn(0x7f07c00fabb0 msgr2=0x7f07c00fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.907+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 shutdown_connections 2023-10-08T09:26:01.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:01.907+0000 7f07c582e700 1 -- 172.21.15.49:0/1295606073 wait complete. 2023-10-08T09:26:01.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:01 smithi184 ceph-mon[103675]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.1 MiB/s wr, 436 op/s 2023-10-08T09:26:02.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:01 smithi132 ceph-mon[103389]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.1 MiB/s wr, 436 op/s 2023-10-08T09:26:03.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:02 smithi184 ceph-mon[103675]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.8 MiB/s wr, 496 op/s 2023-10-08T09:26:03.302 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:02 smithi132 ceph-mon[103389]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.8 MiB/s wr, 496 op/s 2023-10-08T09:26:03.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:02 smithi049 ceph-mon[99485]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.8 MiB/s wr, 496 op/s 2023-10-08T09:26:04.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 640.0 s, 541.0 tps, lat 185.025 ms stddev 201.333 2023-10-08T09:26:05.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:05 smithi049 ceph-mon[99485]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.3 MiB/s wr, 483 op/s 2023-10-08T09:26:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:05 smithi184 ceph-mon[103675]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.3 MiB/s wr, 483 op/s 2023-10-08T09:26:06.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:05 smithi132 ceph-mon[103389]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 7.3 MiB/s wr, 483 op/s 2023-10-08T09:26:07.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:07 smithi049 ceph-mon[99485]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.5 MiB/s wr, 502 op/s 2023-10-08T09:26:07.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:07 smithi184 ceph-mon[103675]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.5 MiB/s wr, 502 op/s 2023-10-08T09:26:08.052 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:07 smithi132 ceph-mon[103389]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.5 MiB/s wr, 502 op/s 2023-10-08T09:26:09.316 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 645.0 s, 527.2 tps, lat 191.339 ms stddev 203.310 2023-10-08T09:26:09.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:09 smithi049 ceph-mon[99485]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.2 MiB/s wr, 511 op/s 2023-10-08T09:26:09.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:09 smithi184 ceph-mon[103675]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.2 MiB/s wr, 511 op/s 2023-10-08T09:26:10.054 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:09 smithi132 ceph-mon[103389]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.2 MiB/s wr, 511 op/s 2023-10-08T09:26:11.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:11 smithi049 ceph-mon[99485]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.6 MiB/s wr, 461 op/s 2023-10-08T09:26:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:11 smithi184 ceph-mon[103675]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.6 MiB/s wr, 461 op/s 2023-10-08T09:26:12.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:11 smithi132 ceph-mon[103389]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.6 MiB/s wr, 461 op/s 2023-10-08T09:26:13.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:13 smithi132 ceph-mon[103389]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 519 op/s 2023-10-08T09:26:13.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:13 smithi049 ceph-mon[99485]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 519 op/s 2023-10-08T09:26:13.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:13 smithi184 ceph-mon[103675]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.6 MiB/s wr, 519 op/s 2023-10-08T09:26:14.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 650.0 s, 591.1 tps, lat 167.049 ms stddev 188.417 2023-10-08T09:26:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.152+0000 7fc1f8b68700 1 -- 172.21.15.49:0/4173748383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410a160 msgr2=0x7fc1f410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.152+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/4173748383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410a160 0x7fc1f410c5f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc1e801a9a0 tx=0x7fc1e801acb0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.163+0000 7fc1f8b68700 1 -- 172.21.15.49:0/4173748383 shutdown_connections 2023-10-08T09:26:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.163+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/4173748383 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f410cb30 0x7fc1f410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.163+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/4173748383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1f41077f0 0x7fc1f4109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.163+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/4173748383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410a160 0x7fc1f410c5f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.163+0000 7fc1f8b68700 1 -- 172.21.15.49:0/4173748383 >> 172.21.15.49:0/4173748383 conn(0x7fc1f406d770 msgr2=0x7fc1f406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:15.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.181+0000 7fc1f8b68700 1 -- 172.21.15.49:0/4173748383 shutdown_connections 2023-10-08T09:26:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.188+0000 7fc1f8b68700 1 -- 172.21.15.49:0/4173748383 wait complete. 2023-10-08T09:26:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 Processor -- start 2023-10-08T09:26:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 -- start start 2023-10-08T09:26:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f41077f0 0x7fc1f419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1f410a160 0x7fc1f419d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410cb30 0x7fc1f41c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f41c6900 con 0x7fc1f410cb30 2023-10-08T09:26:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f41c6a70 con 0x7fc1f410a160 2023-10-08T09:26:15.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.189+0000 7fc1f8b68700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f41c6c30 con 0x7fc1f41077f0 2023-10-08T09:26:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.197+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410cb30 0x7fc1f41c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.198 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.197+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410cb30 0x7fc1f41c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53260/0 (socket says 172.21.15.49:53260) 2023-10-08T09:26:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.197+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/925346363 learned_addr learned my addr 172.21.15.49:0/925346363 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.201+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1f410a160 0x7fc1f419d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.202+0000 7fc1f259c700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f41077f0 0x7fc1f419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.205+0000 7fc1f1d9b700 1 -- 172.21.15.49:0/925346363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f41077f0 msgr2=0x7fc1f419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.205+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f41077f0 0x7fc1f419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.205+0000 7fc1f1d9b700 1 -- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410cb30 msgr2=0x7fc1f41c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.205+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410cb30 0x7fc1f41c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.205+0000 7fc1f1d9b700 1 -- 172.21.15.49:0/925346363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e801a600 con 0x7fc1f410a160 2023-10-08T09:26:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.210+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1f410a160 0x7fc1f419d2e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8003780 tx=0x7fc1e8005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.220+0000 7fc1e37fe700 1 -- 172.21.15.49:0/925346363 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8004070 con 0x7fc1f410a160 2023-10-08T09:26:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.220+0000 7fc1e37fe700 1 -- 172.21.15.49:0/925346363 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e80041d0 con 0x7fc1f410a160 2023-10-08T09:26:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.220+0000 7fc1e37fe700 1 -- 172.21.15.49:0/925346363 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e801f6c0 con 0x7fc1f410a160 2023-10-08T09:26:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.220+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f41c6eb0 con 0x7fc1f410a160 2023-10-08T09:26:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.221+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f41c7220 con 0x7fc1f410a160 2023-10-08T09:26:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.225+0000 7fc204d51b80 1 Processor -- start 2023-10-08T09:26:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.226+0000 7fc204d51b80 1 -- start start 2023-10-08T09:26:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.227+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52f2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.227+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.227+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f32b0 0x564db52f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.227+0000 7fc204d51b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db525a400 con 0x564db52f32b0 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.227+0000 7fc204d51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db524bbc0 con 0x564db52f1e40 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.227+0000 7fc204d51b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db52494e0 con 0x564db52f27a0 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.232+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.232+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52f2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42882/0 (socket says 172.21.15.49:42882) 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.232+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/2573493209 learned_addr learned my addr 172.21.15.49:0/2573493209 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.233+0000 7fc1f259c700 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.239+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f32b0 0x564db52f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.239+0000 7fc1e37fe700 1 -- 172.21.15.49:0/925346363 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc1e800f480 con 0x7fc1f410a160 2023-10-08T09:26:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.240+0000 7fc1e37fe700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1dc0a1680 0x7fc1dc0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.240+0000 7fc1e37fe700 1 -- 172.21.15.49:0/925346363 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc1e80d0730 con 0x7fc1f410a160 2023-10-08T09:26:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.241+0000 7fc1f1d9b700 1 -- 172.21.15.49:0/2573493209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 msgr2=0x564db52f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.241+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.241+0000 7fc1f1d9b700 1 -- 172.21.15.49:0/2573493209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 msgr2=0x564db52f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.241+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.241+0000 7fc1f1d9b700 1 -- 172.21.15.49:0/2573493209 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db506fd80 con 0x564db52f32b0 2023-10-08T09:26:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.242+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f32b0 0x564db52f3730 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005b40 tx=0x7fc1e801a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7728f878e8ff8073 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.245+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1dc0a1680 0x7fc1dc0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.247+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1dc0a1680 0x7fc1dc0a3b40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009500 tx=0x7fc1ec004b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.254+0000 7fc1e0ff9700 1 -- 172.21.15.49:0/2573493209 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e809d140 con 0x564db52f32b0 2023-10-08T09:26:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.254+0000 7fc1e0ff9700 1 -- 172.21.15.49:0/2573493209 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e809dee0 con 0x564db52f32b0 2023-10-08T09:26:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.254+0000 7fc1e0ff9700 1 -- 172.21.15.49:0/2573493209 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e809bb80 con 0x564db52f32b0 2023-10-08T09:26:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.256+0000 7fc204d51b80 1 -- 172.21.15.49:0/2573493209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f32b0 msgr2=0x564db52f3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.256+0000 7fc204d51b80 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f32b0 0x564db52f3730 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005b40 tx=0x7fc1e801a320 comp rx=0 tx=0).stop 2023-10-08T09:26:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.256+0000 7fc204d51b80 1 -- 172.21.15.49:0/2573493209 shutdown_connections 2023-10-08T09:26:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.256+0000 7fc204d51b80 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f32b0 0x564db52f3730 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.256+0000 7fc204d51b80 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.256+0000 7fc204d51b80 1 --2- 172.21.15.49:0/2573493209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.257+0000 7fc204d51b80 1 -- 172.21.15.49:0/2573493209 >> 172.21.15.49:0/2573493209 conn(0x564db5266200 msgr2=0x564db52ec8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.261+0000 7fc204d51b80 1 -- 172.21.15.49:0/2573493209 shutdown_connections 2023-10-08T09:26:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.273+0000 7fc204d51b80 1 -- 172.21.15.49:0/2573493209 wait complete. 2023-10-08T09:26:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.273+0000 7fc204d51b80 1 -- start start 2023-10-08T09:26:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.273+0000 7fc204d51b80 1 Processor -- start 2023-10-08T09:26:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 0x564db52fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5303080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc204d51b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5244dd0 con 0x564db52f27a0 2023-10-08T09:26:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc204d51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db51e16b0 con 0x564db52f1e40 2023-10-08T09:26:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc204d51b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db51e1350 con 0x564db5300c90 2023-10-08T09:26:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc204d51b80 10 client.0 _ll_register_callbacks cb 0x564db503e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5303080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5303080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40308/0 (socket says 172.21.15.49:40308) 2023-10-08T09:26:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.274+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/1759955313 learned_addr learned my addr 172.21.15.49:0/1759955313 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.275+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/1759955313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 msgr2=0x564db52fb720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:26:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.275+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fb720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.275+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/1759955313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 msgr2=0x564db52fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.275+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 0x564db52fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.275+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/1759955313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f41c7220 con 0x564db5300c90 2023-10-08T09:26:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.277+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5303080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0389c0 tx=0x7fc1ec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511aeacfb7f16845 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.277+0000 7fc1d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:26:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.277+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/1759955313 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec037c80 con 0x564db5300c90 2023-10-08T09:26:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.277+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/1759955313 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc1ec040040 con 0x564db5300c90 2023-10-08T09:26:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.277+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564db520ce50 con 0x564db5300c90 2023-10-08T09:26:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/1759955313 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1ec038e30 con 0x564db5300c90 2023-10-08T09:26:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 10 client.26521 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:15.279 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564db5058c00 con 0x564db5300c90 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/1759955313 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564db520ce50 con 0x564db5300c90 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/1759955313 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1ec04c070 con 0x564db5300c90 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 10 client.26521 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 10 client.26521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 20 client.26521 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db5307a30 0x564db5309e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 4 client.26521 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:26:15.280 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.278+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564db523ddb0 con 0x564db5307a30 2023-10-08T09:26:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.287+0000 7fc1f259c700 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db5307a30 0x564db5309e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.300+0000 7fc1f259c700 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db5307a30 0x564db5309e50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4874d99e3d241297 server_cookie=3580140c0148a9a5 in_seq=0 out_seq=0 2023-10-08T09:26:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.311+0000 7fc1d9ffb700 10 client.26521 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.322+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/1759955313 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564db523ddb0 con 0x564db5307a30 2023-10-08T09:26:15.323 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.322+0000 7fc1d9ffb700 10 client.26521 handle_command_reply: tid=0 2023-10-08T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.322+0000 7fc204d51b80 1 client.26521 shutdown 2023-10-08T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 msgr2=0x564db5303080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5303080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0389c0 tx=0x7fc1ec00b040 comp rx=0 tx=0).stop 2023-10-08T09:26:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 shutdown_connections 2023-10-08T09:26:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db5307a30 0x564db5309e50 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5303080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1759955313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 0x564db52fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.323+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 >> 172.21.15.49:0/1759955313 conn(0x564db5266200 msgr2=0x564db52ec010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.334+0000 7fc1d9ffb700 0 client.26521 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.338+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 shutdown_connections 2023-10-08T09:26:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.338+0000 7fc204d51b80 1 -- 172.21.15.49:0/1759955313 wait complete. 2023-10-08T09:26:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.338+0000 7fc204d51b80 20 client.26521 trim_cache size 0 max 16384 2023-10-08T09:26:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 Processor -- start 2023-10-08T09:26:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 -- start start 2023-10-08T09:26:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db5300c90 0x564db5317630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db50c58c0 con 0x564db5300c90 2023-10-08T09:26:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db523ddb0 con 0x564db52f1e40 2023-10-08T09:26:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.367+0000 7fc204d51b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db52e9830 con 0x564db52f27a0 2023-10-08T09:26:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.368+0000 7fc1f259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.368+0000 7fc1f259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42894/0 (socket says 172.21.15.49:42894) 2023-10-08T09:26:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.368+0000 7fc1f259c700 1 -- 172.21.15.49:0/1610078112 learned_addr learned my addr 172.21.15.49:0/1610078112 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.373+0000 7fc1f1d9b700 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.379+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db5300c90 0x564db5317630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.380+0000 7fc1f259c700 1 -- 172.21.15.49:0/1610078112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 msgr2=0x564db52f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.380+0000 7fc1f259c700 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.380+0000 7fc1f259c700 1 -- 172.21.15.49:0/1610078112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db5300c90 msgr2=0x564db5317630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.380+0000 7fc1f259c700 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db5300c90 0x564db5317630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.380+0000 7fc1f259c700 1 -- 172.21.15.49:0/1610078112 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564db50961f0 con 0x564db52f1e40 2023-10-08T09:26:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.388+0000 7fc1f259c700 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fa280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4017da0 tx=0x7fc1e4017dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8639b5e954c72593 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.388+0000 7fc1e17fa700 1 -- 172.21.15.49:0/1610078112 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4017160 con 0x564db52f1e40 2023-10-08T09:26:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.388+0000 7fc1e17fa700 1 -- 172.21.15.49:0/1610078112 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e400c040 con 0x564db52f1e40 2023-10-08T09:26:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.388+0000 7fc1e17fa700 1 -- 172.21.15.49:0/1610078112 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4014df0 con 0x564db52f1e40 2023-10-08T09:26:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.398+0000 7fc204d51b80 1 -- 172.21.15.49:0/1610078112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 msgr2=0x564db52fa280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.398+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fa280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4017da0 tx=0x7fc1e4017dd0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.414+0000 7fc204d51b80 1 -- 172.21.15.49:0/1610078112 shutdown_connections 2023-10-08T09:26:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.414+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db5300c90 0x564db5317630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.414+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db52fa280 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.414+0000 7fc204d51b80 1 --2- 172.21.15.49:0/1610078112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db52f27a0 0x564db52f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.414+0000 7fc204d51b80 1 -- 172.21.15.49:0/1610078112 >> 172.21.15.49:0/1610078112 conn(0x564db5266200 msgr2=0x564db530e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.424+0000 7fc204d51b80 1 -- 172.21.15.49:0/1610078112 shutdown_connections 2023-10-08T09:26:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.425+0000 7fc204d51b80 1 -- 172.21.15.49:0/1610078112 wait complete. 2023-10-08T09:26:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 -- start start 2023-10-08T09:26:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 Processor -- start 2023-10-08T09:26:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db5316710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 0x564db5316c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5359f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db5317320 con 0x564db52f27a0 2023-10-08T09:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db535a4c0 con 0x564db52f1e40 2023-10-08T09:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564db535a650 con 0x564db5300c90 2023-10-08T09:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.426+0000 7fc204d51b80 10 client.0 _ll_register_callbacks cb 0x7fc1ec038790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5359f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5359f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40320/0 (socket says 172.21.15.49:40320) 2023-10-08T09:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/521518473 learned_addr learned my addr 172.21.15.49:0/521518473 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/521518473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 msgr2=0x564db5316710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db5316710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/521518473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 msgr2=0x564db5316c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 0x564db5316c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.427+0000 7fc1f2d9d700 1 -- 172.21.15.49:0/521518473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e4017a80 con 0x564db5300c90 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc1f2d9d700 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5359f80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec043870 tx=0x7fc1ec007eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a915e72918e503 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc1d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/521518473 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec043d00 con 0x564db5300c90 2023-10-08T09:26:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/521518473 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc1ec043e60 con 0x564db5300c90 2023-10-08T09:26:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1e809dee0 con 0x564db5300c90 2023-10-08T09:26:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/521518473 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1ec044410 con 0x564db5300c90 2023-10-08T09:26:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc204d51b80 10 client.26527 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.428+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x564db506e250 con 0x564db5300c90 2023-10-08T09:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.432+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/521518473 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e809dee0 con 0x564db5300c90 2023-10-08T09:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.432+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/521518473 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc1ec04c070 con 0x564db5300c90 2023-10-08T09:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.433+0000 7fc204d51b80 10 client.26527 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.433+0000 7fc204d51b80 10 client.26527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.433+0000 7fc204d51b80 20 client.26527 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.433+0000 7fc204d51b80 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db535c020 0x564db535e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.433+0000 7fc204d51b80 4 client.26527 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.433+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564db523dad0 con 0x564db535c020 2023-10-08T09:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.434+0000 7fc1f259c700 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db535c020 0x564db535e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.435+0000 7fc1f259c700 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db535c020 0x564db535e460 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea17fa89f04e086 server_cookie=63f5749a7fa072f0 in_seq=0 out_seq=0 2023-10-08T09:26:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.435+0000 7fc1d9ffb700 10 client.26527 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc1d9ffb700 1 -- 172.21.15.49:0/521518473 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564db523dad0 con 0x564db535c020 2023-10-08T09:26:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc1d9ffb700 10 client.26527 handle_command_reply: tid=0 2023-10-08T09:26:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 client.26527 shutdown 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 msgr2=0x564db5359f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5359f80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec043870 tx=0x7fc1ec007eb0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 shutdown_connections 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x564db535c020 0x564db535e460 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564db5300c90 0x564db5359f80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x564db52f1e40 0x564db5316710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 --2- 172.21.15.49:0/521518473 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564db52f27a0 0x564db5316c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc1d9ffb700 0 client.26527 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 >> 172.21.15.49:0/521518473 conn(0x564db5266200 msgr2=0x564db52fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 shutdown_connections 2023-10-08T09:26:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.456+0000 7fc204d51b80 1 -- 172.21.15.49:0/521518473 wait complete. 2023-10-08T09:26:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc204d51b80 20 client.26527 trim_cache size 0 max 16384 2023-10-08T09:26:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:26:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:26:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:26:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:26:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1dc0a1680 msgr2=0x7fc1dc0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1dc0a1680 0x7fc1dc0a3b40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009500 tx=0x7fc1ec004b60 comp rx=0 tx=0).stop 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1f410a160 msgr2=0x7fc1f419d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1f410a160 0x7fc1f419d2e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8003780 tx=0x7fc1e8005ab0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 shutdown_connections 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fc1dc0a1680 0x7fc1dc0a3b40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1f41077f0 0x7fc1f419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc1f410cb30 0x7fc1f41c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 --2- 172.21.15.49:0/925346363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1f410a160 0x7fc1f419d2e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 >> 172.21.15.49:0/925346363 conn(0x7fc1f406d770 msgr2=0x7fc1f410a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.457+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 shutdown_connections 2023-10-08T09:26:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:15.458+0000 7fc1f8b68700 1 -- 172.21.15.49:0/925346363 wait complete. 2023-10-08T09:26:15.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:15 smithi049 ceph-mon[99485]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.5 MiB/s wr, 502 op/s 2023-10-08T09:26:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:15 smithi184 ceph-mon[103675]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.5 MiB/s wr, 502 op/s 2023-10-08T09:26:16.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:15 smithi132 ceph-mon[103389]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.5 MiB/s wr, 502 op/s 2023-10-08T09:26:18.001 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:17 smithi184 ceph-mon[103675]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 548 op/s 2023-10-08T09:26:18.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:17 smithi132 ceph-mon[103389]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 548 op/s 2023-10-08T09:26:18.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:17 smithi049 ceph-mon[99485]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 548 op/s 2023-10-08T09:26:19.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 655.0 s, 514.7 tps, lat 196.715 ms stddev 219.865 2023-10-08T09:26:19.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:26:19.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.739+0000 7f465162c700 1 -- 172.21.15.49:0/4143971692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c101230 msgr2=0x7f464c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.741 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.739+0000 7f465162c700 1 --2- 172.21.15.49:0/4143971692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c101230 0x7f464c107ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f463c009990 tx=0x7f463c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.751+0000 7f465162c700 1 -- 172.21.15.49:0/4143971692 shutdown_connections 2023-10-08T09:26:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.751+0000 7f465162c700 1 --2- 172.21.15.49:0/4143971692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c101230 0x7f464c107ae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.751+0000 7f465162c700 1 --2- 172.21.15.49:0/4143971692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c0ff6a0 0x7f464c0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.751+0000 7f465162c700 1 --2- 172.21.15.49:0/4143971692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f464c100890 0x7f464c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.754 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.751+0000 7f465162c700 1 -- 172.21.15.49:0/4143971692 >> 172.21.15.49:0/4143971692 conn(0x7f464c0fac10 msgr2=0x7f464c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:19.763 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.754+0000 7f465162c700 1 -- 172.21.15.49:0/4143971692 shutdown_connections 2023-10-08T09:26:19.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.762+0000 7f465162c700 1 -- 172.21.15.49:0/4143971692 wait complete. 2023-10-08T09:26:19.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.763+0000 7f465162c700 1 Processor -- start 2023-10-08T09:26:19.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.766+0000 7f465162c700 1 -- start start 2023-10-08T09:26:19.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.766+0000 7f465162c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f464c0ff6a0 0x7f464c06a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.766+0000 7f465162c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c100890 0x7f464c06a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.766+0000 7f465162c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101230 0x7f464c071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.766+0000 7f465162c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c071f60 con 0x7f464c101230 2023-10-08T09:26:19.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.766+0000 7f465162c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0720a0 con 0x7f464c0ff6a0 2023-10-08T09:26:19.768 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.766+0000 7f465162c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c072230 con 0x7f464c100890 2023-10-08T09:26:19.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.769+0000 7f464a7fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c100890 0x7f464c06a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.774+0000 7f464b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101230 0x7f464c071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.774+0000 7f464b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101230 0x7f464c071990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53300/0 (socket says 172.21.15.49:53300) 2023-10-08T09:26:19.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.774+0000 7f464b7fe700 1 -- 172.21.15.49:0/3078403062 learned_addr learned my addr 172.21.15.49:0/3078403062 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:19.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.780+0000 7f464affd700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f464c0ff6a0 0x7f464c06a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.781+0000 7f464b7fe700 1 -- 172.21.15.49:0/3078403062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c100890 msgr2=0x7f464c06a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.781+0000 7f464b7fe700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c100890 0x7f464c06a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.783 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.781+0000 7f464b7fe700 1 -- 172.21.15.49:0/3078403062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f464c0ff6a0 msgr2=0x7f464c06a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.781+0000 7f464b7fe700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f464c0ff6a0 0x7f464c06a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.781+0000 7f464b7fe700 1 -- 172.21.15.49:0/3078403062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c0095f0 con 0x7f464c101230 2023-10-08T09:26:19.784 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.781+0000 7f464b7fe700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101230 0x7f464c071990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f463c006010 tx=0x7f463c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:19.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.782+0000 7f4643fff700 1 -- 172.21.15.49:0/3078403062 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c01d070 con 0x7f464c101230 2023-10-08T09:26:19.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.782+0000 7f4643fff700 1 -- 172.21.15.49:0/3078403062 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f463c00fbf0 con 0x7f464c101230 2023-10-08T09:26:19.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.782+0000 7f4643fff700 1 -- 172.21.15.49:0/3078403062 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c017650 con 0x7f464c101230 2023-10-08T09:26:19.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.783+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c0724b0 con 0x7f464c101230 2023-10-08T09:26:19.785 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.783+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c10f5f0 con 0x7f464c101230 2023-10-08T09:26:19.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.787+0000 7f4643fff700 1 -- 172.21.15.49:0/3078403062 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f463c0177b0 con 0x7f464c101230 2023-10-08T09:26:19.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.788+0000 7f4643fff700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46380a1680 0x7f46380a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.788+0000 7f4643fff700 1 -- 172.21.15.49:0/3078403062 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f463c0cf1b0 con 0x7f464c101230 2023-10-08T09:26:19.790 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.789+0000 7f465d815b80 1 Processor -- start 2023-10-08T09:26:19.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.789+0000 7f465d815b80 1 -- start start 2023-10-08T09:26:19.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.790+0000 7f465d815b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d2060 0x561a5b5d2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.790+0000 7f465d815b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.791 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.790+0000 7f465d815b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 0x561a5b5d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.790+0000 7f465d815b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b53a460 con 0x561a5b5d2060 2023-10-08T09:26:19.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.790+0000 7f465d815b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b52bc20 con 0x561a5b5d29c0 2023-10-08T09:26:19.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.790+0000 7f465d815b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b529540 con 0x561a5b5d3800 2023-10-08T09:26:19.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.790+0000 7f464affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d2060 0x561a5b5d2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d2060 0x561a5b5d2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53312/0 (socket says 172.21.15.49:53312) 2023-10-08T09:26:19.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 -- 172.21.15.49:0/2380005824 learned_addr learned my addr 172.21.15.49:0/2380005824 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:19.792 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46380a1680 0x7f46380a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 -- 172.21.15.49:0/2380005824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 msgr2=0x561a5b5d3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:19.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 --2- 172.21.15.49:0/2380005824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 0x561a5b5d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 -- 172.21.15.49:0/2380005824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 msgr2=0x561a5b5d3270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:26:19.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 --2- 172.21.15.49:0/2380005824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d3270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 -- 172.21.15.49:0/2380005824 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a5b34fde0 con 0x561a5b5d2060 2023-10-08T09:26:19.793 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.791+0000 7f464affd700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46380a1680 0x7f46380a3b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4644009aa0 tx=0x7f4644009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:19.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.792+0000 7f464affd700 1 --2- 172.21.15.49:0/2380005824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d2060 0x561a5b5d2480 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f46440079b0 tx=0x7f464401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b341e6008b1cfd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:19.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.792+0000 7f46417fa700 1 -- 172.21.15.49:0/2380005824 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400e030 con 0x561a5b5d2060 2023-10-08T09:26:19.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.792+0000 7f46417fa700 1 -- 172.21.15.49:0/2380005824 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464401ae20 con 0x561a5b5d2060 2023-10-08T09:26:19.794 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.792+0000 7f46417fa700 1 -- 172.21.15.49:0/2380005824 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46440227a0 con 0x561a5b5d2060 2023-10-08T09:26:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.793+0000 7f465d815b80 1 -- 172.21.15.49:0/2380005824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d2060 msgr2=0x561a5b5d2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.793+0000 7f465d815b80 1 --2- 172.21.15.49:0/2380005824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d2060 0x561a5b5d2480 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f46440079b0 tx=0x7f464401b000 comp rx=0 tx=0).stop 2023-10-08T09:26:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.794+0000 7f465d815b80 1 -- 172.21.15.49:0/2380005824 shutdown_connections 2023-10-08T09:26:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.794+0000 7f465d815b80 1 --2- 172.21.15.49:0/2380005824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 0x561a5b5d3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.795 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.794+0000 7f465d815b80 1 --2- 172.21.15.49:0/2380005824 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d2060 0x561a5b5d2480 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.794+0000 7f465d815b80 1 --2- 172.21.15.49:0/2380005824 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.796 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.794+0000 7f465d815b80 1 -- 172.21.15.49:0/2380005824 >> 172.21.15.49:0/2380005824 conn(0x561a5b546260 msgr2=0x561a5b5cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:19.800 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.796+0000 7f465d815b80 1 -- 172.21.15.49:0/2380005824 shutdown_connections 2023-10-08T09:26:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.796+0000 7f465d815b80 1 -- 172.21.15.49:0/2380005824 wait complete. 2023-10-08T09:26:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 -- start start 2023-10-08T09:26:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 Processor -- start 2023-10-08T09:26:19.802 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5dbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5dc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d3800 0x561a5b5e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b524e30 con 0x561a5b5d29c0 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b4c1710 con 0x561a5b5d3800 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b4c13b0 con 0x561a5b5d2060 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f465d815b80 10 client.0 _ll_register_callbacks cb 0x561a5b31e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.797+0000 7f464affd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5dbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.803 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5dc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53318/0 (socket says 172.21.15.49:53318) 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 -- 172.21.15.49:0/290974830 learned_addr learned my addr 172.21.15.49:0/290974830 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 -- 172.21.15.49:0/290974830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 msgr2=0x561a5b5dbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5dbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 -- 172.21.15.49:0/290974830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d3800 msgr2=0x561a5b5e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d3800 0x561a5b5e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 -- 172.21.15.49:0/290974830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644007e20 con 0x561a5b5d29c0 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f464a7fc700 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5dc340 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4634011690 tx=0x7f4634011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46944bf8c9bf6e53 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:19.804 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.798+0000 7f46327fc700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.799+0000 7f46327fc700 1 -- 172.21.15.49:0/290974830 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634012920 con 0x561a5b5d29c0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.799+0000 7f46327fc700 1 -- 172.21.15.49:0/290974830 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4634012a80 con 0x561a5b5d29c0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.799+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a5b4eceb0 con 0x561a5b5d29c0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.800+0000 7f46327fc700 1 -- 172.21.15.49:0/290974830 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4634012c40 con 0x561a5b5d29c0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.800+0000 7f465d815b80 10 client.16890 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.800+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561a5b338c60 con 0x561a5b5d29c0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f46327fc700 1 -- 172.21.15.49:0/290974830 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a5b4eceb0 con 0x561a5b5d29c0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f46327fc700 1 -- 172.21.15.49:0/290974830 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f463401a3f0 con 0x561a5b5d29c0 2023-10-08T09:26:19.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f465d815b80 10 client.16890 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:19.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f465d815b80 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:19.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f465d815b80 20 client.16890 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:19.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f465d815b80 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b5e7d50 0x561a5b5ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f465d815b80 4 client.16890 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:26:19.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.801+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a5b51de10 con 0x561a5b5e7d50 2023-10-08T09:26:19.812 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.805+0000 7f464affd700 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b5e7d50 0x561a5b5ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.811+0000 7f464affd700 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b5e7d50 0x561a5b5ea1b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d42911425b9ec7 server_cookie=6d2bdc70007cc09a in_seq=0 out_seq=0 2023-10-08T09:26:19.813 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.811+0000 7f46327fc700 10 client.16890 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.812+0000 7f46327fc700 1 -- 172.21.15.49:0/290974830 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561a5b51de10 con 0x561a5b5e7d50 2023-10-08T09:26:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.812+0000 7f46327fc700 10 client.16890 handle_command_reply: tid=0 2023-10-08T09:26:19.814 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.812+0000 7f465d815b80 1 client.16890 shutdown 2023-10-08T09:26:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.815+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 msgr2=0x561a5b5dc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.818 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.815+0000 7f465d815b80 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5dc340 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4634011690 tx=0x7f4634011a50 comp rx=0 tx=0).stop 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.815+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 shutdown_connections 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.815+0000 7f465d815b80 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b5e7d50 0x561a5b5ea1b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.815+0000 7f465d815b80 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5dbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.815+0000 7f465d815b80 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d3800 0x561a5b5e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.815+0000 7f465d815b80 1 --2- 172.21.15.49:0/290974830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5dc340 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.816+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 >> 172.21.15.49:0/290974830 conn(0x561a5b546260 msgr2=0x561a5b5cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.816+0000 7f46327fc700 0 client.16890 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.816+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 shutdown_connections 2023-10-08T09:26:19.819 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.816+0000 7f465d815b80 1 -- 172.21.15.49:0/290974830 wait complete. 2023-10-08T09:26:19.820 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.816+0000 7f465d815b80 20 client.16890 trim_cache size 0 max 16384 2023-10-08T09:26:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.835+0000 7f465d815b80 1 Processor -- start 2023-10-08T09:26:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.835+0000 7f465d815b80 1 -- start start 2023-10-08T09:26:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f465d815b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f465d815b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f465d815b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d3800 0x561a5b5e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f465d815b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b3a5920 con 0x561a5b5d3800 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f465d815b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b5e5110 con 0x561a5b5d29c0 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f465d815b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b5da870 con 0x561a5b5d2060 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464a7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d3800 0x561a5b5e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464affd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464a7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42938/0 (socket says 172.21.15.49:42938) 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464a7fc700 1 -- 172.21.15.49:0/2995242352 learned_addr learned my addr 172.21.15.49:0/2995242352 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d3800 0x561a5b5e32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53326/0 (socket says 172.21.15.49:53326) 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464affd700 1 -- 172.21.15.49:0/2995242352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 msgr2=0x561a5b5d6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:19.840 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464affd700 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464affd700 1 -- 172.21.15.49:0/2995242352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d3800 msgr2=0x561a5b5e32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:19.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464affd700 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d3800 0x561a5b5e32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464affd700 1 -- 172.21.15.49:0/2995242352 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a5b376250 con 0x561a5b5d2060 2023-10-08T09:26:19.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464b7fe700 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d3800 0x561a5b5e32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:26:19.841 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.836+0000 7f464affd700 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5d68a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f46440093a0 tx=0x7f4644009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81291e19604bce41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:19.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.838+0000 7f4641ffb700 1 -- 172.21.15.49:0/2995242352 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464400e040 con 0x561a5b5d2060 2023-10-08T09:26:19.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.838+0000 7f4641ffb700 1 -- 172.21.15.49:0/2995242352 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4644030070 con 0x561a5b5d2060 2023-10-08T09:26:19.845 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.838+0000 7f4641ffb700 1 -- 172.21.15.49:0/2995242352 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4644030440 con 0x561a5b5d2060 2023-10-08T09:26:19.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.838+0000 7f465d815b80 1 -- 172.21.15.49:0/2995242352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 msgr2=0x561a5b5d68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.838+0000 7f465d815b80 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5d68a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f46440093a0 tx=0x7f4644009040 comp rx=0 tx=0).stop 2023-10-08T09:26:19.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- 172.21.15.49:0/2995242352 shutdown_connections 2023-10-08T09:26:19.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d2060 0x561a5b5d68a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.846 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d3800 0x561a5b5e32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 --2- 172.21.15.49:0/2995242352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d29c0 0x561a5b5d6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- 172.21.15.49:0/2995242352 >> 172.21.15.49:0/2995242352 conn(0x561a5b546260 msgr2=0x561a5b5e9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:19.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- 172.21.15.49:0/2995242352 shutdown_connections 2023-10-08T09:26:19.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- 172.21.15.49:0/2995242352 wait complete. 2023-10-08T09:26:19.847 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- start start 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 Processor -- start 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d2060 0x561a5b5e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 0x561a5b5e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b63bbf0 con 0x561a5b5d29c0 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b63bd30 con 0x561a5b5d2060 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a5b63be70 con 0x561a5b5d3800 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f465d815b80 10 client.0 _ll_register_callbacks cb 0x7f4634009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.839+0000 7f464a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5e2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53330/0 (socket says 172.21.15.49:53330) 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 -- 172.21.15.49:0/3919427240 learned_addr learned my addr 172.21.15.49:0/3919427240 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464b7fe700 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 0x561a5b5e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464affd700 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d2060 0x561a5b5e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 -- 172.21.15.49:0/3919427240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 msgr2=0x561a5b5e25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 0x561a5b5e25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 -- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d2060 msgr2=0x561a5b5e1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d2060 0x561a5b5e1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 -- 172.21.15.49:0/3919427240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c10f5f0 con 0x561a5b5d29c0 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f464a7fc700 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5e2070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f463401a850 tx=0x7f463401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca42136cc70d82d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:19.849 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f46327fc700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f46327fc700 1 -- 172.21.15.49:0/3919427240 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634005040 con 0x561a5b5d29c0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f46327fc700 1 -- 172.21.15.49:0/3919427240 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f46340051a0 con 0x561a5b5d29c0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f464401ae20 con 0x561a5b5d29c0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f46327fc700 1 -- 172.21.15.49:0/3919427240 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4634005360 con 0x561a5b5d29c0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f465d815b80 10 client.16896 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.840+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561a5b34fde0 con 0x561a5b5d29c0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f46327fc700 1 -- 172.21.15.49:0/3919427240 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464401ae20 con 0x561a5b5d29c0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f46327fc700 1 -- 172.21.15.49:0/3919427240 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f463402b070 con 0x561a5b5d29c0 2023-10-08T09:26:19.850 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f465d815b80 10 client.16896 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:19.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f465d815b80 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:19.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f465d815b80 20 client.16896 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:19.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f465d815b80 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b63d8a0 0x561a5b63fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:19.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f465d815b80 4 client.16896 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:26:19.851 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.841+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a5b51db30 con 0x561a5b63d8a0 2023-10-08T09:26:19.852 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.844+0000 7f464affd700 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b63d8a0 0x561a5b63fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f464affd700 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b63d8a0 0x561a5b63fcc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab6b488b733ae40 server_cookie=f3a8cf7550f8e489 in_seq=0 out_seq=0 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f46327fc700 10 client.16896 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f46327fc700 1 -- 172.21.15.49:0/3919427240 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561a5b51db30 con 0x561a5b63d8a0 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f46327fc700 10 client.16896 handle_command_reply: tid=0 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 client.16896 shutdown 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 msgr2=0x561a5b5e2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5e2070 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f463401a850 tx=0x7f463401abd0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 shutdown_connections 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561a5b63d8a0 0x561a5b63fcc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561a5b5d3800 0x561a5b5e25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.853 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f46327fc700 0 client.16896 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:19.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561a5b5d2060 0x561a5b5e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 --2- 172.21.15.49:0/3919427240 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561a5b5d29c0 0x561a5b5e2070 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 >> 172.21.15.49:0/3919427240 conn(0x561a5b546260 msgr2=0x561a5b5d68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:19.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 shutdown_connections 2023-10-08T09:26:19.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.845+0000 7f465d815b80 1 -- 172.21.15.49:0/3919427240 wait complete. 2023-10-08T09:26:19.854 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465d815b80 20 client.16896 trim_cache size 0 max 16384 2023-10-08T09:26:19.854 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:26:19.855 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:26:19.855 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:26:19.855 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:26:19.855 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:26:19.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:26:19.855 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:26:19.855 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:26:19.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:26:19.856 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:26:19.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46380a1680 msgr2=0x7f46380a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46380a1680 0x7f46380a3b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4644009aa0 tx=0x7f4644009940 comp rx=0 tx=0).stop 2023-10-08T09:26:19.857 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101230 msgr2=0x7f464c071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:19.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101230 0x7f464c071990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f463c006010 tx=0x7f463c00f460 comp rx=0 tx=0).stop 2023-10-08T09:26:19.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 shutdown_connections 2023-10-08T09:26:19.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f46380a1680 0x7f46380a3b40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f464c101230 0x7f464c071990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.858 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f464c0ff6a0 0x7f464c06a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 --2- 172.21.15.49:0/3078403062 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f464c100890 0x7f464c06a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:19.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.846+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 >> 172.21.15.49:0/3078403062 conn(0x7f464c0fac10 msgr2=0x7f464c10caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:19.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.847+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 shutdown_connections 2023-10-08T09:26:19.859 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:19.847+0000 7f465162c700 1 -- 172.21.15.49:0/3078403062 wait complete. 2023-10-08T09:26:19.861 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:19 smithi049 ceph-mon[99485]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 564 op/s 2023-10-08T09:26:19.862 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:26:19.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:19 smithi184 ceph-mon[103675]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 564 op/s 2023-10-08T09:26:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:19 smithi132 ceph-mon[103389]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 564 op/s 2023-10-08T09:26:21.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:21 smithi184 ceph-mon[103675]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 519 op/s 2023-10-08T09:26:22.051 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:21 smithi132 ceph-mon[103389]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 519 op/s 2023-10-08T09:26:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:21 smithi049 ceph-mon[99485]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.4 MiB/s wr, 519 op/s 2023-10-08T09:26:23.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:23 smithi049 ceph-mon[99485]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 579 op/s 2023-10-08T09:26:23.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:23 smithi184 ceph-mon[103675]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 579 op/s 2023-10-08T09:26:23.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:23 smithi132 ceph-mon[103389]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 579 op/s 2023-10-08T09:26:24.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 660.0 s, 493.6 tps, lat 201.403 ms stddev 259.648 2023-10-08T09:26:25.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:25 smithi049 ceph-mon[99485]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 553 op/s 2023-10-08T09:26:26.005 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:25 smithi184 ceph-mon[103675]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 553 op/s 2023-10-08T09:26:26.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:25 smithi132 ceph-mon[103389]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.3 MiB/s wr, 553 op/s 2023-10-08T09:26:27.999 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:27 smithi184 ceph-mon[103675]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 578 op/s 2023-10-08T09:26:28.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:27 smithi132 ceph-mon[103389]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 578 op/s 2023-10-08T09:26:28.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:27 smithi049 ceph-mon[99485]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 578 op/s 2023-10-08T09:26:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.683+0000 7efdab725700 1 -- 172.21.15.49:0/3947042888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4077ee0 msgr2=0x7efda407a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.683+0000 7efdab725700 1 --2- 172.21.15.49:0/3947042888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4077ee0 0x7efda407a370 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00b0a0 tx=0x7efd9c00b3b0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.690+0000 7efdab725700 1 -- 172.21.15.49:0/3947042888 shutdown_connections 2023-10-08T09:26:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.690+0000 7efdab725700 1 --2- 172.21.15.49:0/3947042888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40725e0 0x7efda4072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.690+0000 7efdab725700 1 --2- 172.21.15.49:0/3947042888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4077ee0 0x7efda407a370 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.690+0000 7efdab725700 1 --2- 172.21.15.49:0/3947042888 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4075510 0x7efda40779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.690+0000 7efdab725700 1 -- 172.21.15.49:0/3947042888 >> 172.21.15.49:0/3947042888 conn(0x7efda406d770 msgr2=0x7efda406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.701+0000 7efdab725700 1 -- 172.21.15.49:0/3947042888 shutdown_connections 2023-10-08T09:26:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.709+0000 7efdab725700 1 -- 172.21.15.49:0/3947042888 wait complete. 2023-10-08T09:26:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 Processor -- start 2023-10-08T09:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 -- start start 2023-10-08T09:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40725e0 0x7efda407ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4075510 0x7efda407f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4077ee0 0x7efda4082380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda4082950 con 0x7efda4075510 2023-10-08T09:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda4082a90 con 0x7efda4077ee0 2023-10-08T09:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.710+0000 7efdab725700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda4082c20 con 0x7efda40725e0 2023-10-08T09:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.711+0000 7efda9cc2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4077ee0 0x7efda4082380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.711+0000 7efda9cc2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4077ee0 0x7efda4082380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42956/0 (socket says 172.21.15.49:42956) 2023-10-08T09:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.711+0000 7efda9cc2700 1 -- 172.21.15.49:0/1655225897 learned_addr learned my addr 172.21.15.49:0/1655225897 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.713+0000 7efda9cc2700 1 -- 172.21.15.49:0/1655225897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40725e0 msgr2=0x7efda407ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.713+0000 7efda9cc2700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40725e0 0x7efda407ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.713+0000 7efda9cc2700 1 -- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4075510 msgr2=0x7efda407f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.713+0000 7efda9cc2700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4075510 0x7efda407f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.713+0000 7efda9cc2700 1 -- 172.21.15.49:0/1655225897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c009d00 con 0x7efda4077ee0 2023-10-08T09:26:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.713+0000 7efda9cc2700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4077ee0 0x7efda4082380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007d00 tx=0x7efd9c005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.714+0000 7efd9a7fc700 1 -- 172.21.15.49:0/1655225897 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00e040 con 0x7efda4077ee0 2023-10-08T09:26:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.714+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda4082ea0 con 0x7efda4077ee0 2023-10-08T09:26:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.714+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda4083420 con 0x7efda4077ee0 2023-10-08T09:26:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.714+0000 7efd9a7fc700 1 -- 172.21.15.49:0/1655225897 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd9c003e10 con 0x7efda4077ee0 2023-10-08T09:26:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.714+0000 7efd9a7fc700 1 -- 172.21.15.49:0/1655225897 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0055a0 con 0x7efda4077ee0 2023-10-08T09:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.719+0000 7efdb790eb80 1 Processor -- start 2023-10-08T09:26:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.720+0000 7efd9a7fc700 1 -- 172.21.15.49:0/1655225897 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efd9c0057a0 con 0x7efda4077ee0 2023-10-08T09:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.721+0000 7efd9a7fc700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd900a1500 0x7efd900a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efd9a7fc700 1 -- 172.21.15.49:0/1655225897 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efd9c0ce5b0 con 0x7efda4077ee0 2023-10-08T09:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efdb790eb80 1 -- start start 2023-10-08T09:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e362260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e362cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 0x55ec6e363730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efdb790eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e2ca400 con 0x55ec6e361e40 2023-10-08T09:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efdb790eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e2bbbc0 con 0x55ec6e3627a0 2023-10-08T09:26:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.722+0000 7efdb790eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e2b94e0 con 0x55ec6e3632b0 2023-10-08T09:26:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.724+0000 7efda8cc0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e362cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.724+0000 7efda8cc0700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd900a1500 0x7efd900a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.724+0000 7efda8cc0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e362cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42966/0 (socket says 172.21.15.49:42966) 2023-10-08T09:26:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.724+0000 7efda8cc0700 1 -- 172.21.15.49:0/2205607012 learned_addr learned my addr 172.21.15.49:0/2205607012 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.725+0000 7efda8cc0700 1 -- 172.21.15.49:0/2205607012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 msgr2=0x55ec6e363730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.725+0000 7efda8cc0700 1 --2- 172.21.15.49:0/2205607012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 0x55ec6e363730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.725+0000 7efda8cc0700 1 -- 172.21.15.49:0/2205607012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 msgr2=0x55ec6e362260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.725+0000 7efda8cc0700 1 --2- 172.21.15.49:0/2205607012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e362260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.725+0000 7efda8cc0700 1 -- 172.21.15.49:0/2205607012 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec6e0dfd80 con 0x55ec6e3627a0 2023-10-08T09:26:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.725+0000 7efda8cc0700 1 --2- 172.21.15.49:0/2205607012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e362cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd94012c60 tx=0x7efd94012f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce371fd3d13c134c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.725+0000 7efda8cc0700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd900a1500 0x7efd900a39c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efd94014770 tx=0x7efd94010000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.726+0000 7efd8f7fe700 1 -- 172.21.15.49:0/2205607012 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94004220 con 0x55ec6e3627a0 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.726+0000 7efd8f7fe700 1 -- 172.21.15.49:0/2205607012 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd94004de0 con 0x55ec6e3627a0 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.726+0000 7efd8f7fe700 1 -- 172.21.15.49:0/2205607012 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94015b20 con 0x55ec6e3627a0 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.727+0000 7efdb790eb80 1 -- 172.21.15.49:0/2205607012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 msgr2=0x55ec6e362cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.727+0000 7efdb790eb80 1 --2- 172.21.15.49:0/2205607012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e362cd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd94012c60 tx=0x7efd94012f70 comp rx=0 tx=0).stop 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 -- 172.21.15.49:0/2205607012 shutdown_connections 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 --2- 172.21.15.49:0/2205607012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 0x55ec6e363730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 --2- 172.21.15.49:0/2205607012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e362260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 --2- 172.21.15.49:0/2205607012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e362cd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 -- 172.21.15.49:0/2205607012 >> 172.21.15.49:0/2205607012 conn(0x55ec6e2d6200 msgr2=0x55ec6e35c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 -- 172.21.15.49:0/2205607012 shutdown_connections 2023-10-08T09:26:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 -- 172.21.15.49:0/2205607012 wait complete. 2023-10-08T09:26:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.728+0000 7efdb790eb80 1 -- start start 2023-10-08T09:26:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 1 Processor -- start 2023-10-08T09:26:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e361e40 0x55ec6e36b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3627a0 0x55ec6e36be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 0x55ec6e372e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e2b4dd0 con 0x55ec6e3627a0 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e2516b0 con 0x55ec6e361e40 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e251350 con 0x55ec6e3632b0 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.729+0000 7efdb790eb80 10 client.0 _ll_register_callbacks cb 0x55ec6e0ae7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e361e40 0x55ec6e36b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e361e40 0x55ec6e36b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42970/0 (socket says 172.21.15.49:42970) 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 -- 172.21.15.49:0/3588592394 learned_addr learned my addr 172.21.15.49:0/3588592394 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda9cc2700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 0x55ec6e372e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda8cc0700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3627a0 0x55ec6e36be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 -- 172.21.15.49:0/3588592394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 msgr2=0x55ec6e372e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 0x55ec6e372e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 -- 172.21.15.49:0/3588592394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3627a0 msgr2=0x55ec6e36be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3627a0 0x55ec6e36be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda94c1700 1 -- 172.21.15.49:0/3588592394 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd94012810 con 0x55ec6e361e40 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.730+0000 7efda8cc0700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3627a0 0x55ec6e36be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.731+0000 7efda94c1700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e361e40 0x55ec6e36b910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efda0009f00 tx=0x7efda000b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82d0ee1aba66b61 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.731+0000 7efd8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.731+0000 7efd8cff9700 1 -- 172.21.15.49:0/3588592394 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0011420 con 0x55ec6e361e40 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.731+0000 7efd8cff9700 1 -- 172.21.15.49:0/3588592394 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efda0011580 con 0x55ec6e361e40 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.731+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec6e27ce50 con 0x55ec6e361e40 2023-10-08T09:26:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.732+0000 7efd8cff9700 1 -- 172.21.15.49:0/3588592394 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efda0011740 con 0x55ec6e361e40 2023-10-08T09:26:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 10 client.26474 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ec6e0c8c00 con 0x55ec6e361e40 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efd8cff9700 1 -- 172.21.15.49:0/3588592394 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec6e27ce50 con 0x55ec6e361e40 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efd8cff9700 1 -- 172.21.15.49:0/3588592394 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efda0019930 con 0x55ec6e361e40 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 10 client.26474 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 20 client.26474 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3777d0 0x55ec6e379bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 4 client.26474 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.734+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec6e2addb0 con 0x55ec6e3777d0 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.735+0000 7efda8cc0700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3777d0 0x55ec6e379bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.735+0000 7efd8cff9700 10 client.26474 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.735+0000 7efda8cc0700 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3777d0 0x55ec6e379bf0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6100c1596d5eac7 server_cookie=9e87c33c4c28d325 in_seq=0 out_seq=0 2023-10-08T09:26:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.736+0000 7efd8cff9700 1 -- 172.21.15.49:0/3588592394 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec6e2addb0 con 0x55ec6e3777d0 2023-10-08T09:26:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.736+0000 7efd8cff9700 10 client.26474 handle_command_reply: tid=0 2023-10-08T09:26:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.736+0000 7efdb790eb80 1 client.26474 shutdown 2023-10-08T09:26:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.736+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e361e40 msgr2=0x55ec6e36b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e361e40 0x55ec6e36b910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efda0009f00 tx=0x7efda000b3c0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 shutdown_connections 2023-10-08T09:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3777d0 0x55ec6e379bf0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3632b0 0x55ec6e372e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e361e40 0x55ec6e36b910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 --2- 172.21.15.49:0/3588592394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3627a0 0x55ec6e36be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 >> 172.21.15.49:0/3588592394 conn(0x55ec6e2d6200 msgr2=0x55ec6e35c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efd8cff9700 0 client.26474 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 shutdown_connections 2023-10-08T09:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 1 -- 172.21.15.49:0/3588592394 wait complete. 2023-10-08T09:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.737+0000 7efdb790eb80 20 client.26474 trim_cache size 0 max 16384 2023-10-08T09:26:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.761+0000 7efdb790eb80 1 Processor -- start 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.761+0000 7efdb790eb80 1 -- start start 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e361e40 0x55ec6e3636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e372db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3632b0 0x55ec6e3698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efdb790eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e1358c0 con 0x55ec6e3632b0 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efdb790eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e363160 con 0x55ec6e3627a0 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efdb790eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e374c20 con 0x55ec6e361e40 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda9cc2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3632b0 0x55ec6e3698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda8cc0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e372db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e361e40 0x55ec6e3636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e361e40 0x55ec6e3636d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40398/0 (socket says 172.21.15.49:40398) 2023-10-08T09:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda8cc0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e372db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42978/0 (socket says 172.21.15.49:42978) 2023-10-08T09:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 -- 172.21.15.49:0/107745810 learned_addr learned my addr 172.21.15.49:0/107745810 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 -- 172.21.15.49:0/107745810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 msgr2=0x55ec6e372db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 --2- 172.21.15.49:0/107745810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e372db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 -- 172.21.15.49:0/107745810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3632b0 msgr2=0x55ec6e3698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 --2- 172.21.15.49:0/107745810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3632b0 0x55ec6e3698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.762+0000 7efda94c1700 1 -- 172.21.15.49:0/107745810 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec6e1061f0 con 0x55ec6e361e40 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.763+0000 7efda94c1700 1 --2- 172.21.15.49:0/107745810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e361e40 0x55ec6e3636d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efda000b0b0 tx=0x7efda000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a16ef1dc6fdd516 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.763+0000 7efd8ffff700 1 -- 172.21.15.49:0/107745810 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0010380 con 0x55ec6e361e40 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.763+0000 7efd8ffff700 1 -- 172.21.15.49:0/107745810 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efda000bd20 con 0x55ec6e361e40 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.763+0000 7efd8ffff700 1 -- 172.21.15.49:0/107745810 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0010a30 con 0x55ec6e361e40 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.764+0000 7efdb790eb80 1 -- 172.21.15.49:0/107745810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e361e40 msgr2=0x55ec6e3636d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.764+0000 7efdb790eb80 1 --2- 172.21.15.49:0/107745810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e361e40 0x55ec6e3636d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efda000b0b0 tx=0x7efda000b8f0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 -- 172.21.15.49:0/107745810 shutdown_connections 2023-10-08T09:26:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 --2- 172.21.15.49:0/107745810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e361e40 0x55ec6e3636d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 --2- 172.21.15.49:0/107745810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e3632b0 0x55ec6e3698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 --2- 172.21.15.49:0/107745810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3627a0 0x55ec6e372db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 -- 172.21.15.49:0/107745810 >> 172.21.15.49:0/107745810 conn(0x55ec6e2d6200 msgr2=0x55ec6e37e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 -- 172.21.15.49:0/107745810 shutdown_connections 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 -- 172.21.15.49:0/107745810 wait complete. 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.765+0000 7efdb790eb80 1 -- start start 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 1 Processor -- start 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e370710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3627a0 0x55ec6e370c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3632b0 0x55ec6e371c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e3721a0 con 0x55ec6e361e40 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e372310 con 0x55ec6e3632b0 2023-10-08T09:26:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec6e3724d0 con 0x55ec6e3627a0 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efdb790eb80 10 client.0 _ll_register_callbacks cb 0x7efda0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efda94c1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e370710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efda9cc2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3632b0 0x55ec6e371c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efda8cc0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3627a0 0x55ec6e370c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efda9cc2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3632b0 0x55ec6e371c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:42988/0 (socket says 172.21.15.49:42988) 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efda94c1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e370710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53370/0 (socket says 172.21.15.49:53370) 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efda9cc2700 1 -- 172.21.15.49:0/1936472609 learned_addr learned my addr 172.21.15.49:0/1936472609 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.766+0000 7efda9cc2700 1 -- 172.21.15.49:0/1936472609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3627a0 msgr2=0x55ec6e370c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efda9cc2700 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3627a0 0x55ec6e370c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efda9cc2700 1 -- 172.21.15.49:0/1936472609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 msgr2=0x55ec6e370710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efda9cc2700 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e370710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efda9cc2700 1 -- 172.21.15.49:0/1936472609 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec6e0c8c00 con 0x55ec6e3632b0 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efda94c1700 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e370710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efda9cc2700 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3632b0 0x55ec6e371c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efd9c097bc0 tx=0x7efd9c09b020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289a05a28d6ac45 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efd8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efd8cff9700 1 -- 172.21.15.49:0/1936472609 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c09f810 con 0x55ec6e3632b0 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd94004de0 con 0x55ec6e3632b0 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efd8cff9700 1 -- 172.21.15.49:0/1936472609 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7efd9c0a39c0 con 0x55ec6e3632b0 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efd8cff9700 1 -- 172.21.15.49:0/1936472609 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd9c0860b0 con 0x55ec6e3632b0 2023-10-08T09:26:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efdb790eb80 10 client.26480 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55ec6e0de250 con 0x55ec6e3632b0 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efd8cff9700 1 -- 172.21.15.49:0/1936472609 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd9c07e020 con 0x55ec6e3632b0 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.767+0000 7efd8cff9700 1 -- 172.21.15.49:0/1936472609 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efd9c0ce070 con 0x55ec6e3632b0 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efdb790eb80 10 client.26480 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efdb790eb80 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efdb790eb80 20 client.26480 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efdb790eb80 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3cbd90 0x55ec6e3ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efdb790eb80 4 client.26480 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec6e2adad0 con 0x55ec6e3cbd90 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efda94c1700 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3cbd90 0x55ec6e3ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efd8cff9700 10 client.26480 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.768+0000 7efda94c1700 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3cbd90 0x55ec6e3ce180 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff832cabfbdf8b8 server_cookie=22cccfdc5d925096 in_seq=0 out_seq=0 2023-10-08T09:26:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efd8cff9700 1 -- 172.21.15.49:0/1936472609 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec6e2adad0 con 0x55ec6e3cbd90 2023-10-08T09:26:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efd8cff9700 10 client.26480 handle_command_reply: tid=0 2023-10-08T09:26:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 client.26480 shutdown 2023-10-08T09:26:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3632b0 msgr2=0x55ec6e371c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3632b0 0x55ec6e371c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efd9c097bc0 tx=0x7efd9c09b020 comp rx=0 tx=0).stop 2023-10-08T09:26:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 shutdown_connections 2023-10-08T09:26:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ec6e3cbd90 0x55ec6e3ce180 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ec6e3632b0 0x55ec6e371c60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ec6e361e40 0x55ec6e370710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 --2- 172.21.15.49:0/1936472609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ec6e3627a0 0x55ec6e370c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 >> 172.21.15.49:0/1936472609 conn(0x55ec6e2d6200 msgr2=0x55ec6e36aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.777+0000 7efd8cff9700 0 client.26480 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.778+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 shutdown_connections 2023-10-08T09:26:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.778+0000 7efdb790eb80 1 -- 172.21.15.49:0/1936472609 wait complete. 2023-10-08T09:26:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.778+0000 7efdb790eb80 20 client.26480 trim_cache size 0 max 16384 2023-10-08T09:26:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:26:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:26:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:26:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:26:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd900a1500 msgr2=0x7efd900a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd900a1500 0x7efd900a39c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efd94014770 tx=0x7efd94010000 comp rx=0 tx=0).stop 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4077ee0 msgr2=0x7efda4082380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4077ee0 0x7efda4082380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007d00 tx=0x7efd9c005b80 comp rx=0 tx=0).stop 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 shutdown_connections 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7efd900a1500 0x7efd900a39c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efda40725e0 0x7efda407ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efda4077ee0 0x7efda4082380 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 --2- 172.21.15.49:0/1655225897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efda4075510 0x7efda407f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 >> 172.21.15.49:0/1655225897 conn(0x7efda406d770 msgr2=0x7efda4076b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 shutdown_connections 2023-10-08T09:26:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:28.779+0000 7efdab725700 1 -- 172.21.15.49:0/1655225897 wait complete. 2023-10-08T09:26:29.007 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:28 smithi184 ceph-mon[103675]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 541 op/s 2023-10-08T09:26:29.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:28 smithi132 ceph-mon[103389]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 541 op/s 2023-10-08T09:26:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:28 smithi049 ceph-mon[99485]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 541 op/s 2023-10-08T09:26:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 665.0 s, 549.6 tps, lat 180.817 ms stddev 217.701 2023-10-08T09:26:31.885 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:31 smithi049 ceph-mon[99485]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 491 op/s 2023-10-08T09:26:31.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:31 smithi184 ceph-mon[103675]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 491 op/s 2023-10-08T09:26:32.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:31 smithi132 ceph-mon[103389]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.4 MiB/s wr, 491 op/s 2023-10-08T09:26:33.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:33 smithi132 ceph-mon[103389]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.8 MiB/s wr, 535 op/s 2023-10-08T09:26:33.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:33 smithi049 ceph-mon[99485]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.8 MiB/s wr, 535 op/s 2023-10-08T09:26:33.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:33 smithi184 ceph-mon[103675]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.8 MiB/s wr, 535 op/s 2023-10-08T09:26:34.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 670.0 s, 477.2 tps, lat 211.113 ms stddev 245.937 2023-10-08T09:26:36.003 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:35 smithi184 ceph-mon[103675]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 6.1 MiB/s wr, 503 op/s 2023-10-08T09:26:36.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:35 smithi132 ceph-mon[103389]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 6.1 MiB/s wr, 503 op/s 2023-10-08T09:26:36.139 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:35 smithi049 ceph-mon[99485]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 6.1 MiB/s wr, 503 op/s 2023-10-08T09:26:37.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:37 smithi184 ceph-mon[103675]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 7.0 MiB/s wr, 548 op/s 2023-10-08T09:26:38.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:37 smithi132 ceph-mon[103389]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 7.0 MiB/s wr, 548 op/s 2023-10-08T09:26:38.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:37 smithi049 ceph-mon[99485]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 7.0 MiB/s wr, 548 op/s 2023-10-08T09:26:39.314 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 675.0 s, 492.2 tps, lat 203.238 ms stddev 214.383 2023-10-08T09:26:39.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:39 smithi184 ceph-mon[103675]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 6.6 MiB/s wr, 540 op/s 2023-10-08T09:26:40.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:39 smithi132 ceph-mon[103389]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 6.6 MiB/s wr, 540 op/s 2023-10-08T09:26:40.141 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:39 smithi049 ceph-mon[99485]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 6.6 MiB/s wr, 540 op/s 2023-10-08T09:26:42.001 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:41 smithi184 ceph-mon[103675]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 6.7 MiB/s wr, 515 op/s 2023-10-08T09:26:42.001 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:41 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.021+0000 7f07e6254700 1 -- 172.21.15.49:0/2838913030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 msgr2=0x7f07e010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.021+0000 7f07e6254700 1 --2- 172.21.15.49:0/2838913030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 0x7f07e010cfb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009f90 tx=0x7f07d800b330 comp rx=0 tx=0).stop 2023-10-08T09:26:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.032+0000 7f07e6254700 1 -- 172.21.15.49:0/2838913030 shutdown_connections 2023-10-08T09:26:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.032+0000 7f07e6254700 1 --2- 172.21.15.49:0/2838913030 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 0x7f07e010cfb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.032+0000 7f07e6254700 1 --2- 172.21.15.49:0/2838913030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07e01077f0 0x7f07e0109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.032+0000 7f07e6254700 1 --2- 172.21.15.49:0/2838913030 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07e010a160 0x7f07e010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.032+0000 7f07e6254700 1 -- 172.21.15.49:0/2838913030 >> 172.21.15.49:0/2838913030 conn(0x7f07e006d770 msgr2=0x7f07e006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.040+0000 7f07e6254700 1 -- 172.21.15.49:0/2838913030 shutdown_connections 2023-10-08T09:26:42.046 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:41 smithi132 ceph-mon[103389]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 6.7 MiB/s wr, 515 op/s 2023-10-08T09:26:42.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:41 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:26:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.050+0000 7f07e6254700 1 -- 172.21.15.49:0/2838913030 wait complete. 2023-10-08T09:26:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.051+0000 7f07e6254700 1 Processor -- start 2023-10-08T09:26:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.056+0000 7f07e6254700 1 -- start start 2023-10-08T09:26:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.056+0000 7f07e6254700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07e01077f0 0x7f07e01a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.056+0000 7f07e6254700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07e010a160 0x7f07e01a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.056+0000 7f07e6254700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 0x7f07e01aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.056+0000 7f07e6254700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01ad060 con 0x7f07e010cb30 2023-10-08T09:26:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.056+0000 7f07e6254700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01ad1a0 con 0x7f07e01077f0 2023-10-08T09:26:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.056+0000 7f07e6254700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01ad330 con 0x7f07e010a160 2023-10-08T09:26:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.057+0000 7f07dffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 0x7f07e01aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.057+0000 7f07dffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 0x7f07e01aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53386/0 (socket says 172.21.15.49:53386) 2023-10-08T09:26:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.057+0000 7f07dffff700 1 -- 172.21.15.49:0/3065727857 learned_addr learned my addr 172.21.15.49:0/3065727857 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:42.058 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.057+0000 7f07deffd700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07e010a160 0x7f07e01a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.061+0000 7f07df7fe700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07e01077f0 0x7f07e01a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.062+0000 7f07deffd700 1 -- 172.21.15.49:0/3065727857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07e01077f0 msgr2=0x7f07e01a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.062+0000 7f07deffd700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07e01077f0 0x7f07e01a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.062+0000 7f07deffd700 1 -- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 msgr2=0x7f07e01aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.062+0000 7f07deffd700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 0x7f07e01aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.062+0000 7f07deffd700 1 -- 172.21.15.49:0/3065727857 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8009c40 con 0x7f07e010a160 2023-10-08T09:26:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.063+0000 7f07deffd700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07e010a160 0x7f07e01a5ac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07d400e9c0 tx=0x7f07d400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.066+0000 7f07dcff9700 1 -- 172.21.15.49:0/3065727857 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d400c9c0 con 0x7f07e010a160 2023-10-08T09:26:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.066+0000 7f07dcff9700 1 -- 172.21.15.49:0/3065727857 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07d4010430 con 0x7f07e010a160 2023-10-08T09:26:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.066+0000 7f07dcff9700 1 -- 172.21.15.49:0/3065727857 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4018680 con 0x7f07e010a160 2023-10-08T09:26:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.066+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e01ad610 con 0x7f07e010a160 2023-10-08T09:26:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.066+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e01adad0 con 0x7f07e010a160 2023-10-08T09:26:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.071+0000 7f07f243db80 1 Processor -- start 2023-10-08T09:26:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.073+0000 7f07dcff9700 1 -- 172.21.15.49:0/3065727857 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f07d400cb20 con 0x7f07e010a160 2023-10-08T09:26:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.073+0000 7f07f243db80 1 -- start start 2023-10-08T09:26:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07f243db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f247260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07f243db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f247cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07f243db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f248730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07f243db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f1af400 con 0x55d17f2477a0 2023-10-08T09:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07f243db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f1a0bc0 con 0x55d17f2482b0 2023-10-08T09:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07f243db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f19e4e0 con 0x55d17f246e40 2023-10-08T09:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07dcff9700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07c80a1650 0x7f07c80a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07dcff9700 1 -- 172.21.15.49:0/3065727857 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07d4014070 con 0x7f07e010a160 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07dffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f247260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07dffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f247260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40428/0 (socket says 172.21.15.49:40428) 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07dffff700 1 -- 172.21.15.49:0/385755811 learned_addr learned my addr 172.21.15.49:0/385755811 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07df7fe700 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f247cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.074+0000 7f07deffd700 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f248730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.075+0000 7f07deffd700 1 -- 172.21.15.49:0/385755811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 msgr2=0x55d17f247260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.075+0000 7f07deffd700 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f247260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.075+0000 7f07deffd700 1 -- 172.21.15.49:0/385755811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 msgr2=0x55d17f247cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.075+0000 7f07deffd700 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f247cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.075+0000 7f07deffd700 1 -- 172.21.15.49:0/385755811 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17efc4d80 con 0x55d17f2482b0 2023-10-08T09:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.075+0000 7f07dffff700 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f247260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.075+0000 7f07dffff700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07c80a1650 0x7f07c80a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.077+0000 7f07deffd700 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f248730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f07d4099fc0 tx=0x7f07d4099ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a62cade0e80642 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.083+0000 7f07cdffb700 1 -- 172.21.15.49:0/385755811 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d409a730 con 0x55d17f2482b0 2023-10-08T09:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.083+0000 7f07cdffb700 1 -- 172.21.15.49:0/385755811 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07d409a030 con 0x55d17f2482b0 2023-10-08T09:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.083+0000 7f07cdffb700 1 -- 172.21.15.49:0/385755811 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d409a4f0 con 0x55d17f2482b0 2023-10-08T09:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07f243db80 1 -- 172.21.15.49:0/385755811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 msgr2=0x55d17f248730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07dffff700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07c80a1650 0x7f07c80a3b10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f07d80078a0 tx=0x7f07d800b330 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07f243db80 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f248730 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f07d4099fc0 tx=0x7f07d4099ed0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07f243db80 1 -- 172.21.15.49:0/385755811 shutdown_connections 2023-10-08T09:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07f243db80 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f247260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07f243db80 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f248730 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07f243db80 1 --2- 172.21.15.49:0/385755811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f247cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.084+0000 7f07f243db80 1 -- 172.21.15.49:0/385755811 >> 172.21.15.49:0/385755811 conn(0x55d17f1bb200 msgr2=0x55d17f2418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.085+0000 7f07f243db80 1 -- 172.21.15.49:0/385755811 shutdown_connections 2023-10-08T09:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.085+0000 7f07f243db80 1 -- 172.21.15.49:0/385755811 wait complete. 2023-10-08T09:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 -- start start 2023-10-08T09:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 Processor -- start 2023-10-08T09:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f250850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f250d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f257dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f199dd0 con 0x55d17f2477a0 2023-10-08T09:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f1366b0 con 0x55d17f2482b0 2023-10-08T09:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.086+0000 7f07f243db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f136350 con 0x55d17f246e40 2023-10-08T09:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.087+0000 7f07f243db80 10 client.0 _ll_register_callbacks cb 0x55d17ef937a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.087+0000 7f07df7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f250850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.087+0000 7f07df7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f250850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40436/0 (socket says 172.21.15.49:40436) 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.087+0000 7f07df7fe700 1 -- 172.21.15.49:0/2750468784 learned_addr learned my addr 172.21.15.49:0/2750468784 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.087+0000 7f07dffff700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f257dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.088+0000 7f07df7fe700 1 -- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 msgr2=0x55d17f257dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.088+0000 7f07df7fe700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f257dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.088+0000 7f07df7fe700 1 -- 172.21.15.49:0/2750468784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 msgr2=0x55d17f250d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.088+0000 7f07deffd700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f250d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.088+0000 7f07df7fe700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f250d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.088+0000 7f07df7fe700 1 -- 172.21.15.49:0/2750468784 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e010da70 con 0x55d17f246e40 2023-10-08T09:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.088+0000 7f07deffd700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f250d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.089+0000 7f07df7fe700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f250850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f07d0010500 tx=0x7f07d00108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8528a5d04ac6024 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.089+0000 7f07c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.090+0000 7f07c2ffd700 1 -- 172.21.15.49:0/2750468784 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0014700 con 0x55d17f246e40 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.090+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d17f161e50 con 0x55d17f246e40 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.090+0000 7f07c2ffd700 1 -- 172.21.15.49:0/2750468784 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07d0014860 con 0x55d17f246e40 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.090+0000 7f07c2ffd700 1 -- 172.21.15.49:0/2750468784 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07d0014a20 con 0x55d17f246e40 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.090+0000 7f07f243db80 10 client.26560 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.090+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f07e01adad0 con 0x55d17f246e40 2023-10-08T09:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07c2ffd700 1 -- 172.21.15.49:0/2750468784 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d17f161e50 con 0x55d17f246e40 2023-10-08T09:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07c2ffd700 1 -- 172.21.15.49:0/2750468784 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07d0013d00 con 0x55d17f246e40 2023-10-08T09:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07f243db80 10 client.26560 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07f243db80 10 client.26560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07f243db80 20 client.26560 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07f243db80 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f26d770 0x55d17f26fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07f243db80 4 client.26560 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d17f192db0 con 0x55d17f26d770 2023-10-08T09:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.091+0000 7f07deffd700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f26d770 0x55d17f26fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.092+0000 7f07c2ffd700 10 client.26560 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.092+0000 7f07deffd700 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f26d770 0x55d17f26fb90 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4c842441bf8b9 server_cookie=2c8b3a091ef21297 in_seq=0 out_seq=0 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.093+0000 7f07c2ffd700 1 -- 172.21.15.49:0/2750468784 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d17f192db0 con 0x55d17f26d770 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.093+0000 7f07c2ffd700 10 client.26560 handle_command_reply: tid=0 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.093+0000 7f07f243db80 1 client.26560 shutdown 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.093+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 msgr2=0x55d17f250850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.093+0000 7f07f243db80 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f250850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f07d0010500 tx=0x7f07d00108c0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.093+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 shutdown_connections 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.093+0000 7f07f243db80 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f26d770 0x55d17f26fb90 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07f243db80 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f246e40 0x55d17f250850 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07f243db80 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f257dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07f243db80 1 --2- 172.21.15.49:0/2750468784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f250d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 >> 172.21.15.49:0/2750468784 conn(0x55d17f1bb200 msgr2=0x55d17f241010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07c2ffd700 0 client.26560 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 shutdown_connections 2023-10-08T09:26:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07f243db80 1 -- 172.21.15.49:0/2750468784 wait complete. 2023-10-08T09:26:42.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.094+0000 7f07f243db80 20 client.26560 trim_cache size 0 max 16384 2023-10-08T09:26:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 Processor -- start 2023-10-08T09:26:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 -- start start 2023-10-08T09:26:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f246e40 0x55d17f26fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f257d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2482b0 0x55d17f24fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f01a8c0 con 0x55d17f2477a0 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f23e830 con 0x55d17f246e40 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07f243db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f23c2b0 con 0x55d17f2482b0 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f257d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f257d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53410/0 (socket says 172.21.15.49:53410) 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 -- 172.21.15.49:0/3076193021 learned_addr learned my addr 172.21.15.49:0/3076193021 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07df7fe700 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f246e40 0x55d17f26fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07dffff700 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2482b0 0x55d17f24fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 -- 172.21.15.49:0/3076193021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2482b0 msgr2=0x55d17f24fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2482b0 0x55d17f24fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 -- 172.21.15.49:0/3076193021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f246e40 msgr2=0x55d17f26fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f246e40 0x55d17f26fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07deffd700 1 -- 172.21.15.49:0/3076193021 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17efadc00 con 0x55d17f2477a0 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.116+0000 7f07df7fe700 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f246e40 0x55d17f26fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.117+0000 7f07deffd700 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f257d90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f07d4096530 tx=0x7f07d4010bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9848a8e1b36edf14 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.117+0000 7f07ce7fc700 1 -- 172.21.15.49:0/3076193021 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40ca920 con 0x55d17f2477a0 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.117+0000 7f07ce7fc700 1 -- 172.21.15.49:0/3076193021 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07d40983d0 con 0x55d17f2477a0 2023-10-08T09:26:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.117+0000 7f07ce7fc700 1 -- 172.21.15.49:0/3076193021 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4099780 con 0x55d17f2477a0 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.118+0000 7f07f243db80 1 -- 172.21.15.49:0/3076193021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 msgr2=0x55d17f257d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.118+0000 7f07f243db80 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f257d90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f07d4096530 tx=0x7f07d4010bc0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.119+0000 7f07f243db80 1 -- 172.21.15.49:0/3076193021 shutdown_connections 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.119+0000 7f07f243db80 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2482b0 0x55d17f24fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.119+0000 7f07f243db80 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f246e40 0x55d17f26fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.119+0000 7f07f243db80 1 --2- 172.21.15.49:0/3076193021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f2477a0 0x55d17f257d90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.119+0000 7f07f243db80 1 -- 172.21.15.49:0/3076193021 >> 172.21.15.49:0/3076193021 conn(0x55d17f1bb200 msgr2=0x55d17f274510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.119+0000 7f07f243db80 1 -- 172.21.15.49:0/3076193021 shutdown_connections 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.119+0000 7f07f243db80 1 -- 172.21.15.49:0/3076193021 wait complete. 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.120+0000 7f07f243db80 1 -- start start 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.120+0000 7f07f243db80 1 Processor -- start 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07f243db80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f246e40 0x55d17f255b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07f243db80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2477a0 0x55d17f256080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07f243db80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f256fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07f243db80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f2574e0 con 0x55d17f246e40 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07f243db80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f257650 con 0x55d17f2482b0 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07f243db80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17f2577c0 con 0x55d17f2477a0 2023-10-08T09:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07f243db80 10 client.0 _ll_register_callbacks cb 0x7f07d000bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07dffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f256fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07dffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f256fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43028/0 (socket says 172.21.15.49:43028) 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.121+0000 7f07dffff700 1 -- 172.21.15.49:0/929297315 learned_addr learned my addr 172.21.15.49:0/929297315 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07dffff700 1 -- 172.21.15.49:0/929297315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2477a0 msgr2=0x55d17f256080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07dffff700 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2477a0 0x55d17f256080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07dffff700 1 -- 172.21.15.49:0/929297315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f246e40 msgr2=0x55d17f255b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07dffff700 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f246e40 0x55d17f255b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07dffff700 1 -- 172.21.15.49:0/929297315 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e01adad0 con 0x55d17f2482b0 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07dffff700 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f256fa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f07d8003ab0 tx=0x7f07d8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56fb7dd3d61841c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07c2ffd700 1 -- 172.21.15.49:0/929297315 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800e040 con 0x55d17f2482b0 2023-10-08T09:26:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07d409a030 con 0x55d17f2482b0 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.122+0000 7f07c2ffd700 1 -- 172.21.15.49:0/929297315 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f07d8020720 con 0x55d17f2482b0 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.123+0000 7f07c2ffd700 1 -- 172.21.15.49:0/929297315 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07d80208e0 con 0x55d17f2482b0 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.123+0000 7f07f243db80 10 client.26495 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.123+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d17efc4d80 con 0x55d17f2482b0 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.123+0000 7f07c2ffd700 1 -- 172.21.15.49:0/929297315 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07d409a030 con 0x55d17f2482b0 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07c2ffd700 1 -- 172.21.15.49:0/929297315 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f07d80057b0 con 0x55d17f2482b0 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07f243db80 10 client.26495 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07f243db80 10 client.26495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07f243db80 20 client.26495 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07f243db80 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f2e4550 0x55d17f2e6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07f243db80 4 client.26495 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:26:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d17f192ad0 con 0x55d17f2e4550 2023-10-08T09:26:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.124+0000 7f07df7fe700 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f2e4550 0x55d17f2e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.130+0000 7f07df7fe700 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f2e4550 0x55d17f2e6970 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a8fc4411f16727 server_cookie=abd141303edb254e in_seq=0 out_seq=0 2023-10-08T09:26:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.130+0000 7f07c2ffd700 10 client.26495 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:42.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:41 smithi049 ceph-mon[99485]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 6.7 MiB/s wr, 515 op/s 2023-10-08T09:26:42.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:41 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.147+0000 7f07c2ffd700 1 -- 172.21.15.49:0/929297315 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d17f192ad0 con 0x55d17f2e4550 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.147+0000 7f07c2ffd700 10 client.26495 handle_command_reply: tid=0 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.147+0000 7f07f243db80 1 client.26495 shutdown 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.147+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 msgr2=0x55d17f256fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.147+0000 7f07f243db80 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f256fa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f07d8003ab0 tx=0x7f07d8009920 comp rx=0 tx=0).stop 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 shutdown_connections 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d17f2e4550 0x55d17f2e6970 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d17f2482b0 0x55d17f256fa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d17f246e40 0x55d17f255b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 --2- 172.21.15.49:0/929297315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d17f2477a0 0x55d17f256080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.149 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 >> 172.21.15.49:0/929297315 conn(0x55d17f1bb200 msgr2=0x55d17f24fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07c2ffd700 0 client.26495 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 shutdown_connections 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 1 -- 172.21.15.49:0/929297315 wait complete. 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07f243db80 20 client.26495 trim_cache size 0 max 16384 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:26:42.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:26:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07c80a1650 msgr2=0x7f07c80a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07e6254700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07c80a1650 0x7f07c80a3b10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f07d80078a0 tx=0x7f07d800b330 comp rx=0 tx=0).stop 2023-10-08T09:26:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07e010a160 msgr2=0x7f07e01a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:42.151 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.148+0000 7f07e6254700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07e010a160 0x7f07e01a5ac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07d400e9c0 tx=0x7f07d400ecd0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 shutdown_connections 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f07c80a1650 0x7f07c80a3b10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07e010cb30 0x7f07e01aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f07e01077f0 0x7f07e01a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 --2- 172.21.15.49:0/3065727857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07e010a160 0x7f07e01a5ac0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 >> 172.21.15.49:0/3065727857 conn(0x7f07e006d770 msgr2=0x7f07e0108d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 shutdown_connections 2023-10-08T09:26:42.152 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:42.149+0000 7f07e6254700 1 -- 172.21.15.49:0/3065727857 wait complete. 2023-10-08T09:26:43.307 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:43 smithi132 ceph-mon[103389]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 6.6 MiB/s wr, 542 op/s 2023-10-08T09:26:43.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:43 smithi049 ceph-mon[99485]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 6.6 MiB/s wr, 542 op/s 2023-10-08T09:26:43.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:43 smithi184 ceph-mon[103675]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 6.6 MiB/s wr, 542 op/s 2023-10-08T09:26:44.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:26:44.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:26:44.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:26:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 680.0 s, 492.4 tps, lat 201.833 ms stddev 200.370 2023-10-08T09:26:44.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:26:44.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:26:44.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:26:44.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:26:44.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:26:44.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:26:45.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:45 smithi049 ceph-mon[99485]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 6.5 MiB/s wr, 507 op/s 2023-10-08T09:26:45.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:45 smithi184 ceph-mon[103675]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 6.5 MiB/s wr, 507 op/s 2023-10-08T09:26:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:45 smithi132 ceph-mon[103389]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 6.5 MiB/s wr, 507 op/s 2023-10-08T09:26:47.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:47 smithi184 ceph-mon[103675]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 543 op/s 2023-10-08T09:26:48.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:47 smithi132 ceph-mon[103389]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 543 op/s 2023-10-08T09:26:48.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:47 smithi049 ceph-mon[99485]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 543 op/s 2023-10-08T09:26:49.002 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:48 smithi184 ceph-mon[103675]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 526 op/s 2023-10-08T09:26:49.057 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:48 smithi132 ceph-mon[103389]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 526 op/s 2023-10-08T09:26:49.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:48 smithi049 ceph-mon[99485]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 526 op/s 2023-10-08T09:26:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 685.0 s, 545.8 tps, lat 183.120 ms stddev 205.070 2023-10-08T09:26:49.863 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:26:50.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.108+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/3183993839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2810cb30 msgr2=0x7f5a2810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.113 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.108+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/3183993839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2810cb30 0x7f5a2810cfb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a1801a9a0 tx=0x7f5a1801acb0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.129+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/3183993839 shutdown_connections 2023-10-08T09:26:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.129+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/3183993839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a281077f0 0x7f5a28109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.129+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/3183993839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2810cb30 0x7f5a2810cfb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.129+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/3183993839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a2810a160 0x7f5a2810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.131 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.129+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/3183993839 >> 172.21.15.49:0/3183993839 conn(0x7f5a2806d790 msgr2=0x7f5a2806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.146+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/3183993839 shutdown_connections 2023-10-08T09:26:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.147+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/3183993839 wait complete. 2023-10-08T09:26:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.147+0000 7f5a2ffd1700 1 Processor -- start 2023-10-08T09:26:50.153 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.147+0000 7f5a2ffd1700 1 -- start start 2023-10-08T09:26:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.148+0000 7f5a2ffd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a281077f0 0x7f5a2819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.148+0000 7f5a2ffd1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a2810a160 0x7f5a2819d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.148+0000 7f5a2ffd1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a2810cb30 0x7f5a28078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.148+0000 7f5a2ffd1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28079050 con 0x7f5a2810a160 2023-10-08T09:26:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.148+0000 7f5a2ffd1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a280791c0 con 0x7f5a281077f0 2023-10-08T09:26:50.154 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.148+0000 7f5a2ffd1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28079330 con 0x7f5a2810cb30 2023-10-08T09:26:50.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.153+0000 7f5a2e56e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a2810cb30 0x7f5a28078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.156 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.155+0000 7f5a2dd6d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a281077f0 0x7f5a2819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.155+0000 7f5a2dd6d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a281077f0 0x7f5a2819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43044/0 (socket says 172.21.15.49:43044) 2023-10-08T09:26:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.155+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/1029448021 learned_addr learned my addr 172.21.15.49:0/1029448021 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.156+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/1029448021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a2810cb30 msgr2=0x7f5a28078a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.156+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a2810cb30 0x7f5a28078a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.156+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a2810a160 msgr2=0x7f5a2819d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.156+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a2810a160 0x7f5a2819d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.156+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/1029448021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1801a600 con 0x7f5a281077f0 2023-10-08T09:26:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.157+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a281077f0 0x7f5a2819ccc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c920 tx=0x7f5a2400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.158+0000 7f5a1effd700 1 -- 172.21.15.49:0/1029448021 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a24007b00 con 0x7f5a281077f0 2023-10-08T09:26:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.158+0000 7f5a1effd700 1 -- 172.21.15.49:0/1029448021 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a24004cf0 con 0x7f5a281077f0 2023-10-08T09:26:50.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.158+0000 7f5a1effd700 1 -- 172.21.15.49:0/1029448021 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a24005650 con 0x7f5a281077f0 2023-10-08T09:26:50.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.159+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a28079610 con 0x7f5a281077f0 2023-10-08T09:26:50.160 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.159+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a28079ad0 con 0x7f5a281077f0 2023-10-08T09:26:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.163+0000 7f5a1effd700 1 -- 172.21.15.49:0/1029448021 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5a24007c60 con 0x7f5a281077f0 2023-10-08T09:26:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.164+0000 7f5a1effd700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5a140a1680 0x7f5a140a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.164+0000 7f5a1effd700 1 -- 172.21.15.49:0/1029448021 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a240ccee0 con 0x7f5a281077f0 2023-10-08T09:26:50.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.169+0000 7f5a3c1bab80 1 Processor -- start 2023-10-08T09:26:50.170 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.170+0000 7f5a3c1bab80 1 -- start start 2023-10-08T09:26:50.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.170+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8e9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.170+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.170+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.170+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b851460 con 0x55e94b8ea800 2023-10-08T09:26:50.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.170+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b842c20 con 0x55e94b8e9060 2023-10-08T09:26:50.171 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.170+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b840540 con 0x55e94b8e99c0 2023-10-08T09:26:50.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.171+0000 7f5a2e56e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.171+0000 7f5a2e56e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8eac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53434/0 (socket says 172.21.15.49:53434) 2023-10-08T09:26:50.172 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.171+0000 7f5a2e56e700 1 -- 172.21.15.49:0/1852416650 learned_addr learned my addr 172.21.15.49:0/1852416650 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:50.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.172+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.173 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.172+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.173+0000 7f5a2d56c700 1 -- 172.21.15.49:0/1852416650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 msgr2=0x55e94b8e9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.173+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8e9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.173+0000 7f5a2d56c700 1 -- 172.21.15.49:0/1852416650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 msgr2=0x55e94b8eac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.173+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8eac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.174 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.173+0000 7f5a2d56c700 1 -- 172.21.15.49:0/1852416650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94b666de0 con 0x55e94b8e99c0 2023-10-08T09:26:50.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.173+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8ea270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009c10 tx=0x7f5a20008510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32311ed124f89232 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:50.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.174+0000 7f5a13fff700 1 -- 172.21.15.49:0/1852416650 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20008d60 con 0x55e94b8e99c0 2023-10-08T09:26:50.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.174+0000 7f5a13fff700 1 -- 172.21.15.49:0/1852416650 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a200158e0 con 0x55e94b8e99c0 2023-10-08T09:26:50.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.174+0000 7f5a13fff700 1 -- 172.21.15.49:0/1852416650 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200147f0 con 0x55e94b8e99c0 2023-10-08T09:26:50.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.175+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1852416650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 msgr2=0x55e94b8ea270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.176 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.175+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8ea270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009c10 tx=0x7f5a20008510 comp rx=0 tx=0).stop 2023-10-08T09:26:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.175+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5a140a1680 0x7f5a140a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.177 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.176+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5a140a1680 0x7f5a140a3b40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55e94b8e9e00 tx=0x7f5a20008510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:50.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.178+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1852416650 shutdown_connections 2023-10-08T09:26:50.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.178+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.178+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.178+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1852416650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8ea270 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.178 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.178+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1852416650 >> 172.21.15.49:0/1852416650 conn(0x55e94b85d260 msgr2=0x55e94b8e3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:50.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.178+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1852416650 shutdown_connections 2023-10-08T09:26:50.179 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.179+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1852416650 wait complete. 2023-10-08T09:26:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.179+0000 7f5a3c1bab80 1 -- start start 2023-10-08T09:26:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.179+0000 7f5a3c1bab80 1 Processor -- start 2023-10-08T09:26:50.181 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8fa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b83be30 con 0x55e94b8ea800 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b7d8710 con 0x55e94b8e9060 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b7d83b0 con 0x55e94b8e99c0 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a3c1bab80 10 client.0 _ll_register_callbacks cb 0x55e94b635800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a2d56c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a2d56c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40476/0 (socket says 172.21.15.49:40476) 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.180+0000 7f5a2d56c700 1 -- 172.21.15.49:0/850141822 learned_addr learned my addr 172.21.15.49:0/850141822 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a2d56c700 1 -- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 msgr2=0x55e94b8f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.182 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a2d56c700 1 -- 172.21.15.49:0/850141822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 msgr2=0x55e94b8fa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a2d56c700 1 -- 172.21.15.49:0/850141822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a20011040 con 0x55e94b8e99c0 2023-10-08T09:26:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f33d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5a20004a30 tx=0x7f5a20009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868856e127e09594 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a117fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:26:50.183 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a117fa700 1 -- 172.21.15.49:0/850141822 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20008d60 con 0x55e94b8e99c0 2023-10-08T09:26:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.181+0000 7f5a117fa700 1 -- 172.21.15.49:0/850141822 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a20015630 con 0x55e94b8e99c0 2023-10-08T09:26:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.182+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e94b803eb0 con 0x55e94b8e99c0 2023-10-08T09:26:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.182+0000 7f5a117fa700 1 -- 172.21.15.49:0/850141822 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a2002c470 con 0x55e94b8e99c0 2023-10-08T09:26:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.183+0000 7f5a3c1bab80 10 client.26569 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:50.184 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.183+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e94b64fc60 con 0x55e94b8e99c0 2023-10-08T09:26:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a117fa700 1 -- 172.21.15.49:0/850141822 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e94b803eb0 con 0x55e94b8e99c0 2023-10-08T09:26:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a117fa700 1 -- 172.21.15.49:0/850141822 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a2001e4b0 con 0x55e94b8e99c0 2023-10-08T09:26:50.187 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a3c1bab80 10 client.26569 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a3c1bab80 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a3c1bab80 20 client.26569 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b8fec30 0x55e94b901050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a3c1bab80 4 client.26569 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:26:50.188 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.187+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e94b834e10 con 0x55e94b8fec30 2023-10-08T09:26:50.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.190+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b8fec30 0x55e94b901050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.190+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b8fec30 0x55e94b901050 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4990e19267507de8 server_cookie=1a2eb32af72f27f0 in_seq=0 out_seq=0 2023-10-08T09:26:50.192 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.191+0000 7f5a117fa700 10 client.26569 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:50.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.202+0000 7f5a117fa700 1 -- 172.21.15.49:0/850141822 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e94b834e10 con 0x55e94b8fec30 2023-10-08T09:26:50.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.202+0000 7f5a117fa700 10 client.26569 handle_command_reply: tid=0 2023-10-08T09:26:50.202 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.202+0000 7f5a3c1bab80 1 client.26569 shutdown 2023-10-08T09:26:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.218+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 msgr2=0x55e94b8f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.219 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.218+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f33d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5a20004a30 tx=0x7f5a20009eb0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.223+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 shutdown_connections 2023-10-08T09:26:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.223+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b8fec30 0x55e94b901050 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.223+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8ea800 0x55e94b8fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.223+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8e9060 0x55e94b8f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.223+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/850141822 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f33d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.223+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 >> 172.21.15.49:0/850141822 conn(0x55e94b85d260 msgr2=0x55e94b8e34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:50.224 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.224+0000 7f5a117fa700 0 client.26569 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:50.227 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.227+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 shutdown_connections 2023-10-08T09:26:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.230+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/850141822 wait complete. 2023-10-08T09:26:50.230 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.230+0000 7f5a3c1bab80 20 client.26569 trim_cache size 0 max 16384 2023-10-08T09:26:50.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.268+0000 7f5a3c1bab80 1 Processor -- start 2023-10-08T09:26:50.270 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.269+0000 7f5a3c1bab80 1 -- start start 2023-10-08T09:26:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.269+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e9060 0x55e94b8f2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.269+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e99c0 0x55e94b8f2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.269+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.269+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b6bc920 con 0x55e94b8e99c0 2023-10-08T09:26:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.269+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b8f1930 con 0x55e94b8ea800 2023-10-08T09:26:50.271 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.269+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b8de570 con 0x55e94b8e9060 2023-10-08T09:26:50.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2e56e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2e56e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8fa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43066/0 (socket says 172.21.15.49:43066) 2023-10-08T09:26:50.272 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2e56e700 1 -- 172.21.15.49:0/1262324687 learned_addr learned my addr 172.21.15.49:0/1262324687 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e99c0 0x55e94b8f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2d56c700 1 -- 172.21.15.49:0/1262324687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e9060 msgr2=0x55e94b8f2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e9060 0x55e94b8f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2d56c700 1 -- 172.21.15.49:0/1262324687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 msgr2=0x55e94b8fa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8fa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.270+0000 7f5a2d56c700 1 -- 172.21.15.49:0/1262324687 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94b68d250 con 0x55e94b8e99c0 2023-10-08T09:26:50.273 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.271+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e99c0 0x55e94b8f2590 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5a2001e840 tx=0x7f5a20009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99527ed808c39b40 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:50.274 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.271+0000 7f5a1cff9700 1 -- 172.21.15.49:0/1262324687 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20043070 con 0x55e94b8e99c0 2023-10-08T09:26:50.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.271+0000 7f5a1cff9700 1 -- 172.21.15.49:0/1262324687 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a20014380 con 0x55e94b8e99c0 2023-10-08T09:26:50.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.271+0000 7f5a1cff9700 1 -- 172.21.15.49:0/1262324687 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001ede0 con 0x55e94b8e99c0 2023-10-08T09:26:50.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.272+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1262324687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e99c0 msgr2=0x55e94b8f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.275 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.272+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e99c0 0x55e94b8f2590 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f5a2001e840 tx=0x7f5a20009530 comp rx=0 tx=0).stop 2023-10-08T09:26:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.277+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1262324687 shutdown_connections 2023-10-08T09:26:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.277+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e9060 0x55e94b8f2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.277+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.277+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/1262324687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e99c0 0x55e94b8f2590 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.285 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.277+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1262324687 >> 172.21.15.49:0/1262324687 conn(0x55e94b85d260 msgr2=0x55e94b8f0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:50.290 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.289+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1262324687 shutdown_connections 2023-10-08T09:26:50.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.292+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/1262324687 wait complete. 2023-10-08T09:26:50.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.293+0000 7f5a3c1bab80 1 -- start start 2023-10-08T09:26:50.294 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.293+0000 7f5a3c1bab80 1 Processor -- start 2023-10-08T09:26:50.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.293+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e9060 0x55e94b8f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.293+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a3c1bab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8f9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b8f9a90 con 0x55e94b8e9060 2023-10-08T09:26:50.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b8f9c00 con 0x55e94b8ea800 2023-10-08T09:26:50.295 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a3c1bab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94b8f9dc0 con 0x55e94b8e99c0 2023-10-08T09:26:50.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a3c1bab80 10 client.0 _ll_register_callbacks cb 0x7f5a20018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:50.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a2dd6d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e9060 0x55e94b8f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a2dd6d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e9060 0x55e94b8f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53454/0 (socket says 172.21.15.49:53454) 2023-10-08T09:26:50.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.294+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/2617058098 learned_addr learned my addr 172.21.15.49:0/2617058098 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:50.296 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/2617058098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 msgr2=0x55e94b8f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/2617058098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 msgr2=0x55e94b8f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:50.297 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a2dd6d700 1 -- 172.21.15.49:0/2617058098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e94b64fc60 con 0x55e94b8e9060 2023-10-08T09:26:50.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a2dd6d700 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e9060 0x55e94b8f6040 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5a240968e0 tx=0x7f5a2409e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747584554cb1c0eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:50.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.295+0000 7f5a117fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:26:50.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.296+0000 7f5a117fa700 1 -- 172.21.15.49:0/2617058098 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a24098070 con 0x55e94b8e9060 2023-10-08T09:26:50.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.296+0000 7f5a117fa700 1 -- 172.21.15.49:0/2617058098 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5a24096580 con 0x55e94b8e9060 2023-10-08T09:26:50.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.296+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a200158e0 con 0x55e94b8e9060 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.296+0000 7f5a117fa700 1 -- 172.21.15.49:0/2617058098 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a24096740 con 0x55e94b8e9060 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.296+0000 7f5a3c1bab80 10 client.16932 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.296+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55e94b6652b0 con 0x55e94b8e9060 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.296+0000 7f5a117fa700 1 -- 172.21.15.49:0/2617058098 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a200158e0 con 0x55e94b8e9060 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.297+0000 7f5a117fa700 1 -- 172.21.15.49:0/2617058098 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5a24099480 con 0x55e94b8e9060 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.302+0000 7f5a3c1bab80 10 client.16932 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.302+0000 7f5a3c1bab80 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:50.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.302+0000 7f5a3c1bab80 20 client.16932 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:50.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.302+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b953710 0x55e94b955b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:50.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.302+0000 7f5a3c1bab80 4 client.16932 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:26:50.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.302+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e94b834b30 con 0x55e94b953710 2023-10-08T09:26:50.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.303+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b953710 0x55e94b955b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.304+0000 7f5a117fa700 10 client.16932 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.304+0000 7f5a2d56c700 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b953710 0x55e94b955b30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96eb642c945020b6 server_cookie=55118f2d93590982 in_seq=0 out_seq=0 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.304+0000 7f5a117fa700 1 -- 172.21.15.49:0/2617058098 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e94b834b30 con 0x55e94b953710 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.304+0000 7f5a117fa700 10 client.16932 handle_command_reply: tid=0 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.304+0000 7f5a3c1bab80 1 client.16932 shutdown 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.304+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e9060 msgr2=0x55e94b8f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.304+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e9060 0x55e94b8f6040 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5a240968e0 tx=0x7f5a2409e140 comp rx=0 tx=0).stop 2023-10-08T09:26:50.308 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 shutdown_connections 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e94b953710 0x55e94b955b30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e94b8ea800 0x55e94b8f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e94b8e9060 0x55e94b8f6040 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 --2- 172.21.15.49:0/2617058098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e94b8e99c0 0x55e94b8f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 >> 172.21.15.49:0/2617058098 conn(0x55e94b85d260 msgr2=0x55e94b8ed780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a117fa700 0 client.16932 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 shutdown_connections 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 1 -- 172.21.15.49:0/2617058098 wait complete. 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.306+0000 7f5a3c1bab80 20 client.16932 trim_cache size 0 max 16384 2023-10-08T09:26:50.309 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:26:50.310 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:26:50.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5a140a1680 msgr2=0x7f5a140a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5a140a1680 0x7f5a140a3b40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55e94b8e9e00 tx=0x7f5a20008510 comp rx=0 tx=0).stop 2023-10-08T09:26:50.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a281077f0 msgr2=0x7f5a2819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:50.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a281077f0 0x7f5a2819ccc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400c920 tx=0x7f5a2400cce0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 shutdown_connections 2023-10-08T09:26:50.312 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5a140a1680 0x7f5a140a3b40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a2810cb30 0x7f5a28078a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a281077f0 0x7f5a2819ccc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 --2- 172.21.15.49:0/1029448021 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a2810a160 0x7f5a2819d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 >> 172.21.15.49:0/1029448021 conn(0x7f5a2806d790 msgr2=0x7f5a28107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 shutdown_connections 2023-10-08T09:26:50.313 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:26:50.307+0000 7f5a2ffd1700 1 -- 172.21.15.49:0/1029448021 wait complete. 2023-10-08T09:26:50.323 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:26:51.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:51 smithi184 ceph-mon[103675]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 482 op/s 2023-10-08T09:26:52.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:51 smithi132 ceph-mon[103389]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 482 op/s 2023-10-08T09:26:52.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:51 smithi049 ceph-mon[99485]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 482 op/s 2023-10-08T09:26:53.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:53 smithi049 ceph-mon[99485]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.3 MiB/s wr, 540 op/s 2023-10-08T09:26:53.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:53 smithi184 ceph-mon[103675]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.3 MiB/s wr, 540 op/s 2023-10-08T09:26:53.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:53 smithi132 ceph-mon[103389]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.3 MiB/s wr, 540 op/s 2023-10-08T09:26:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 690.0 s, 544.6 tps, lat 181.850 ms stddev 201.906 2023-10-08T09:26:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.349+0000 7fd571adf700 1 -- 172.21.15.49:0/3310383546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c077ee0 msgr2=0x7fd56c07a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.349+0000 7fd571adf700 1 --2- 172.21.15.49:0/3310383546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c077ee0 0x7fd56c07a370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd560009990 tx=0x7fd560009ca0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.359+0000 7fd571adf700 1 -- 172.21.15.49:0/3310383546 shutdown_connections 2023-10-08T09:26:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.359+0000 7fd571adf700 1 --2- 172.21.15.49:0/3310383546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c077ee0 0x7fd56c07a370 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.359+0000 7fd571adf700 1 --2- 172.21.15.49:0/3310383546 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd56c0725e0 0x7fd56c072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.359+0000 7fd571adf700 1 --2- 172.21.15.49:0/3310383546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56c075510 0x7fd56c0779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.359+0000 7fd571adf700 1 -- 172.21.15.49:0/3310383546 >> 172.21.15.49:0/3310383546 conn(0x7fd56c06d770 msgr2=0x7fd56c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.364+0000 7fd571adf700 1 -- 172.21.15.49:0/3310383546 shutdown_connections 2023-10-08T09:26:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.367+0000 7fd571adf700 1 -- 172.21.15.49:0/3310383546 wait complete. 2023-10-08T09:26:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.367+0000 7fd571adf700 1 Processor -- start 2023-10-08T09:26:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd571adf700 1 -- start start 2023-10-08T09:26:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd571adf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c0725e0 0x7fd56c1aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd571adf700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56c075510 0x7fd56c1b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd571adf700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd56c077ee0 0x7fd56c1b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd571adf700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c1b7a80 con 0x7fd56c077ee0 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd571adf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c1b7bc0 con 0x7fd56c0725e0 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd571adf700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c1b7d50 con 0x7fd56c075510 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c0725e0 0x7fd56c1aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c0725e0 0x7fd56c1aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43076/0 (socket says 172.21.15.49:43076) 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd56b7fe700 1 -- 172.21.15.49:0/710462034 learned_addr learned my addr 172.21.15.49:0/710462034 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.368+0000 7fd56bfff700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd56c077ee0 0x7fd56c1b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd56b7fe700 1 -- 172.21.15.49:0/710462034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56c075510 msgr2=0x7fd56c1b0490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56c075510 0x7fd56c1b0490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd56b7fe700 1 -- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd56c077ee0 msgr2=0x7fd56c1b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd56c077ee0 0x7fd56c1b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd56b7fe700 1 -- 172.21.15.49:0/710462034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600095f0 con 0x7fd56c0725e0 2023-10-08T09:26:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c0725e0 0x7fd56c1aff50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd56400ba50 tx=0x7fd56400bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd568ff9700 1 -- 172.21.15.49:0/710462034 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56400c9b0 con 0x7fd56c0725e0 2023-10-08T09:26:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.369+0000 7fd568ff9700 1 -- 172.21.15.49:0/710462034 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd564014440 con 0x7fd56c0725e0 2023-10-08T09:26:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.370+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c1b7fd0 con 0x7fd56c0725e0 2023-10-08T09:26:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.370+0000 7fd568ff9700 1 -- 172.21.15.49:0/710462034 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564007590 con 0x7fd56c0725e0 2023-10-08T09:26:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.370+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c1b84e0 con 0x7fd56c0725e0 2023-10-08T09:26:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.372+0000 7fd568ff9700 1 -- 172.21.15.49:0/710462034 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd5640145b0 con 0x7fd56c0725e0 2023-10-08T09:26:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.373+0000 7fd568ff9700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5540a1480 0x7fd5540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.373+0000 7fd56affd700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5540a1480 0x7fd5540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.373+0000 7fd568ff9700 1 -- 172.21.15.49:0/710462034 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd5640cc7a0 con 0x7fd56c0725e0 2023-10-08T09:26:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.374+0000 7fd56affd700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5540a1480 0x7fd5540a3940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009710 tx=0x7fd55c006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.374+0000 7fd57dcc8b80 1 Processor -- start 2023-10-08T09:26:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.374+0000 7fd57dcc8b80 1 -- start start 2023-10-08T09:26:55.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.375+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c939053e40 0x55c939054260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.375+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390547a0 0x55c939054cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.375+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390552b0 0x55c939055730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.375+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c938fbc400 con 0x55c939053e40 2023-10-08T09:26:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.375+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c938fadbc0 con 0x55c9390552b0 2023-10-08T09:26:55.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.375+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c938fab4e0 con 0x55c9390547a0 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c939053e40 0x55c939054260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c939053e40 0x55c939054260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53470/0 (socket says 172.21.15.49:53470) 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 -- 172.21.15.49:0/3482145734 learned_addr learned my addr 172.21.15.49:0/3482145734 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56bfff700 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390552b0 0x55c939055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 -- 172.21.15.49:0/3482145734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390547a0 msgr2=0x55c939054cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390547a0 0x55c939054cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 -- 172.21.15.49:0/3482145734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390552b0 msgr2=0x55c939055730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:55.379 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390552b0 0x55c939055730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 -- 172.21.15.49:0/3482145734 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c938dd1d80 con 0x55c939053e40 2023-10-08T09:26:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56bfff700 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390552b0 0x55c939055730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:26:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c939053e40 0x55c939054260 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd564095c30 tx=0x7fd5640992f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8accaeaadcc6816 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd551ffb700 1 -- 172.21.15.49:0/3482145734 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564099730 con 0x55c939053e40 2023-10-08T09:26:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd551ffb700 1 -- 172.21.15.49:0/3482145734 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd56409d880 con 0x55c939053e40 2023-10-08T09:26:55.380 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.376+0000 7fd551ffb700 1 -- 172.21.15.49:0/3482145734 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640840b0 con 0x55c939053e40 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.377+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3482145734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c939053e40 msgr2=0x55c939054260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.377+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c939053e40 0x55c939054260 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd564095c30 tx=0x7fd5640992f0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.378+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3482145734 shutdown_connections 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.378+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390552b0 0x55c939055730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.378+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c939053e40 0x55c939054260 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.378+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3482145734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390547a0 0x55c939054cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.378+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3482145734 >> 172.21.15.49:0/3482145734 conn(0x55c938fc8200 msgr2=0x55c93904e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.378+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3482145734 shutdown_connections 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.378+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3482145734 wait complete. 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 -- start start 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 Processor -- start 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c939053e40 0x55c9390660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390547a0 0x55c939066600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390552b0 0x55c93906d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c938fa6dd0 con 0x55c9390552b0 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c938f436b0 con 0x55c9390547a0 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c938f43350 con 0x55c939053e40 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd57dcc8b80 10 client.0 _ll_register_callbacks cb 0x55c938da07a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390552b0 0x55c93906d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd56affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390547a0 0x55c939066600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.379+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390552b0 0x55c93906d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53478/0 (socket says 172.21.15.49:53478) 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56bfff700 1 -- 172.21.15.49:0/1660320173 learned_addr learned my addr 172.21.15.49:0/1660320173 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c939053e40 0x55c9390660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56affd700 1 -- 172.21.15.49:0/1660320173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c939053e40 msgr2=0x55c9390660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56affd700 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c939053e40 0x55c9390660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56affd700 1 -- 172.21.15.49:0/1660320173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390552b0 msgr2=0x55c93906d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56affd700 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390552b0 0x55c93906d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56affd700 1 -- 172.21.15.49:0/1660320173 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c1b84e0 con 0x55c9390547a0 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56bfff700 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390552b0 0x55c93906d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:26:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd56affd700 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390547a0 0x55c939066600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009e60 tx=0x7fd55c004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489de87acb9d33fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd54affd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd54affd700 1 -- 172.21.15.49:0/1660320173 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c01a070 con 0x55c9390547a0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd54affd700 1 -- 172.21.15.49:0/1660320173 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd55c012a80 con 0x55c9390547a0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c938f6ee50 con 0x55c9390547a0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd54affd700 1 -- 172.21.15.49:0/1660320173 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55c0152e0 con 0x55c9390547a0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd57dcc8b80 10 client.26522 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.380+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c938dbac00 con 0x55c9390547a0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd54affd700 1 -- 172.21.15.49:0/1660320173 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c938f6ee50 con 0x55c9390547a0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd54affd700 1 -- 172.21.15.49:0/1660320173 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd55c010360 con 0x55c9390547a0 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd57dcc8b80 10 client.26522 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd57dcc8b80 10 client.26522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd57dcc8b80 20 client.26522 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c939072070 0x55c939074490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd57dcc8b80 4 client.26522 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c938f9fdb0 con 0x55c939072070 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.381+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c939072070 0x55c939074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.382+0000 7fd54affd700 10 client.26522 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.382+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c939072070 0x55c939074490 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353839c627d1bf1f server_cookie=135206a18f1ae4a7 in_seq=0 out_seq=0 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.382+0000 7fd54affd700 1 -- 172.21.15.49:0/1660320173 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c938f9fdb0 con 0x55c939072070 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.382+0000 7fd54affd700 10 client.26522 handle_command_reply: tid=0 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.382+0000 7fd57dcc8b80 1 client.26522 shutdown 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.382+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390547a0 msgr2=0x55c939066600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.382+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390547a0 0x55c939066600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009e60 tx=0x7fd55c004800 comp rx=0 tx=0).stop 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 shutdown_connections 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c939072070 0x55c939074490 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c939053e40 0x55c9390660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390552b0 0x55c93906d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd54affd700 0 client.26522 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1660320173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c9390547a0 0x55c939066600 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 >> 172.21.15.49:0/1660320173 conn(0x55c938fc8200 msgr2=0x55c93904e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 shutdown_connections 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1660320173 wait complete. 2023-10-08T09:26:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.383+0000 7fd57dcc8b80 20 client.26522 trim_cache size 0 max 16384 2023-10-08T09:26:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 Processor -- start 2023-10-08T09:26:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 -- start start 2023-10-08T09:26:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c93906dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c93906d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93905b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c938e278c0 con 0x55c9390547a0 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c939065c60 con 0x55c939053e40 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93904f4c0 con 0x55c9390552b0 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c93906dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd56affd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c93906d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c93906dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43096/0 (socket says 172.21.15.49:43096) 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd56b7fe700 1 -- 172.21.15.49:0/1573990363 learned_addr learned my addr 172.21.15.49:0/1573990363 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.406+0000 7fd56bfff700 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93905b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd56bfff700 1 -- 172.21.15.49:0/1573990363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 msgr2=0x55c93906dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd56bfff700 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c93906dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd56bfff700 1 -- 172.21.15.49:0/1573990363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 msgr2=0x55c93906d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd56bfff700 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c93906d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd56bfff700 1 -- 172.21.15.49:0/1573990363 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c938df81f0 con 0x55c9390552b0 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd56bfff700 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93905b780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd560003790 tx=0x7fd560003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=105bb3cf27853944 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd5527fc700 1 -- 172.21.15.49:0/1573990363 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56001d070 con 0x55c9390552b0 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd5527fc700 1 -- 172.21.15.49:0/1573990363 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd56001e040 con 0x55c9390552b0 2023-10-08T09:26:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.407+0000 7fd5527fc700 1 -- 172.21.15.49:0/1573990363 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560024480 con 0x55c9390552b0 2023-10-08T09:26:55.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1573990363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 msgr2=0x55c93905b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93905b780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd560003790 tx=0x7fd560003680 comp rx=0 tx=0).stop 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1573990363 shutdown_connections 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93905b780 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c93906dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/1573990363 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c93906d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1573990363 >> 172.21.15.49:0/1573990363 conn(0x55c938fc8200 msgr2=0x55c939078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1573990363 shutdown_connections 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.408+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/1573990363 wait complete. 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.409+0000 7fd57dcc8b80 1 -- start start 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.409+0000 7fd57dcc8b80 1 Processor -- start 2023-10-08T09:26:55.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.409+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c9390690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c939069610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd57dcc8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93906c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93906ca80 con 0x55c9390547a0 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93906cbf0 con 0x55c939053e40 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd57dcc8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93906cdb0 con 0x55c9390552b0 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd57dcc8b80 10 client.0 _ll_register_callbacks cb 0x7fd55c0159b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c9390690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93906c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56b7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c9390690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43102/0 (socket says 172.21.15.49:43102) 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93906c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40524/0 (socket says 172.21.15.49:40524) 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56b7fe700 1 -- 172.21.15.49:0/3887888331 learned_addr learned my addr 172.21.15.49:0/3887888331 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56affd700 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c939069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56affd700 1 -- 172.21.15.49:0/3887888331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 msgr2=0x55c93906c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56affd700 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93906c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56affd700 1 -- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 msgr2=0x55c9390690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56affd700 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c9390690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56affd700 1 -- 172.21.15.49:0/3887888331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c07ada0 con 0x55c9390547a0 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.410+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c9390690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd56affd700 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c939069610 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd55c000c00 tx=0x7fd55c00b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad83a9d4881a258 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd54affd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd54affd700 1 -- 172.21.15.49:0/3887888331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c01a070 con 0x55c9390547a0 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd54affd700 1 -- 172.21.15.49:0/3887888331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd55c012970 con 0x55c9390547a0 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd56409d880 con 0x55c9390547a0 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd54affd700 1 -- 172.21.15.49:0/3887888331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55c010ab0 con 0x55c9390547a0 2023-10-08T09:26:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd57dcc8b80 10 client.16947 fetch_fsmap learned FSMap version 40 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55c938dd0250 con 0x55c9390547a0 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd54affd700 1 -- 172.21.15.49:0/3887888331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd56409d880 con 0x55c9390547a0 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.411+0000 7fd54affd700 1 -- 172.21.15.49:0/3887888331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd55c0123b0 con 0x55c9390547a0 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd57dcc8b80 10 client.16947 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd57dcc8b80 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd57dcc8b80 20 client.16947 populate_metadata read hostname 'smithi049' 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c9390c6ab0 0x55c9390c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd57dcc8b80 4 client.16947 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:26:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c938f9fad0 con 0x55c9390c6ab0 2023-10-08T09:26:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c9390c6ab0 0x55c9390c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:26:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd54affd700 10 client.16947 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.412+0000 7fd56b7fe700 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c9390c6ab0 0x55c9390c8f30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e33615f85af6b54 server_cookie=c9b240dd04bb3643 in_seq=0 out_seq=0 2023-10-08T09:26:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd54affd700 1 -- 172.21.15.49:0/3887888331 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c938f9fad0 con 0x55c9390c6ab0 2023-10-08T09:26:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd54affd700 10 client.16947 handle_command_reply: tid=0 2023-10-08T09:26:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 client.16947 shutdown 2023-10-08T09:26:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 msgr2=0x55c939069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c939069610 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd55c000c00 tx=0x7fd55c00b150 comp rx=0 tx=0).stop 2023-10-08T09:26:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 shutdown_connections 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c9390c6ab0 0x55c9390c8f30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c9390552b0 0x55c93906c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c939053e40 0x55c9390690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd54affd700 0 client.16947 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 --2- 172.21.15.49:0/3887888331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c9390547a0 0x55c939069610 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 >> 172.21.15.49:0/3887888331 conn(0x55c938fc8200 msgr2=0x55c93905b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 shutdown_connections 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 1 -- 172.21.15.49:0/3887888331 wait complete. 2023-10-08T09:26:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.422+0000 7fd57dcc8b80 20 client.16947 trim_cache size 0 max 16384 2023-10-08T09:26:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:26:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:26:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:26:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:26:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.423+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5540a1480 msgr2=0x7fd5540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.423+0000 7fd571adf700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5540a1480 0x7fd5540a3940 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009710 tx=0x7fd55c006c60 comp rx=0 tx=0).stop 2023-10-08T09:26:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.423+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c0725e0 msgr2=0x7fd56c1aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:26:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.423+0000 7fd571adf700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c0725e0 0x7fd56c1aff50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd56400ba50 tx=0x7fd56400bd60 comp rx=0 tx=0).stop 2023-10-08T09:26:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.427+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 shutdown_connections 2023-10-08T09:26:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.427+0000 7fd571adf700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd5540a1480 0x7fd5540a3940 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.427+0000 7fd571adf700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd56c077ee0 0x7fd56c1b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.427+0000 7fd571adf700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd56c0725e0 0x7fd56c1aff50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.427+0000 7fd571adf700 1 --2- 172.21.15.49:0/710462034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd56c075510 0x7fd56c1b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:26:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.427+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 >> 172.21.15.49:0/710462034 conn(0x7fd56c06d770 msgr2=0x7fd56c076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:26:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.429+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 shutdown_connections 2023-10-08T09:26:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:26:55.429+0000 7fd571adf700 1 -- 172.21.15.49:0/710462034 wait complete. 2023-10-08T09:26:55.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:55 smithi184 ceph-mon[103675]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 513 op/s 2023-10-08T09:26:56.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:55 smithi132 ceph-mon[103389]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 513 op/s 2023-10-08T09:26:56.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:55 smithi049 ceph-mon[99485]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.5 MiB/s wr, 513 op/s 2023-10-08T09:26:57.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:57 smithi184 ceph-mon[103675]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 554 op/s 2023-10-08T09:26:58.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:57 smithi132 ceph-mon[103389]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 554 op/s 2023-10-08T09:26:58.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:57 smithi049 ceph-mon[99485]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 554 op/s 2023-10-08T09:26:59.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:26:58 smithi049 ceph-mon[99485]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 539 op/s 2023-10-08T09:26:59.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:26:58 smithi184 ceph-mon[103675]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 539 op/s 2023-10-08T09:26:59.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:26:58 smithi132 ceph-mon[103389]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 539 op/s 2023-10-08T09:26:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 695.0 s, 530.8 tps, lat 193.030 ms stddev 216.670 2023-10-08T09:27:01.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:01 smithi184 ceph-mon[103675]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 497 op/s 2023-10-08T09:27:02.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:01 smithi132 ceph-mon[103389]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 497 op/s 2023-10-08T09:27:02.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:01 smithi049 ceph-mon[99485]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 497 op/s 2023-10-08T09:27:03.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:03 smithi049 ceph-mon[99485]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 543 op/s 2023-10-08T09:27:03.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:03 smithi184 ceph-mon[103675]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 543 op/s 2023-10-08T09:27:03.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:03 smithi132 ceph-mon[103389]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.6 MiB/s wr, 543 op/s 2023-10-08T09:27:04.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 700.0 s, 507.0 tps, lat 197.225 ms stddev 220.691 2023-10-08T09:27:05.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:05 smithi184 ceph-mon[103675]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2023-10-08T09:27:06.048 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:05 smithi132 ceph-mon[103389]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2023-10-08T09:27:06.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:05 smithi049 ceph-mon[99485]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2023-10-08T09:27:07.998 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:07 smithi184 ceph-mon[103675]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 519 op/s 2023-10-08T09:27:08.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:07 smithi132 ceph-mon[103389]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 519 op/s 2023-10-08T09:27:08.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:07 smithi049 ceph-mon[99485]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 519 op/s 2023-10-08T09:27:08.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.665+0000 7fbd7f34c700 1 -- 172.21.15.49:0/3534781999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810a160 msgr2=0x7fbd7810c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.665+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/3534781999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810a160 0x7fbd7810c5f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbd6801a9a0 tx=0x7fbd6801acb0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.675+0000 7fbd7f34c700 1 -- 172.21.15.49:0/3534781999 shutdown_connections 2023-10-08T09:27:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.675+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/3534781999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd781077f0 0x7fbd78109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.675+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/3534781999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd7810cb30 0x7fbd7810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.675+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/3534781999 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810a160 0x7fbd7810c5f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.675+0000 7fbd7f34c700 1 -- 172.21.15.49:0/3534781999 >> 172.21.15.49:0/3534781999 conn(0x7fbd7806d770 msgr2=0x7fbd7806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.681+0000 7fbd7f34c700 1 -- 172.21.15.49:0/3534781999 shutdown_connections 2023-10-08T09:27:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.692+0000 7fbd7f34c700 1 -- 172.21.15.49:0/3534781999 wait complete. 2023-10-08T09:27:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.692+0000 7fbd7f34c700 1 Processor -- start 2023-10-08T09:27:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.692+0000 7fbd7f34c700 1 -- start start 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7f34c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd781077f0 0x7fbd7819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7f34c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7810a160 0x7fbd7819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7f34c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810cb30 0x7fbd781a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7f34c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781a48f0 con 0x7fbd781077f0 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7f34c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781a4a60 con 0x7fbd7810cb30 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7f34c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd781a4c20 con 0x7fbd7810a160 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd781077f0 0x7fbd7819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd781077f0 0x7fbd7819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53500/0 (socket says 172.21.15.49:53500) 2023-10-08T09:27:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/2097624778 learned_addr learned my addr 172.21.15.49:0/2097624778 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.693+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810cb30 0x7fbd781a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.694+0000 7fbd7d8e9700 1 -- 172.21.15.49:0/2097624778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7810a160 msgr2=0x7fbd7819d340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:27:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.694+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7810a160 0x7fbd7819d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.694+0000 7fbd7d8e9700 1 -- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd781077f0 msgr2=0x7fbd7819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.694+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd781077f0 0x7fbd7819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.694+0000 7fbd7d8e9700 1 -- 172.21.15.49:0/2097624778 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6801a600 con 0x7fbd7810cb30 2023-10-08T09:27:08.697 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.695+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810cb30 0x7fbd781a4320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000be00 tx=0x7fbd70009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.699+0000 7fbd6e7fc700 1 -- 172.21.15.49:0/2097624778 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000cc10 con 0x7fbd7810cb30 2023-10-08T09:27:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.699+0000 7fbd6e7fc700 1 -- 172.21.15.49:0/2097624778 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd7000cd70 con 0x7fbd7810cb30 2023-10-08T09:27:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.699+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd781a4f00 con 0x7fbd7810cb30 2023-10-08T09:27:08.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.699+0000 7fbd6e7fc700 1 -- 172.21.15.49:0/2097624778 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70012740 con 0x7fbd7810cb30 2023-10-08T09:27:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.699+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd781a52d0 con 0x7fbd7810cb30 2023-10-08T09:27:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.703+0000 7fbd8b535b80 1 Processor -- start 2023-10-08T09:27:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.713+0000 7fbd8b535b80 1 -- start start 2023-10-08T09:27:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.713+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c0e40 0x557e532c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.713+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.713+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 0x557e532c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.713+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53229400 con 0x557e532c22b0 2023-10-08T09:27:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.713+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e5321abc0 con 0x557e532c0e40 2023-10-08T09:27:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.713+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e532184e0 con 0x557e532c17a0 2023-10-08T09:27:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532c1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40538/0 (socket says 172.21.15.49:40538) 2023-10-08T09:27:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/267439163 learned_addr learned my addr 172.21.15.49:0/267439163 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/267439163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c0e40 msgr2=0x557e532c1260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/267439163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c0e40 0x557e532c1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/267439163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 msgr2=0x557e532c2730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/267439163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 0x557e532c2730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/267439163 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e5303ed80 con 0x557e532c17a0 2023-10-08T09:27:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.714+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/267439163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532c1cd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009b80 tx=0x7fbd74009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15a383fb6dce310c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.723+0000 7fbd637fe700 1 -- 172.21.15.49:0/267439163 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7400e9f0 con 0x557e532c17a0 2023-10-08T09:27:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.723+0000 7fbd637fe700 1 -- 172.21.15.49:0/267439163 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd74013470 con 0x557e532c17a0 2023-10-08T09:27:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.723+0000 7fbd637fe700 1 -- 172.21.15.49:0/267439163 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd74012630 con 0x557e532c17a0 2023-10-08T09:27:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.724+0000 7fbd8b535b80 1 -- 172.21.15.49:0/267439163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 msgr2=0x557e532c1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.724+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/267439163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532c1cd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009b80 tx=0x7fbd74009e90 comp rx=0 tx=0).stop 2023-10-08T09:27:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.728+0000 7fbd8b535b80 1 -- 172.21.15.49:0/267439163 shutdown_connections 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.728+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/267439163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 0x557e532c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.728+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/267439163 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c0e40 0x557e532c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.728+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/267439163 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532c1cd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.728+0000 7fbd8b535b80 1 -- 172.21.15.49:0/267439163 >> 172.21.15.49:0/267439163 conn(0x557e53235200 msgr2=0x557e532bb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.729+0000 7fbd6e7fc700 1 -- 172.21.15.49:0/2097624778 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbd700128a0 con 0x7fbd7810cb30 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.729+0000 7fbd6e7fc700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd640a1680 0x7fbd640a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.730+0000 7fbd6e7fc700 1 -- 172.21.15.49:0/2097624778 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbd700cc810 con 0x7fbd7810cb30 2023-10-08T09:27:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.730+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd640a1680 0x7fbd640a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.734+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd640a1680 0x7fbd640a3b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbd74013cd0 tx=0x7fbd74008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.741+0000 7fbd8b535b80 1 -- 172.21.15.49:0/267439163 shutdown_connections 2023-10-08T09:27:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.744+0000 7fbd8b535b80 1 -- 172.21.15.49:0/267439163 wait complete. 2023-10-08T09:27:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.744+0000 7fbd8b535b80 1 -- start start 2023-10-08T09:27:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.744+0000 7fbd8b535b80 1 Processor -- start 2023-10-08T09:27:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c0e40 0x557e532ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c17a0 0x557e532cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 0x557e532d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e53213dd0 con 0x557e532c22b0 2023-10-08T09:27:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e531b06b0 con 0x557e532c17a0 2023-10-08T09:27:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e531b0350 con 0x557e532c0e40 2023-10-08T09:27:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd8b535b80 10 client.0 _ll_register_callbacks cb 0x557e5300d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd7c8e7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c17a0 0x557e532cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd7c8e7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c17a0 0x557e532cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43122/0 (socket says 172.21.15.49:43122) 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd7c8e7700 1 -- 172.21.15.49:0/2417502040 learned_addr learned my addr 172.21.15.49:0/2417502040 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.745+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c0e40 0x557e532ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.746+0000 7fbd7c8e7700 1 -- 172.21.15.49:0/2417502040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c0e40 msgr2=0x557e532ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.746+0000 7fbd7c8e7700 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c0e40 0x557e532ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.746+0000 7fbd7c8e7700 1 -- 172.21.15.49:0/2417502040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 msgr2=0x557e532d1dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.746+0000 7fbd7c8e7700 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 0x557e532d1dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.746+0000 7fbd7c8e7700 1 -- 172.21.15.49:0/2417502040 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd740097b0 con 0x557e532c17a0 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.747+0000 7fbd7c8e7700 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c17a0 0x557e532cad90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd6801af50 tx=0x7fbd68005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77cdb4d2e537a6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.754+0000 7fbd60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:27:08.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.754+0000 7fbd60ff9700 1 -- 172.21.15.49:0/2417502040 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6801cd40 con 0x557e532c17a0 2023-10-08T09:27:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.754+0000 7fbd60ff9700 1 -- 172.21.15.49:0/2417502040 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbd680039f0 con 0x557e532c17a0 2023-10-08T09:27:08.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.754+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e531dbe50 con 0x557e532c17a0 2023-10-08T09:27:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.765+0000 7fbd60ff9700 1 -- 172.21.15.49:0/2417502040 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd6801cea0 con 0x557e532c17a0 2023-10-08T09:27:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.765+0000 7fbd8b535b80 10 client.26537 fetch_fsmap learned FSMap version 40 2023-10-08T09:27:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.765+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557e53027c00 con 0x557e532c17a0 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd60ff9700 1 -- 172.21.15.49:0/2417502040 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e531dbe50 con 0x557e532c17a0 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd60ff9700 1 -- 172.21.15.49:0/2417502040 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd680308a0 con 0x557e532c17a0 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd8b535b80 10 client.26537 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd8b535b80 10 client.26537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd8b535b80 20 client.26537 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e532d6770 0x557e532d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd8b535b80 4 client.26537 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:27:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.767+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e5320cdb0 con 0x557e532d6770 2023-10-08T09:27:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.770+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e532d6770 0x557e532d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.770+0000 7fbd60ff9700 10 client.26537 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.771+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e532d6770 0x557e532d8b90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c8a2611918f525 server_cookie=b79d8a9db17bb535 in_seq=0 out_seq=0 2023-10-08T09:27:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.777+0000 7fbd60ff9700 1 -- 172.21.15.49:0/2417502040 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557e5320cdb0 con 0x557e532d6770 2023-10-08T09:27:08.778 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.777+0000 7fbd60ff9700 10 client.26537 handle_command_reply: tid=0 2023-10-08T09:27:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.786+0000 7fbd8b535b80 1 client.26537 shutdown 2023-10-08T09:27:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.791+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c17a0 msgr2=0x557e532cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.791+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c17a0 0x557e532cad90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd6801af50 tx=0x7fbd68005c50 comp rx=0 tx=0).stop 2023-10-08T09:27:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.803+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 shutdown_connections 2023-10-08T09:27:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.803+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e532d6770 0x557e532d8b90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.803+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c0e40 0x557e532ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.803+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c22b0 0x557e532d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.803+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/2417502040 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c17a0 0x557e532cad90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.803+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 >> 172.21.15.49:0/2417502040 conn(0x557e53235200 msgr2=0x557e532bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.808+0000 7fbd60ff9700 0 client.26537 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.814+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 shutdown_connections 2023-10-08T09:27:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.814+0000 7fbd8b535b80 1 -- 172.21.15.49:0/2417502040 wait complete. 2023-10-08T09:27:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.827+0000 7fbd8b535b80 20 client.26537 trim_cache size 0 max 16384 2023-10-08T09:27:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.866+0000 7fbd8b535b80 1 Processor -- start 2023-10-08T09:27:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.878+0000 7fbd8b535b80 1 -- start start 2023-10-08T09:27:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.878+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.878+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.878+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.884 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.878+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e530948c0 con 0x557e532c0e40 2023-10-08T09:27:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.878+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e532b62b0 con 0x557e532c22b0 2023-10-08T09:27:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.878+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e532d2f50 con 0x557e532c17a0 2023-10-08T09:27:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.892+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.893+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53516/0 (socket says 172.21.15.49:53516) 2023-10-08T09:27:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.893+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/3333136978 learned_addr learned my addr 172.21.15.49:0/3333136978 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.893+0000 7fbd7c8e7700 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.897+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.902+0000 7fbd7d8e9700 1 -- 172.21.15.49:0/3333136978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 msgr2=0x557e532d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.902+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532d1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.902+0000 7fbd7d8e9700 1 -- 172.21.15.49:0/3333136978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 msgr2=0x557e532c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.902+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.902+0000 7fbd7d8e9700 1 -- 172.21.15.49:0/3333136978 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e530651f0 con 0x557e532c22b0 2023-10-08T09:27:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.903+0000 7fbd7d8e9700 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532c87c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000be30 tx=0x7fbd70009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2065cbc574e59da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.910+0000 7fbd63fff700 1 -- 172.21.15.49:0/3333136978 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70099100 con 0x557e532c22b0 2023-10-08T09:27:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.910+0000 7fbd63fff700 1 -- 172.21.15.49:0/3333136978 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd70099ea0 con 0x557e532c22b0 2023-10-08T09:27:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.910+0000 7fbd63fff700 1 -- 172.21.15.49:0/3333136978 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70098c90 con 0x557e532c22b0 2023-10-08T09:27:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.911+0000 7fbd8b535b80 1 -- 172.21.15.49:0/3333136978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 msgr2=0x557e532c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.911+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532c87c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000be30 tx=0x7fbd70009900 comp rx=0 tx=0).stop 2023-10-08T09:27:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.916+0000 7fbd8b535b80 1 -- 172.21.15.49:0/3333136978 shutdown_connections 2023-10-08T09:27:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.916+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532c87c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.916+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.916+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/3333136978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.916+0000 7fbd8b535b80 1 -- 172.21.15.49:0/3333136978 >> 172.21.15.49:0/3333136978 conn(0x557e53235200 msgr2=0x557e532c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.916+0000 7fbd8b535b80 1 -- 172.21.15.49:0/3333136978 shutdown_connections 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.916+0000 7fbd8b535b80 1 -- 172.21.15.49:0/3333136978 wait complete. 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 -- start start 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 Processor -- start 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e532d1360 con 0x557e532c0e40 2023-10-08T09:27:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e532d14d0 con 0x557e532c22b0 2023-10-08T09:27:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e532d1690 con 0x557e532c17a0 2023-10-08T09:27:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.917+0000 7fbd8b535b80 10 client.0 _ll_register_callbacks cb 0x557e5300d7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53524/0 (socket says 172.21.15.49:53524) 2023-10-08T09:27:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/197964124 learned_addr learned my addr 172.21.15.49:0/197964124 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/197964124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 msgr2=0x557e532cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/197964124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 msgr2=0x557e532d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.918+0000 7fbd7d0e8700 1 -- 172.21.15.49:0/197964124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd781a52d0 con 0x557e532c0e40 2023-10-08T09:27:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.919+0000 7fbd7d0e8700 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532cd960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009b80 tx=0x7fbd74004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db529bb42b91d123 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.921+0000 7fbd60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:27:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.921+0000 7fbd60ff9700 1 -- 172.21.15.49:0/197964124 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7402d040 con 0x557e532c0e40 2023-10-08T09:27:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.921+0000 7fbd60ff9700 1 -- 172.21.15.49:0/197964124 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fbd74005db0 con 0x557e532c0e40 2023-10-08T09:27:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.923+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd74013470 con 0x557e532c0e40 2023-10-08T09:27:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.924+0000 7fbd60ff9700 1 -- 172.21.15.49:0/197964124 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd740343c0 con 0x557e532c0e40 2023-10-08T09:27:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.924+0000 7fbd8b535b80 10 client.16956 fetch_fsmap learned FSMap version 40 2023-10-08T09:27:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.924+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557e5303d250 con 0x557e532c0e40 2023-10-08T09:27:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.924+0000 7fbd60ff9700 1 -- 172.21.15.49:0/197964124 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd74013470 con 0x557e532c0e40 2023-10-08T09:27:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.925+0000 7fbd60ff9700 1 -- 172.21.15.49:0/197964124 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbd74034ce0 con 0x557e532c0e40 2023-10-08T09:27:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.925+0000 7fbd8b535b80 10 client.16956 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:27:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.925+0000 7fbd8b535b80 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.925+0000 7fbd8b535b80 20 client.16956 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.925+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e5334d3c0 0x557e5334f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.925+0000 7fbd8b535b80 4 client.16956 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:27:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.925+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557e5320cad0 con 0x557e5334d3c0 2023-10-08T09:27:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.926+0000 7fbd7c8e7700 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e5334d3c0 0x557e5334f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.926+0000 7fbd60ff9700 10 client.16956 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.926+0000 7fbd7c8e7700 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e5334d3c0 0x557e5334f7e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26fdb2eb17e4a439 server_cookie=e23afe749546e55e in_seq=0 out_seq=0 2023-10-08T09:27:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.958+0000 7fbd60ff9700 1 -- 172.21.15.49:0/197964124 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557e5320cad0 con 0x557e5334d3c0 2023-10-08T09:27:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.958+0000 7fbd60ff9700 10 client.16956 handle_command_reply: tid=0 2023-10-08T09:27:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.958+0000 7fbd8b535b80 1 client.16956 shutdown 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 msgr2=0x557e532cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532cd960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009b80 tx=0x7fbd74004860 comp rx=0 tx=0).stop 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 shutdown_connections 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557e5334d3c0 0x557e5334f7e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557e532c22b0 0x557e532d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557e532c0e40 0x557e532cd960 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 --2- 172.21.15.49:0/197964124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e532c17a0 0x557e532cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 >> 172.21.15.49:0/197964124 conn(0x557e53235200 msgr2=0x557e532c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd60ff9700 0 client.16956 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 shutdown_connections 2023-10-08T09:27:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 1 -- 172.21.15.49:0/197964124 wait complete. 2023-10-08T09:27:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.961+0000 7fbd8b535b80 20 client.16956 trim_cache size 0 max 16384 2023-10-08T09:27:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:27:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:27:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:27:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:27:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd640a1680 msgr2=0x7fbd640a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd640a1680 0x7fbd640a3b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbd74013cd0 tx=0x7fbd74008040 comp rx=0 tx=0).stop 2023-10-08T09:27:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810cb30 msgr2=0x7fbd781a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810cb30 0x7fbd781a4320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000be00 tx=0x7fbd70009ed0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 shutdown_connections 2023-10-08T09:27:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fbd640a1680 0x7fbd640a3b40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd7810cb30 0x7fbd781a4320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbd781077f0 0x7fbd7819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 --2- 172.21.15.49:0/2097624778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd7810a160 0x7fbd7819d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 >> 172.21.15.49:0/2097624778 conn(0x7fbd7806d770 msgr2=0x7fbd7810a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 shutdown_connections 2023-10-08T09:27:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:08.962+0000 7fbd7f34c700 1 -- 172.21.15.49:0/2097624778 wait complete. 2023-10-08T09:27:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 705.0 s, 483.5 tps, lat 204.184 ms stddev 217.819 2023-10-08T09:27:10.002 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:09 smithi184 ceph-mon[103675]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-08T09:27:10.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:09 smithi132 ceph-mon[103389]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-08T09:27:10.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:09 smithi049 ceph-mon[99485]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-08T09:27:11.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:11 smithi184 ceph-mon[103675]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 MiB/s wr, 470 op/s 2023-10-08T09:27:12.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:11 smithi132 ceph-mon[103389]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 MiB/s wr, 470 op/s 2023-10-08T09:27:12.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:11 smithi049 ceph-mon[99485]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 5.6 MiB/s wr, 470 op/s 2023-10-08T09:27:13.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:13 smithi049 ceph-mon[99485]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 514 op/s 2023-10-08T09:27:13.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:13 smithi184 ceph-mon[103675]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 514 op/s 2023-10-08T09:27:13.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:13 smithi132 ceph-mon[103389]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 MiB/s wr, 514 op/s 2023-10-08T09:27:14.316 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 710.0 s, 543.0 tps, lat 185.553 ms stddev 209.233 2023-10-08T09:27:15.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:15 smithi184 ceph-mon[103675]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.8 MiB/s wr, 490 op/s 2023-10-08T09:27:16.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:15 smithi132 ceph-mon[103389]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.8 MiB/s wr, 490 op/s 2023-10-08T09:27:16.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:15 smithi049 ceph-mon[99485]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.8 MiB/s wr, 490 op/s 2023-10-08T09:27:18.050 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:17 smithi132 ceph-mon[103389]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.1 MiB/s wr, 500 op/s 2023-10-08T09:27:18.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:17 smithi049 ceph-mon[99485]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.1 MiB/s wr, 500 op/s 2023-10-08T09:27:18.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:17 smithi184 ceph-mon[103675]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 6.1 MiB/s wr, 500 op/s 2023-10-08T09:27:19.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 715.0 s, 484.8 tps, lat 204.666 ms stddev 223.610 2023-10-08T09:27:20.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:19 smithi132 ceph-mon[103389]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 MiB/s wr, 527 op/s 2023-10-08T09:27:20.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:19 smithi049 ceph-mon[99485]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 MiB/s wr, 527 op/s 2023-10-08T09:27:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:19 smithi184 ceph-mon[103675]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 6.2 MiB/s wr, 527 op/s 2023-10-08T09:27:20.324 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:27:20.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.990+0000 7f4fde789700 1 -- 172.21.15.49:0/66730008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd8101990 msgr2=0x7f4fd8103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:20.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.990+0000 7f4fde789700 1 --2- 172.21.15.49:0/66730008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd8101990 0x7f4fd8103e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00098f0 tx=0x7f4fc0009c00 comp rx=0 tx=0).stop 2023-10-08T09:27:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.995+0000 7f4fde789700 1 -- 172.21.15.49:0/66730008 shutdown_connections 2023-10-08T09:27:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.995+0000 7f4fde789700 1 --2- 172.21.15.49:0/66730008 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8104360 0x7f4fd81047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:20.995 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.995+0000 7f4fde789700 1 --2- 172.21.15.49:0/66730008 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fd80ff020 0x7f4fd8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.995+0000 7f4fde789700 1 --2- 172.21.15.49:0/66730008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd8101990 0x7f4fd8103e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.995+0000 7f4fde789700 1 -- 172.21.15.49:0/66730008 >> 172.21.15.49:0/66730008 conn(0x7f4fd80fac40 msgr2=0x7f4fd80fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.995+0000 7f4fde789700 1 -- 172.21.15.49:0/66730008 shutdown_connections 2023-10-08T09:27:20.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.995+0000 7f4fde789700 1 -- 172.21.15.49:0/66730008 wait complete. 2023-10-08T09:27:20.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 Processor -- start 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 -- start start 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd80ff020 0x7f4fd819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8101990 0x7f4fd819d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fd8104360 0x7f4fd81a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd81a48f0 con 0x7f4fd8104360 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd8105640 con 0x7f4fd80ff020 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fde789700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd81057d0 con 0x7f4fd8101990 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd80ff020 0x7f4fd819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd80ff020 0x7f4fd819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43150/0 (socket says 172.21.15.49:43150) 2023-10-08T09:27:20.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fd7fff700 1 -- 172.21.15.49:0/1371949832 learned_addr learned my addr 172.21.15.49:0/1371949832 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fcffff700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8101990 0x7f4fd819d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.996+0000 7f4fdcd26700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fd8104360 0x7f4fd81a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.997+0000 7f4fd7fff700 1 -- 172.21.15.49:0/1371949832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8101990 msgr2=0x7f4fd819d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.997+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8101990 0x7f4fd819d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.997+0000 7f4fd7fff700 1 -- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fd8104360 msgr2=0x7f4fd81a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.997+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fd8104360 0x7f4fd81a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.997+0000 7f4fd7fff700 1 -- 172.21.15.49:0/1371949832 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80095f0 con 0x7f4fd80ff020 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.997+0000 7f4fcffff700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8101990 0x7f4fd819d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:27:20.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.997+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd80ff020 0x7f4fd819cdc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800ec10 tx=0x7f4fc800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.998+0000 7f4fd5ffb700 1 -- 172.21.15.49:0/1371949832 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800cc90 con 0x7f4fd80ff020 2023-10-08T09:27:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.998+0000 7f4fd5ffb700 1 -- 172.21.15.49:0/1371949832 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc80044e0 con 0x7f4fd80ff020 2023-10-08T09:27:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.998+0000 7f4fd5ffb700 1 -- 172.21.15.49:0/1371949832 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8010430 con 0x7f4fd80ff020 2023-10-08T09:27:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.998+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc0009520 con 0x7f4fd80ff020 2023-10-08T09:27:21.000 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:20.998+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd8105db0 con 0x7f4fd80ff020 2023-10-08T09:27:21.002 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.002+0000 7f4fd5ffb700 1 -- 172.21.15.49:0/1371949832 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4fc8003f70 con 0x7f4fd80ff020 2023-10-08T09:27:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fd5ffb700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4fc40a15d0 0x7f4fc40a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fcffff700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4fc40a15d0 0x7f4fc40a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.003 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fd5ffb700 1 -- 172.21.15.49:0/1371949832 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fc8014070 con 0x7f4fd80ff020 2023-10-08T09:27:21.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fea972b80 1 Processor -- start 2023-10-08T09:27:21.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fea972b80 1 -- start start 2023-10-08T09:27:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e863480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e864270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.003+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e864c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fea972b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e7cb460 con 0x55b54e8639c0 2023-10-08T09:27:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fea972b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e7bcc20 con 0x55b54e863060 2023-10-08T09:27:21.006 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fea972b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e7ba540 con 0x55b54e864800 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4fc40a15d0 0x7f4fc40a3a90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000fa40 tx=0x7f4fc0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e864270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fdcd26700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e864c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e864270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53546/0 (socket says 172.21.15.49:53546) 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 -- 172.21.15.49:0/835382365 learned_addr learned my addr 172.21.15.49:0/835382365 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 -- 172.21.15.49:0/835382365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 msgr2=0x55b54e864c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 --2- 172.21.15.49:0/835382365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e864c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 -- 172.21.15.49:0/835382365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 msgr2=0x55b54e863480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 --2- 172.21.15.49:0/835382365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e863480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 -- 172.21.15.49:0/835382365 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b54e5e0de0 con 0x55b54e8639c0 2023-10-08T09:27:21.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fcffff700 1 --2- 172.21.15.49:0/835382365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e864270 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0016000 tx=0x7f4fc001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5371b4f9ce7b9a75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.004+0000 7f4fce7fc700 1 -- 172.21.15.49:0/835382365 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000bb70 con 0x55b54e8639c0 2023-10-08T09:27:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.005+0000 7f4fce7fc700 1 -- 172.21.15.49:0/835382365 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc001d040 con 0x55b54e8639c0 2023-10-08T09:27:21.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.005+0000 7f4fce7fc700 1 -- 172.21.15.49:0/835382365 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0004bd0 con 0x55b54e8639c0 2023-10-08T09:27:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.005+0000 7f4fea972b80 1 -- 172.21.15.49:0/835382365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 msgr2=0x55b54e864270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:21.010 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.005+0000 7f4fea972b80 1 --2- 172.21.15.49:0/835382365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e864270 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0016000 tx=0x7f4fc001f000 comp rx=0 tx=0).stop 2023-10-08T09:27:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 -- 172.21.15.49:0/835382365 shutdown_connections 2023-10-08T09:27:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 --2- 172.21.15.49:0/835382365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e864c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 --2- 172.21.15.49:0/835382365 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e863480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 --2- 172.21.15.49:0/835382365 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e864270 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 -- 172.21.15.49:0/835382365 >> 172.21.15.49:0/835382365 conn(0x55b54e7d7260 msgr2=0x55b54e85de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 -- 172.21.15.49:0/835382365 shutdown_connections 2023-10-08T09:27:21.011 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 -- 172.21.15.49:0/835382365 wait complete. 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 -- start start 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.006+0000 7f4fea972b80 1 Processor -- start 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e863060 0x55b54e86cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e8639c0 0x55b54e86d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e864800 0x55b54e874320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fea972b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e7b5e30 con 0x55b54e863060 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fea972b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e752710 con 0x55b54e864800 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fea972b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e7523b0 con 0x55b54e8639c0 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fea972b80 10 client.0 _ll_register_callbacks cb 0x55b54e5af800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:21.012 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fdcd26700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e864800 0x55b54e874320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fdcd26700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e864800 0x55b54e874320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43164/0 (socket says 172.21.15.49:43164) 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.007+0000 7f4fdcd26700 1 -- 172.21.15.49:0/3281874529 learned_addr learned my addr 172.21.15.49:0/3281874529 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fcffff700 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e8639c0 0x55b54e86d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fdcd26700 1 -- 172.21.15.49:0/3281874529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e8639c0 msgr2=0x55b54e86d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fdcd26700 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e8639c0 0x55b54e86d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fdcd26700 1 -- 172.21.15.49:0/3281874529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e863060 msgr2=0x55b54e86cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fdcd26700 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e863060 0x55b54e86cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fdcd26700 1 -- 172.21.15.49:0/3281874529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd8105310 con 0x55b54e864800 2023-10-08T09:27:21.013 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fdcd26700 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e864800 0x55b54e874320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000ab40 tx=0x7f4fd000af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1def21b605986a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.008+0000 7f4fb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:27:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.009+0000 7f4fb77fe700 1 -- 172.21.15.49:0/3281874529 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0010040 con 0x55b54e864800 2023-10-08T09:27:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.009+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b54e77deb0 con 0x55b54e864800 2023-10-08T09:27:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.009+0000 7f4fb77fe700 1 -- 172.21.15.49:0/3281874529 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fd0006e40 con 0x55b54e864800 2023-10-08T09:27:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.009+0000 7f4fb77fe700 1 -- 172.21.15.49:0/3281874529 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fd0011c20 con 0x55b54e864800 2023-10-08T09:27:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.009+0000 7f4fea972b80 10 client.26549 fetch_fsmap learned FSMap version 40 2023-10-08T09:27:21.014 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.009+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b54e5c9c60 con 0x55b54e864800 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fb77fe700 1 -- 172.21.15.49:0/3281874529 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b54e77deb0 con 0x55b54e864800 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fb77fe700 1 -- 172.21.15.49:0/3281874529 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4fd0008030 con 0x55b54e864800 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fea972b80 10 client.26549 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fea972b80 10 client.26549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fea972b80 20 client.26549 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e878bd0 0x55b54e87aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fea972b80 4 client.26549 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:27:21.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.010+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b54e7aee10 con 0x55b54e878bd0 2023-10-08T09:27:21.018 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.011+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e878bd0 0x55b54e87aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.013+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e878bd0 0x55b54e87aff0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71018cbe7579e6ba server_cookie=7b228e19c6b62811 in_seq=0 out_seq=0 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.013+0000 7f4fb77fe700 10 client.26549 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fb77fe700 1 -- 172.21.15.49:0/3281874529 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b54e7aee10 con 0x55b54e878bd0 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fb77fe700 10 client.26549 handle_command_reply: tid=0 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 client.26549 shutdown 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e864800 msgr2=0x55b54e874320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e864800 0x55b54e874320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000ab40 tx=0x7f4fd000af00 comp rx=0 tx=0).stop 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 shutdown_connections 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e878bd0 0x55b54e87aff0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e864800 0x55b54e874320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e863060 0x55b54e86cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3281874529 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e8639c0 0x55b54e86d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 >> 172.21.15.49:0/3281874529 conn(0x55b54e7d7260 msgr2=0x55b54e85d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:21.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.014+0000 7f4fb77fe700 0 client.26549 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:21.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.015+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 shutdown_connections 2023-10-08T09:27:21.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.015+0000 7f4fea972b80 1 -- 172.21.15.49:0/3281874529 wait complete. 2023-10-08T09:27:21.021 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.015+0000 7f4fea972b80 20 client.26549 trim_cache size 0 max 16384 2023-10-08T09:27:21.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 Processor -- start 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 -- start start 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e863060 0x55b54e86bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e8639c0 0x55b54e8742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e864800 0x55b54e86c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e636920 con 0x55b54e864800 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e7aee10 con 0x55b54e8639c0 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fea972b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e8754a0 con 0x55b54e863060 2023-10-08T09:27:21.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fcffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e8639c0 0x55b54e8742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fcffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e8639c0 0x55b54e8742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43172/0 (socket says 172.21.15.49:43172) 2023-10-08T09:27:21.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fcffff700 1 -- 172.21.15.49:0/3434858571 learned_addr learned my addr 172.21.15.49:0/3434858571 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:21.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e863060 0x55b54e86bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.039+0000 7f4fdcd26700 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e864800 0x55b54e86c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.040+0000 7f4fd7fff700 1 -- 172.21.15.49:0/3434858571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e8639c0 msgr2=0x55b54e8742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:21.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.040+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e8639c0 0x55b54e8742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.040+0000 7f4fd7fff700 1 -- 172.21.15.49:0/3434858571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e864800 msgr2=0x55b54e86c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:21.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.040+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e864800 0x55b54e86c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.040+0000 7f4fd7fff700 1 -- 172.21.15.49:0/3434858571 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b54e607250 con 0x55b54e863060 2023-10-08T09:27:21.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.040+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e863060 0x55b54e86bf60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8097250 tx=0x7f4fc8003a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc56b4c50f94c418 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:21.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.042+0000 7f4fceffd700 1 -- 172.21.15.49:0/3434858571 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc809aee0 con 0x55b54e863060 2023-10-08T09:27:21.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.042+0000 7f4fceffd700 1 -- 172.21.15.49:0/3434858571 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc809a7e0 con 0x55b54e863060 2023-10-08T09:27:21.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.042+0000 7f4fceffd700 1 -- 172.21.15.49:0/3434858571 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8098b80 con 0x55b54e863060 2023-10-08T09:27:21.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 -- 172.21.15.49:0/3434858571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e863060 msgr2=0x55b54e86bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:21.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e863060 0x55b54e86bf60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8097250 tx=0x7f4fc8003a10 comp rx=0 tx=0).stop 2023-10-08T09:27:21.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 -- 172.21.15.49:0/3434858571 shutdown_connections 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e863060 0x55b54e86bf60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e864800 0x55b54e86c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 --2- 172.21.15.49:0/3434858571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e8639c0 0x55b54e8742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 -- 172.21.15.49:0/3434858571 >> 172.21.15.49:0/3434858571 conn(0x55b54e7d7260 msgr2=0x55b54e8677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 -- 172.21.15.49:0/3434858571 shutdown_connections 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 -- 172.21.15.49:0/3434858571 wait complete. 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 -- start start 2023-10-08T09:27:21.045 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.043+0000 7f4fea972b80 1 Processor -- start 2023-10-08T09:27:21.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e873d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e874240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fea972b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e872310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fea972b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e8729f0 con 0x55b54e8639c0 2023-10-08T09:27:21.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fea972b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e872b60 con 0x55b54e863060 2023-10-08T09:27:21.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fea972b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b54e872d20 con 0x55b54e864800 2023-10-08T09:27:21.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fea972b80 10 client.0 _ll_register_callbacks cb 0x7f4fd000a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:21.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e873d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.048 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e873d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43176/0 (socket says 172.21.15.49:43176) 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.046+0000 7f4fd7fff700 1 -- 172.21.15.49:0/4226270260 learned_addr learned my addr 172.21.15.49:0/4226270260 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.047+0000 7f4fdcd26700 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e872310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.047+0000 7f4fd7fff700 1 -- 172.21.15.49:0/4226270260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 msgr2=0x55b54e872310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.047+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e872310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.047+0000 7f4fd7fff700 1 -- 172.21.15.49:0/4226270260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 msgr2=0x55b54e874240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.047+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e874240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.047+0000 7f4fd7fff700 1 -- 172.21.15.49:0/4226270260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd8105db0 con 0x55b54e863060 2023-10-08T09:27:21.049 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.047+0000 7f4fd7fff700 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e873d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4fc809a4e0 tx=0x7f4fc8005d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=401b85349d1fa708 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:21.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.048+0000 7f4fb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:27:21.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.048+0000 7f4fb77fe700 1 -- 172.21.15.49:0/4226270260 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc809aee0 con 0x55b54e863060 2023-10-08T09:27:21.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.048+0000 7f4fb77fe700 1 -- 172.21.15.49:0/4226270260 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4fc80b8a00 con 0x55b54e863060 2023-10-08T09:27:21.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.048+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fc001d040 con 0x55b54e863060 2023-10-08T09:27:21.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fb77fe700 1 -- 172.21.15.49:0/4226270260 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc8098b80 con 0x55b54e863060 2023-10-08T09:27:21.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 10 client.26555 fetch_fsmap learned FSMap version 40 2023-10-08T09:27:21.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55b54e5df2b0 con 0x55b54e863060 2023-10-08T09:27:21.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fb77fe700 1 -- 172.21.15.49:0/4226270260 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fc001d040 con 0x55b54e863060 2023-10-08T09:27:21.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fb77fe700 1 -- 172.21.15.49:0/4226270260 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4fc80a9080 con 0x55b54e863060 2023-10-08T09:27:21.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 10 client.26555 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:27:21.054 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 10 client.26555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 20 client.26555 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e8cda70 0x55b54e8cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 4 client.26555 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b54e7aeb30 con 0x55b54e8cda70 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.049+0000 7f4fcffff700 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e8cda70 0x55b54e8cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fb77fe700 10 client.26555 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fcffff700 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e8cda70 0x55b54e8cfe90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5215464c9d58fb7 server_cookie=bb2fb867872b119d in_seq=0 out_seq=0 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fb77fe700 1 -- 172.21.15.49:0/4226270260 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b54e7aeb30 con 0x55b54e8cda70 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fb77fe700 10 client.26555 handle_command_reply: tid=0 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 client.26555 shutdown 2023-10-08T09:27:21.055 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 msgr2=0x55b54e873d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e873d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4fc809a4e0 tx=0x7f4fc8005d10 comp rx=0 tx=0).stop 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 shutdown_connections 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b54e8cda70 0x55b54e8cfe90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b54e864800 0x55b54e872310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fb77fe700 0 client.26555 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b54e863060 0x55b54e873d00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 --2- 172.21.15.49:0/4226270260 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b54e8639c0 0x55b54e874240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 >> 172.21.15.49:0/4226270260 conn(0x55b54e7d7260 msgr2=0x55b54e8677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 shutdown_connections 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 1 -- 172.21.15.49:0/4226270260 wait complete. 2023-10-08T09:27:21.056 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.050+0000 7f4fea972b80 20 client.26555 trim_cache size 0 max 16384 2023-10-08T09:27:21.057 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:27:21.057 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:27:21.057 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:27:21.057 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:27:21.057 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:27:21.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:27:21.057 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:27:21.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:27:21.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:27:21.058 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:27:21.058 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.051+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4fc40a15d0 msgr2=0x7f4fc40a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.051+0000 7f4fde789700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4fc40a15d0 0x7f4fc40a3a90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000fa40 tx=0x7f4fc0019040 comp rx=0 tx=0).stop 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.051+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd80ff020 msgr2=0x7f4fd819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.051+0000 7f4fde789700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd80ff020 0x7f4fd819cdc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800ec10 tx=0x7f4fc800ef20 comp rx=0 tx=0).stop 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.052+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 shutdown_connections 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.052+0000 7f4fde789700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4fc40a15d0 0x7f4fc40a3a90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.052+0000 7f4fde789700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4fd8104360 0x7f4fd81a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.052+0000 7f4fde789700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fd80ff020 0x7f4fd819cdc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.052+0000 7f4fde789700 1 --2- 172.21.15.49:0/1371949832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4fd8101990 0x7f4fd819d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.052+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 >> 172.21.15.49:0/1371949832 conn(0x7f4fd80fac40 msgr2=0x7f4fd80ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.054+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 shutdown_connections 2023-10-08T09:27:21.059 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:21.055+0000 7f4fde789700 1 -- 172.21.15.49:0/1371949832 wait complete. 2023-10-08T09:27:21.074 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:27:22.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:21 smithi132 ceph-mon[103389]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2023-10-08T09:27:22.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:21 smithi049 ceph-mon[99485]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2023-10-08T09:27:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.195+0000 7f5c49cba700 1 -- 172.21.15.49:0/747378222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4410cb30 msgr2=0x7f5c4410cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.195+0000 7f5c49cba700 1 --2- 172.21.15.49:0/747378222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4410cb30 0x7f5c4410cfb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009990 tx=0x7f5c38009ca0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.197+0000 7f5c49cba700 1 -- 172.21.15.49:0/747378222 shutdown_connections 2023-10-08T09:27:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.197+0000 7f5c49cba700 1 --2- 172.21.15.49:0/747378222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c441077f0 0x7f5c44109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.197+0000 7f5c49cba700 1 --2- 172.21.15.49:0/747378222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4410cb30 0x7f5c4410cfb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.197+0000 7f5c49cba700 1 --2- 172.21.15.49:0/747378222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c4410a160 0x7f5c4410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.197+0000 7f5c49cba700 1 -- 172.21.15.49:0/747378222 >> 172.21.15.49:0/747378222 conn(0x7f5c4406d770 msgr2=0x7f5c4406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.202+0000 7f5c49cba700 1 -- 172.21.15.49:0/747378222 shutdown_connections 2023-10-08T09:27:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.214+0000 7f5c49cba700 1 -- 172.21.15.49:0/747378222 wait complete. 2023-10-08T09:27:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 Processor -- start 2023-10-08T09:27:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 -- start start 2023-10-08T09:27:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c441077f0 0x7f5c44113da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c4410a160 0x7f5c441142e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c4410cb30 0x7f5c441b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44117370 con 0x7f5c441077f0 2023-10-08T09:27:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c441174e0 con 0x7f5c4410a160 2023-10-08T09:27:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.215+0000 7f5c49cba700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c441b5c50 con 0x7f5c4410cb30 2023-10-08T09:27:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.216+0000 7f5c43fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c4410cb30 0x7f5c441b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.217+0000 7f5c43fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c4410cb30 0x7f5c441b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40612/0 (socket says 172.21.15.49:40612) 2023-10-08T09:27:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.217+0000 7f5c43fff700 1 -- 172.21.15.49:0/2089765143 learned_addr learned my addr 172.21.15.49:0/2089765143 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.217+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c4410a160 0x7f5c441142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.218+0000 7f5c437fe700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c441077f0 0x7f5c44113da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.219+0000 7f5c43fff700 1 -- 172.21.15.49:0/2089765143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c4410a160 msgr2=0x7f5c441142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.219+0000 7f5c43fff700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c4410a160 0x7f5c441142e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.219+0000 7f5c43fff700 1 -- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c441077f0 msgr2=0x7f5c44113da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.219+0000 7f5c43fff700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c441077f0 0x7f5c44113da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.219+0000 7f5c43fff700 1 -- 172.21.15.49:0/2089765143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c380095f0 con 0x7f5c4410cb30 2023-10-08T09:27:22.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.230+0000 7f5c43fff700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c4410cb30 0x7f5c441b5710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c380036f0 tx=0x7f5c3800be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.240+0000 7f5c40ff9700 1 -- 172.21.15.49:0/2089765143 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3801d070 con 0x7f5c4410cb30 2023-10-08T09:27:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.240+0000 7f5c40ff9700 1 -- 172.21.15.49:0/2089765143 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3800f460 con 0x7f5c4410cb30 2023-10-08T09:27:22.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.240+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c441b5df0 con 0x7f5c4410cb30 2023-10-08T09:27:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.240+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c441b61b0 con 0x7f5c4410cb30 2023-10-08T09:27:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.245+0000 7f5c55ea3b80 1 Processor -- start 2023-10-08T09:27:22.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:21 smithi184 ceph-mon[103675]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2023-10-08T09:27:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.249+0000 7f5c40ff9700 1 -- 172.21.15.49:0/2089765143 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38017650 con 0x7f5c4410cb30 2023-10-08T09:27:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.249+0000 7f5c55ea3b80 1 -- start start 2023-10-08T09:27:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.249+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb855fe40 0x559eb8560260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.249+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb8560cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.250+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb85612b0 0x559eb8561730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.250+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb84c8400 con 0x559eb85612b0 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.250+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb84b9bc0 con 0x559eb855fe40 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.250+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb84b74e0 con 0x559eb85607a0 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb855fe40 0x559eb8560260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb855fe40 0x559eb8560260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43194/0 (socket says 172.21.15.49:43194) 2023-10-08T09:27:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 -- 172.21.15.49:0/1497546137 learned_addr learned my addr 172.21.15.49:0/1497546137 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb85612b0 0x559eb8561730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 -- 172.21.15.49:0/1497546137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 msgr2=0x559eb8560cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb8560cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 -- 172.21.15.49:0/1497546137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb85612b0 msgr2=0x559eb8561730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb85612b0 0x559eb8561730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.251+0000 7f5c42ffd700 1 -- 172.21.15.49:0/1497546137 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eb82ddd80 con 0x559eb855fe40 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.252+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb855fe40 0x559eb8560260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c002140 tx=0x7f5c3c001a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf8d3a8e23ccaa5e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:22.253 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.252+0000 7f5c40ff9700 1 -- 172.21.15.49:0/2089765143 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5c380052d0 con 0x7f5c4410cb30 2023-10-08T09:27:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.253+0000 7f5c40ff9700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c2c0a1680 0x7f5c2c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.253+0000 7f5c40ff9700 1 -- 172.21.15.49:0/2089765143 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5c380ce2e0 con 0x7f5c4410cb30 2023-10-08T09:27:22.255 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.253+0000 7f5c29ffb700 1 -- 172.21.15.49:0/1497546137 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0126c0 con 0x559eb855fe40 2023-10-08T09:27:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.253+0000 7f5c29ffb700 1 -- 172.21.15.49:0/1497546137 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3c00e040 con 0x559eb855fe40 2023-10-08T09:27:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.253+0000 7f5c29ffb700 1 -- 172.21.15.49:0/1497546137 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c017aa0 con 0x559eb855fe40 2023-10-08T09:27:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.255+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1497546137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb855fe40 msgr2=0x559eb8560260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.256 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.255+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb855fe40 0x559eb8560260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c002140 tx=0x7f5c3c001a00 comp rx=0 tx=0).stop 2023-10-08T09:27:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.256+0000 7f5c437fe700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c2c0a1680 0x7f5c2c0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.259+0000 7f5c437fe700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c2c0a1680 0x7f5c2c0a3b40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009890 tx=0x7f5c34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.261+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1497546137 shutdown_connections 2023-10-08T09:27:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.261+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb85612b0 0x559eb8561730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.261+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb855fe40 0x559eb8560260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.261+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1497546137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb8560cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.261+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1497546137 >> 172.21.15.49:0/1497546137 conn(0x559eb84d4200 msgr2=0x559eb855a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.263+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1497546137 shutdown_connections 2023-10-08T09:27:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.263+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1497546137 wait complete. 2023-10-08T09:27:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.264+0000 7f5c55ea3b80 1 -- start start 2023-10-08T09:27:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.264+0000 7f5c55ea3b80 1 Processor -- start 2023-10-08T09:27:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.264+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8569910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.264+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8569e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb84b2dd0 con 0x559eb855fe40 2023-10-08T09:27:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb844f6b0 con 0x559eb85607a0 2023-10-08T09:27:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb844f350 con 0x559eb85612b0 2023-10-08T09:27:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c55ea3b80 10 client.0 _ll_register_callbacks cb 0x559eb82ac7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8569e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8569e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43200/0 (socket says 172.21.15.49:43200) 2023-10-08T09:27:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 -- 172.21.15.49:0/178233432 learned_addr learned my addr 172.21.15.49:0/178233432 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:22.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c43fff700 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 -- 172.21.15.49:0/178233432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 msgr2=0x559eb8570e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 -- 172.21.15.49:0/178233432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 msgr2=0x559eb8569910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8569910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.265+0000 7f5c42ffd700 1 -- 172.21.15.49:0/178233432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c0142d0 con 0x559eb85607a0 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.266+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8569e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c017c30 tx=0x7f5c3c017d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3996aa493eb1941e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.266+0000 7f5c22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:27:22.268 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.266+0000 7f5c22ffd700 1 -- 172.21.15.49:0/178233432 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0128d0 con 0x559eb85607a0 2023-10-08T09:27:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.266+0000 7f5c22ffd700 1 -- 172.21.15.49:0/178233432 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5c3c006e40 con 0x559eb85607a0 2023-10-08T09:27:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.266+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559eb847ae50 con 0x559eb85607a0 2023-10-08T09:27:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.271+0000 7f5c22ffd700 1 -- 172.21.15.49:0/178233432 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c3c017430 con 0x559eb85607a0 2023-10-08T09:27:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.271+0000 7f5c55ea3b80 10 client.26561 fetch_fsmap learned FSMap version 40 2023-10-08T09:27:22.271 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.271+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559eb82c6c00 con 0x559eb85607a0 2023-10-08T09:27:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.271+0000 7f5c22ffd700 1 -- 172.21.15.49:0/178233432 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559eb847ae50 con 0x559eb85607a0 2023-10-08T09:27:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.271+0000 7f5c22ffd700 1 -- 172.21.15.49:0/178233432 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5c3c013070 con 0x559eb85607a0 2023-10-08T09:27:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.275+0000 7f5c55ea3b80 10 client.26561 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:27:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.275+0000 7f5c55ea3b80 10 client.26561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.276+0000 7f5c55ea3b80 20 client.26561 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.276+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb8575610 0x559eb8577a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.276+0000 7f5c55ea3b80 4 client.26561 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:27:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.276+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eb84abdb0 con 0x559eb8575610 2023-10-08T09:27:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.284+0000 7f5c437fe700 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb8575610 0x559eb8577a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.318+0000 7f5c22ffd700 10 client.26561 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.318+0000 7f5c437fe700 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb8575610 0x559eb8577a30 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d8870b53b4500c server_cookie=2da4726ed8189334 in_seq=0 out_seq=0 2023-10-08T09:27:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.330+0000 7f5c22ffd700 1 -- 172.21.15.49:0/178233432 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559eb84abdb0 con 0x559eb8575610 2023-10-08T09:27:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.330+0000 7f5c22ffd700 10 client.26561 handle_command_reply: tid=0 2023-10-08T09:27:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.330+0000 7f5c55ea3b80 1 client.26561 shutdown 2023-10-08T09:27:22.331 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 msgr2=0x559eb8569e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8569e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c017c30 tx=0x7f5c3c017d10 comp rx=0 tx=0).stop 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 shutdown_connections 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb8575610 0x559eb8577a30 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8569910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/178233432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8569e50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 >> 172.21.15.49:0/178233432 conn(0x559eb84d4200 msgr2=0x559eb855a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:22.332 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.331+0000 7f5c22ffd700 0 client.26561 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.352+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 shutdown_connections 2023-10-08T09:27:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.352+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/178233432 wait complete. 2023-10-08T09:27:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.357+0000 7f5c55ea3b80 20 client.26561 trim_cache size 0 max 16384 2023-10-08T09:27:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.396+0000 7f5c55ea3b80 1 Processor -- start 2023-10-08T09:27:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.403+0000 7f5c55ea3b80 1 -- start start 2023-10-08T09:27:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.403+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8571540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.403+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8561690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.403+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.403+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb83338c0 con 0x559eb855fe40 2023-10-08T09:27:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.403+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb85712f0 con 0x559eb85607a0 2023-10-08T09:27:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.403+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb85605e0 con 0x559eb85612b0 2023-10-08T09:27:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.407+0000 7f5c437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8571540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.407+0000 7f5c437fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8571540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53594/0 (socket says 172.21.15.49:53594) 2023-10-08T09:27:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.407+0000 7f5c437fe700 1 -- 172.21.15.49:0/1990010703 learned_addr learned my addr 172.21.15.49:0/1990010703 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.407+0000 7f5c43fff700 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.407+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8561690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c42ffd700 1 -- 172.21.15.49:0/1990010703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 msgr2=0x559eb8570db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c42ffd700 1 -- 172.21.15.49:0/1990010703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 msgr2=0x559eb8571540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8571540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c42ffd700 1 -- 172.21.15.49:0/1990010703 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eb83041f0 con 0x559eb85607a0 2023-10-08T09:27:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c43fff700 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:27:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8561690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c02e040 tx=0x7f5c3c023c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1385eb91300c40 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c2a7fc700 1 -- 172.21.15.49:0/1990010703 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c017430 con 0x559eb85607a0 2023-10-08T09:27:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c2a7fc700 1 -- 172.21.15.49:0/1990010703 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3c03a070 con 0x559eb85607a0 2023-10-08T09:27:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.408+0000 7f5c2a7fc700 1 -- 172.21.15.49:0/1990010703 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c014e60 con 0x559eb85607a0 2023-10-08T09:27:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.409+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1990010703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 msgr2=0x559eb8561690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.409+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8561690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c02e040 tx=0x7f5c3c023c30 comp rx=0 tx=0).stop 2023-10-08T09:27:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.410+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1990010703 shutdown_connections 2023-10-08T09:27:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.410+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85612b0 0x559eb8570db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.410+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb8571540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.410+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/1990010703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85607a0 0x559eb8561690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.410+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1990010703 >> 172.21.15.49:0/1990010703 conn(0x559eb84d4200 msgr2=0x559eb857c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.410+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1990010703 shutdown_connections 2023-10-08T09:27:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.411+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/1990010703 wait complete. 2023-10-08T09:27:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.411+0000 7f5c55ea3b80 1 -- start start 2023-10-08T09:27:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.411+0000 7f5c55ea3b80 1 Processor -- start 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb856eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb856f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c55ea3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85612b0 0x559eb8570000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb8570540 con 0x559eb855fe40 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb85706b0 con 0x559eb85612b0 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c55ea3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb8570870 con 0x559eb85607a0 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c55ea3b80 10 client.0 _ll_register_callbacks cb 0x7f5c3c0145a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c43fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85612b0 0x559eb8570000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c43fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85612b0 0x559eb8570000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43218/0 (socket says 172.21.15.49:43218) 2023-10-08T09:27:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c43fff700 1 -- 172.21.15.49:0/4033506108 learned_addr learned my addr 172.21.15.49:0/4033506108 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.412+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb856f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c437fe700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb856eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c42ffd700 1 -- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85612b0 msgr2=0x559eb8570000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85612b0 0x559eb8570000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c42ffd700 1 -- 172.21.15.49:0/4033506108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 msgr2=0x559eb856eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb856eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c42ffd700 1 -- 172.21.15.49:0/4033506108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559eb82c6c00 con 0x559eb85607a0 2023-10-08T09:27:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c43fff700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85612b0 0x559eb8570000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:27:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c42ffd700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb856f040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c001a60 tx=0x7f5c3c023b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdef959f80d4ae9f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.414+0000 7f5c22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c22ffd700 1 -- 172.21.15.49:0/4033506108 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c017430 con 0x559eb85607a0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c3c00e040 con 0x559eb85607a0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c22ffd700 1 -- 172.21.15.49:0/4033506108 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5c3c014e60 con 0x559eb85607a0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c22ffd700 1 -- 172.21.15.49:0/4033506108 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c3c0011c0 con 0x559eb85607a0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 10 client.26611 fetch_fsmap learned FSMap version 40 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559eb82dc250 con 0x559eb85607a0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c22ffd700 1 -- 172.21.15.49:0/4033506108 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3c00e040 con 0x559eb85607a0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c22ffd700 1 -- 172.21.15.49:0/4033506108 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 40) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f5c3c036440 con 0x559eb85607a0 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 10 client.26611 fetch_fsmap finished waiting for FSMap version 40 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 10 client.26611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 20 client.26611 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb85c9fa0 0x559eb85cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 4 client.26611 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:27:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.415+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559eb84abad0 con 0x559eb85c9fa0 2023-10-08T09:27:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.416+0000 7f5c437fe700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb85c9fa0 0x559eb85cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.416+0000 7f5c22ffd700 10 client.26611 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.416+0000 7f5c437fe700 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb85c9fa0 0x559eb85cc390 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d05eb102b04c3e server_cookie=ddd5157b2999dc3c in_seq=0 out_seq=0 2023-10-08T09:27:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c22ffd700 1 -- 172.21.15.49:0/4033506108 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559eb84abad0 con 0x559eb85c9fa0 2023-10-08T09:27:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c22ffd700 10 client.26611 handle_command_reply: tid=0 2023-10-08T09:27:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 client.26611 shutdown 2023-10-08T09:27:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 msgr2=0x559eb856f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb856f040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c001a60 tx=0x7f5c3c023b40 comp rx=0 tx=0).stop 2023-10-08T09:27:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 shutdown_connections 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x559eb85c9fa0 0x559eb85cc390 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x559eb85612b0 0x559eb8570000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c22ffd700 0 client.26611 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x559eb855fe40 0x559eb856eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 --2- 172.21.15.49:0/4033506108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x559eb85607a0 0x559eb856f040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 >> 172.21.15.49:0/4033506108 conn(0x559eb84d4200 msgr2=0x559eb8568e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 shutdown_connections 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 1 -- 172.21.15.49:0/4033506108 wait complete. 2023-10-08T09:27:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.428+0000 7f5c55ea3b80 20 client.26611 trim_cache size 0 max 16384 2023-10-08T09:27:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:27:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:27:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:27:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c2c0a1680 msgr2=0x7f5c2c0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c2c0a1680 0x7f5c2c0a3b40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009890 tx=0x7f5c34008040 comp rx=0 tx=0).stop 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c4410cb30 msgr2=0x7f5c441b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c4410cb30 0x7f5c441b5710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c380036f0 tx=0x7f5c3800be70 comp rx=0 tx=0).stop 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 shutdown_connections 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f5c2c0a1680 0x7f5c2c0a3b40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c4410cb30 0x7f5c441b5710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c441077f0 0x7f5c44113da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 --2- 172.21.15.49:0/2089765143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c4410a160 0x7f5c441142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 >> 172.21.15.49:0/2089765143 conn(0x7f5c4406d770 msgr2=0x7f5c441099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 shutdown_connections 2023-10-08T09:27:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:22.429+0000 7f5c49cba700 1 -- 172.21.15.49:0/2089765143 wait complete. 2023-10-08T09:27:23.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:23 smithi049 ceph-mon[99485]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 517 op/s 2023-10-08T09:27:23.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:23 smithi184 ceph-mon[103675]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 517 op/s 2023-10-08T09:27:23.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:23 smithi132 ceph-mon[103389]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 517 op/s 2023-10-08T09:27:24.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 720.0 s, 542.6 tps, lat 185.708 ms stddev 220.607 2023-10-08T09:27:24.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:24 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:27:24.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:24 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:27:24.552 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:24 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:27:25.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:25 smithi049 ceph-mon[99485]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-10-08T09:27:25.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:25 smithi184 ceph-mon[103675]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-10-08T09:27:25.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:25 smithi132 ceph-mon[103389]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 501 op/s 2023-10-08T09:27:27.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:27 smithi184 ceph-mon[103675]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2023-10-08T09:27:28.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:27 smithi132 ceph-mon[103389]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2023-10-08T09:27:28.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:27 smithi049 ceph-mon[99485]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.8 MiB/s wr, 526 op/s 2023-10-08T09:27:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 725.0 s, 500.3 tps, lat 194.562 ms stddev 205.095 2023-10-08T09:27:29.997 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:29 smithi184 ceph-mon[103675]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 560 op/s 2023-10-08T09:27:30.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:29 smithi132 ceph-mon[103389]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 560 op/s 2023-10-08T09:27:30.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:29 smithi049 ceph-mon[99485]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.2 MiB/s wr, 560 op/s 2023-10-08T09:27:32.049 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:31 smithi132 ceph-mon[103389]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 496 op/s 2023-10-08T09:27:32.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:31 smithi049 ceph-mon[99485]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 496 op/s 2023-10-08T09:27:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:31 smithi184 ceph-mon[103675]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.6 MiB/s wr, 496 op/s 2023-10-08T09:27:33.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:33 smithi049 ceph-mon[99485]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 MiB/s wr, 513 op/s 2023-10-08T09:27:33.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:33 smithi184 ceph-mon[103675]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 MiB/s wr, 513 op/s 2023-10-08T09:27:33.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:33 smithi132 ceph-mon[103389]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.6 MiB/s wr, 513 op/s 2023-10-08T09:27:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 730.0 s, 582.6 tps, lat 176.224 ms stddev 202.780 2023-10-08T09:27:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.623+0000 7f988c6d6700 1 -- 172.21.15.49:0/1886228917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98840725e0 msgr2=0x7f9884072a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.623+0000 7f988c6d6700 1 --2- 172.21.15.49:0/1886228917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98840725e0 0x7f9884072a00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f98800098f0 tx=0x7f9880009c00 comp rx=0 tx=0).stop 2023-10-08T09:27:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.644+0000 7f988c6d6700 1 -- 172.21.15.49:0/1886228917 shutdown_connections 2023-10-08T09:27:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.644+0000 7f988c6d6700 1 --2- 172.21.15.49:0/1886228917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98840725e0 0x7f9884072a00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.644+0000 7f988c6d6700 1 --2- 172.21.15.49:0/1886228917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9884077ee0 0x7f988407a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.644+0000 7f988c6d6700 1 --2- 172.21.15.49:0/1886228917 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9884075510 0x7f98840779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.646 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.644+0000 7f988c6d6700 1 -- 172.21.15.49:0/1886228917 >> 172.21.15.49:0/1886228917 conn(0x7f988406d770 msgr2=0x7f988406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.663+0000 7f988c6d6700 1 -- 172.21.15.49:0/1886228917 shutdown_connections 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.671+0000 7f988c6d6700 1 -- 172.21.15.49:0/1886228917 wait complete. 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.671+0000 7f988c6d6700 1 Processor -- start 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.672+0000 7f988c6d6700 1 -- start start 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.672+0000 7f988c6d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98840725e0 0x7f98841b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.672+0000 7f988c6d6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9884075510 0x7f98841b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.672+0000 7f988c6d6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9884077ee0 0x7f98841b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.672+0000 7f988c6d6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98841b5830 con 0x7f9884075510 2023-10-08T09:27:35.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.672+0000 7f988c6d6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98841b59a0 con 0x7f98840725e0 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.672+0000 7f988c6d6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98841b5b60 con 0x7f9884077ee0 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98840725e0 0x7f98841b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98840725e0 0x7f98841b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43228/0 (socket says 172.21.15.49:43228) 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 -- 172.21.15.49:0/2115054671 learned_addr learned my addr 172.21.15.49:0/2115054671 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 -- 172.21.15.49:0/2115054671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9884077ee0 msgr2=0x7f98841b5060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9884077ee0 0x7f98841b5060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 -- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9884075510 msgr2=0x7f98841b2a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9884075510 0x7f98841b2a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.674+0000 7f988a472700 1 -- 172.21.15.49:0/2115054671 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9880009520 con 0x7f98840725e0 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.675+0000 7f988a472700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98840725e0 0x7f98841b0520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9880009230 tx=0x7f988000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:35.678 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.675+0000 7f987b7fe700 1 -- 172.21.15.49:0/2115054671 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988001c070 con 0x7f98840725e0 2023-10-08T09:27:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.675+0000 7f987b7fe700 1 -- 172.21.15.49:0/2115054671 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f988000f7d0 con 0x7f98840725e0 2023-10-08T09:27:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.675+0000 7f987b7fe700 1 -- 172.21.15.49:0/2115054671 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880004b70 con 0x7f98840725e0 2023-10-08T09:27:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.676+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98841b5de0 con 0x7f98840725e0 2023-10-08T09:27:35.679 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.676+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98841b6210 con 0x7f98840725e0 2023-10-08T09:27:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.681+0000 7f98988bfb80 1 Processor -- start 2023-10-08T09:27:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f98988bfb80 1 -- start start 2023-10-08T09:27:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23842c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23842ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f98988bfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238394400 con 0x55c23842d2b0 2023-10-08T09:27:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f98988bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c238385bc0 con 0x55c23842c7a0 2023-10-08T09:27:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f98988bfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2383834e0 con 0x55c23842be40 2023-10-08T09:27:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f988ac73700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23842c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f988ac73700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23842c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40652/0 (socket says 172.21.15.49:40652) 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.682+0000 7f988ac73700 1 -- 172.21.15.49:0/3927075699 learned_addr learned my addr 172.21.15.49:0/3927075699 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.683+0000 7f988a472700 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23842ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.683+0000 7f988ac73700 1 -- 172.21.15.49:0/3927075699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 msgr2=0x55c23842ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.683+0000 7f988ac73700 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23842ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.683+0000 7f988ac73700 1 -- 172.21.15.49:0/3927075699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 msgr2=0x55c23842d730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.683+0000 7f988ac73700 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.683+0000 7f988ac73700 1 -- 172.21.15.49:0/3927075699 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2381a9d80 con 0x55c23842be40 2023-10-08T09:27:35.688 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.687+0000 7f988ac73700 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23842c260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9874007a10 tx=0x7f9874007d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df11af1887b9dd88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.689+0000 7f987b7fe700 1 -- 172.21.15.49:0/2115054671 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9880020070 con 0x7f98840725e0 2023-10-08T09:27:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.690+0000 7f987b7fe700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f98700a1530 0x7f98700a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.691+0000 7f987b7fe700 1 -- 172.21.15.49:0/2115054671 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f98800cdcb0 con 0x7f98840725e0 2023-10-08T09:27:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.695+0000 7f9878ff9700 1 -- 172.21.15.49:0/3927075699 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874004020 con 0x55c23842be40 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.695+0000 7f9878ff9700 1 -- 172.21.15.49:0/3927075699 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9874004830 con 0x55c23842be40 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.695+0000 7f9878ff9700 1 -- 172.21.15.49:0/3927075699 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874005420 con 0x55c23842be40 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.696+0000 7f98988bfb80 1 -- 172.21.15.49:0/3927075699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 msgr2=0x55c23842c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.696+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23842c260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9874007a10 tx=0x7f9874007d20 comp rx=0 tx=0).stop 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.696+0000 7f98988bfb80 1 -- 172.21.15.49:0/3927075699 shutdown_connections 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.696+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23842c260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.696+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.696+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3927075699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23842ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.696+0000 7f98988bfb80 1 -- 172.21.15.49:0/3927075699 >> 172.21.15.49:0/3927075699 conn(0x55c2383a0200 msgr2=0x55c2384268a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.697+0000 7f98988bfb80 1 -- 172.21.15.49:0/3927075699 shutdown_connections 2023-10-08T09:27:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.697+0000 7f988a472700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f98700a1530 0x7f98700a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.699+0000 7f98988bfb80 1 -- 172.21.15.49:0/3927075699 wait complete. 2023-10-08T09:27:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.699+0000 7f98988bfb80 1 -- start start 2023-10-08T09:27:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.699+0000 7f98988bfb80 1 Processor -- start 2023-10-08T09:27:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.702+0000 7f988a472700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f98700a1530 0x7f98700a39f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f988009a9d0 tx=0x7f98800a1040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.703+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842be40 0x55c238435940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.703+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c238435e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842d2b0 0x55c23843ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f98988bfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23837edd0 con 0x55c23842c7a0 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f98988bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23831b6b0 con 0x55c23842be40 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f98988bfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23831b350 con 0x55c23842d2b0 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f98988bfb80 10 client.0 _ll_register_callbacks cb 0x55c2381787a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f9889c71700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c238435e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f9889c71700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c238435e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53624/0 (socket says 172.21.15.49:53624) 2023-10-08T09:27:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f9889c71700 1 -- 172.21.15.49:0/3731242475 learned_addr learned my addr 172.21.15.49:0/3731242475 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f988ac73700 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842d2b0 0x55c23843ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f988ac73700 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842be40 0x55c238435940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.704+0000 7f9889c71700 1 -- 172.21.15.49:0/3731242475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842d2b0 msgr2=0x55c23843ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.705+0000 7f9889c71700 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842d2b0 0x55c23843ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.705+0000 7f9889c71700 1 -- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842be40 msgr2=0x55c238435940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.705+0000 7f9889c71700 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842be40 0x55c238435940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.705+0000 7f9889c71700 1 -- 172.21.15.49:0/3731242475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9874007670 con 0x55c23842c7a0 2023-10-08T09:27:35.708 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.705+0000 7f9889c71700 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c238435e80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f987c00ec00 tx=0x7f987c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27664ec53de23fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.708+0000 7f986dffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:27:35.712 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.708+0000 7f986dffb700 1 -- 172.21.15.49:0/3731242475 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c017070 con 0x55c23842c7a0 2023-10-08T09:27:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.708+0000 7f986dffb700 1 -- 172.21.15.49:0/3731242475 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f987c0092e0 con 0x55c23842c7a0 2023-10-08T09:27:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.711+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c238346e50 con 0x55c23842c7a0 2023-10-08T09:27:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.712+0000 7f986dffb700 1 -- 172.21.15.49:0/3731242475 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f987c004bb0 con 0x55c23842c7a0 2023-10-08T09:27:35.713 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.712+0000 7f98988bfb80 10 client.16989 fetch_fsmap learned FSMap version 41 2023-10-08T09:27:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.712+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c238192c00 con 0x55c23842c7a0 2023-10-08T09:27:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.712+0000 7f986dffb700 1 -- 172.21.15.49:0/3731242475 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c238346e50 con 0x55c23842c7a0 2023-10-08T09:27:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.715+0000 7f986dffb700 1 -- 172.21.15.49:0/3731242475 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f987c01a020 con 0x55c23842c7a0 2023-10-08T09:27:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.720+0000 7f98988bfb80 10 client.16989 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:27:35.721 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.720+0000 7f98988bfb80 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.720+0000 7f98988bfb80 20 client.16989 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.720+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c2384416d0 0x55c238443af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.720+0000 7f98988bfb80 4 client.16989 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:27:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.720+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c238377db0 con 0x55c2384416d0 2023-10-08T09:27:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.720+0000 7f988ac73700 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c2384416d0 0x55c238443af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.721+0000 7f986dffb700 10 client.16989 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:35.723 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.721+0000 7f988ac73700 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c2384416d0 0x55c238443af0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ea4f443a4af83df server_cookie=6e5ba3c4cb4cdf64 in_seq=0 out_seq=0 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.721+0000 7f986dffb700 1 -- 172.21.15.49:0/3731242475 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c238377db0 con 0x55c2384416d0 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.721+0000 7f986dffb700 10 client.16989 handle_command_reply: tid=0 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.721+0000 7f98988bfb80 1 client.16989 shutdown 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 msgr2=0x55c238435e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c238435e80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f987c00ec00 tx=0x7f987c00c0c0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 shutdown_connections 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c2384416d0 0x55c238443af0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842d2b0 0x55c23843ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f986dffb700 0 client.16989 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842be40 0x55c238435940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 --2- 172.21.15.49:0/3731242475 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c238435e80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 >> 172.21.15.49:0/3731242475 conn(0x55c2383a0200 msgr2=0x55c238426010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 shutdown_connections 2023-10-08T09:27:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 1 -- 172.21.15.49:0/3731242475 wait complete. 2023-10-08T09:27:35.725 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.722+0000 7f98988bfb80 20 client.16989 trim_cache size 0 max 16384 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 Processor -- start 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 -- start start 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c238434e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23843cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2381ff8c0 con 0x55c23842d2b0 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2384343b0 con 0x55c23842c7a0 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f98988bfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2384212b0 con 0x55c23842be40 2023-10-08T09:27:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f988ac73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f988ac73700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53634/0 (socket says 172.21.15.49:53634) 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.745+0000 7f988ac73700 1 -- 172.21.15.49:0/2573779915 learned_addr learned my addr 172.21.15.49:0/2573779915 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f988ac73700 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c238434e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f9889c71700 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23843cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f988ac73700 1 -- 172.21.15.49:0/2573779915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 msgr2=0x55c238434e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f988ac73700 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c238434e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f988ac73700 1 -- 172.21.15.49:0/2573779915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 msgr2=0x55c23843cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f988ac73700 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23843cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f988ac73700 1 -- 172.21.15.49:0/2573779915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2381d01f0 con 0x55c23842d2b0 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f9889c71700 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23843cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:27:35.749 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.746+0000 7f988ac73700 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d690 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9874005d90 tx=0x7f9874000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784fcd851d3e3b19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.748+0000 7f98797fa700 1 -- 172.21.15.49:0/2573779915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874005020 con 0x55c23842d2b0 2023-10-08T09:27:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98797fa700 1 -- 172.21.15.49:0/2573779915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f987402d070 con 0x55c23842d2b0 2023-10-08T09:27:35.750 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98797fa700 1 -- 172.21.15.49:0/2573779915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874027da0 con 0x55c23842d2b0 2023-10-08T09:27:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 -- 172.21.15.49:0/2573779915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 msgr2=0x55c23842d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d690 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9874005d90 tx=0x7f9874000bc0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 -- 172.21.15.49:0/2573779915 shutdown_connections 2023-10-08T09:27:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c238434e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842d2b0 0x55c23842d690 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.751 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 --2- 172.21.15.49:0/2573779915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842c7a0 0x55c23843cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 -- 172.21.15.49:0/2573779915 >> 172.21.15.49:0/2573779915 conn(0x55c2383a0200 msgr2=0x55c238448450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 -- 172.21.15.49:0/2573779915 shutdown_connections 2023-10-08T09:27:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.749+0000 7f98988bfb80 1 -- 172.21.15.49:0/2573779915 wait complete. 2023-10-08T09:27:35.752 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.750+0000 7f98988bfb80 1 -- start start 2023-10-08T09:27:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.750+0000 7f98988bfb80 1 Processor -- start 2023-10-08T09:27:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23843ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c23843b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f98988bfb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842d2b0 0x55c23843c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f98988bfb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23843c800 con 0x55c23842c7a0 2023-10-08T09:27:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f98988bfb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23843c970 con 0x55c23842d2b0 2023-10-08T09:27:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f98988bfb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c23843cb30 con 0x55c23842be40 2023-10-08T09:27:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f98988bfb80 10 client.0 _ll_register_callbacks cb 0x7f987c00e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:36.017 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:35 smithi184 ceph-mon[103675]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.0 MiB/s wr, 534 op/s 2023-10-08T09:27:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f9889c71700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c23843b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f9889c71700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c23843b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53638/0 (socket says 172.21.15.49:53638) 2023-10-08T09:27:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f988ac73700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842d2b0 0x55c23843c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f9889c71700 1 -- 172.21.15.49:0/4102249856 learned_addr learned my addr 172.21.15.49:0/4102249856 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.751+0000 7f988ac73700 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23843ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f988ac73700 1 -- 172.21.15.49:0/4102249856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 msgr2=0x55c23843ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f988ac73700 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23843ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f988ac73700 1 -- 172.21.15.49:0/4102249856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 msgr2=0x55c23843b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f988ac73700 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c23843b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f988ac73700 1 -- 172.21.15.49:0/4102249856 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9874020040 con 0x55c23842d2b0 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f988ac73700 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842d2b0 0x55c23843c2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9874005a40 tx=0x7f9874007fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914afeb460065e41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f986dffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.752+0000 7f986dffb700 1 -- 172.21.15.49:0/4102249856 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874004020 con 0x55c23842d2b0 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9874004830 con 0x55c23842d2b0 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f986dffb700 1 -- 172.21.15.49:0/4102249856 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9874004180 con 0x55c23842d2b0 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f986dffb700 1 -- 172.21.15.49:0/4102249856 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9874004340 con 0x55c23842d2b0 2023-10-08T09:27:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 10 client.26579 fetch_fsmap learned FSMap version 41 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c2381a8250 con 0x55c23842d2b0 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f986dffb700 1 -- 172.21.15.49:0/4102249856 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f987403e030 con 0x55c23842d2b0 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f986dffb700 1 -- 172.21.15.49:0/4102249856 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f987400d8f0 con 0x55c23842d2b0 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 10 client.26579 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 10 client.26579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 20 client.26579 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c238496160 0x55c238498550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 4 client.26579 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c238377ad0 con 0x55c238496160 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.753+0000 7f9889c71700 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c238496160 0x55c238498550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.754+0000 7f986dffb700 10 client.26579 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.754+0000 7f9889c71700 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c238496160 0x55c238498550 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5126305404ba3 server_cookie=54b774f95de4da1e in_seq=0 out_seq=0 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.772+0000 7f986dffb700 1 -- 172.21.15.49:0/4102249856 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c238377ad0 con 0x55c238496160 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.772+0000 7f986dffb700 10 client.26579 handle_command_reply: tid=0 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.772+0000 7f98988bfb80 1 client.26579 shutdown 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842d2b0 msgr2=0x55c23843c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842d2b0 0x55c23843c2c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9874005a40 tx=0x7f9874007fd0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 shutdown_connections 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55c238496160 0x55c238498550 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55c23842be40 0x55c23843ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55c23842d2b0 0x55c23843c2c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f986dffb700 0 client.26579 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 --2- 172.21.15.49:0/4102249856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55c23842c7a0 0x55c23843b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 >> 172.21.15.49:0/4102249856 conn(0x55c2383a0200 msgr2=0x55c2384358e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 shutdown_connections 2023-10-08T09:27:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 1 -- 172.21.15.49:0/4102249856 wait complete. 2023-10-08T09:27:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f98988bfb80 20 client.26579 trim_cache size 0 max 16384 2023-10-08T09:27:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:27:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:27:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:27:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:27:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f98700a1530 msgr2=0x7f98700a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f988c6d6700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f98700a1530 0x7f98700a39f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f988009a9d0 tx=0x7f98800a1040 comp rx=0 tx=0).stop 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.773+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98840725e0 msgr2=0x7f98841b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98840725e0 0x7f98841b0520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9880009230 tx=0x7f988000b840 comp rx=0 tx=0).stop 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 shutdown_connections 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f98700a1530 0x7f98700a39f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9884077ee0 0x7f98841b5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f98840725e0 0x7f98841b0520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 --2- 172.21.15.49:0/2115054671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9884075510 0x7f98841b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 >> 172.21.15.49:0/2115054671 conn(0x7f988406d770 msgr2=0x7f9884076470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 shutdown_connections 2023-10-08T09:27:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:35.774+0000 7f988c6d6700 1 -- 172.21.15.49:0/2115054671 wait complete. 2023-10-08T09:27:36.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:35 smithi132 ceph-mon[103389]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.0 MiB/s wr, 534 op/s 2023-10-08T09:27:36.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:35 smithi049 ceph-mon[99485]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.0 MiB/s wr, 534 op/s 2023-10-08T09:27:38.062 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:37 smithi132 ceph-mon[103389]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 530 op/s 2023-10-08T09:27:38.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:37 smithi049 ceph-mon[99485]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 530 op/s 2023-10-08T09:27:38.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:37 smithi184 ceph-mon[103675]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.1 MiB/s wr, 530 op/s 2023-10-08T09:27:39.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 735.0 s, 531.2 tps, lat 181.864 ms stddev 195.721 2023-10-08T09:27:40.055 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:39 smithi132 ceph-mon[103389]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 489 op/s 2023-10-08T09:27:40.137 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:39 smithi049 ceph-mon[99485]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 489 op/s 2023-10-08T09:27:40.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:39 smithi184 ceph-mon[103675]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.4 MiB/s wr, 489 op/s 2023-10-08T09:27:42.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:41 smithi049 ceph-mon[99485]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.1 MiB/s wr, 485 op/s 2023-10-08T09:27:42.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:41 smithi184 ceph-mon[103675]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.1 MiB/s wr, 485 op/s 2023-10-08T09:27:42.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:41 smithi132 ceph-mon[103389]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.1 MiB/s wr, 485 op/s 2023-10-08T09:27:43.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:43 smithi184 ceph-mon[103675]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2023-10-08T09:27:43.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:43 smithi132 ceph-mon[103389]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2023-10-08T09:27:43.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:43 smithi049 ceph-mon[99485]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2023-10-08T09:27:44.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 740.0 s, 521.3 tps, lat 195.638 ms stddev 201.699 2023-10-08T09:27:44.504 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:44 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:27:44.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:44 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:27:44.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:44 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:27:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:45 smithi184 ceph-mon[103675]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.8 MiB/s wr, 527 op/s 2023-10-08T09:27:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:45 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:45 smithi132 ceph-mon[103389]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.8 MiB/s wr, 527 op/s 2023-10-08T09:27:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:45 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:45 smithi049 ceph-mon[99485]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.8 MiB/s wr, 527 op/s 2023-10-08T09:27:45.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:45.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:45 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:46.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:46 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:27:46.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:46 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:27:46.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:46 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:46.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:46 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:27:46.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:46 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:27:46.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:46 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:46.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:46 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:27:46.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:46 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:27:46.884 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:46 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:27:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:47 smithi132 ceph-mon[103389]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.2 MiB/s wr, 514 op/s 2023-10-08T09:27:47.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:47 smithi049 ceph-mon[99485]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.2 MiB/s wr, 514 op/s 2023-10-08T09:27:48.003 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:47 smithi184 ceph-mon[103675]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.2 MiB/s wr, 514 op/s 2023-10-08T09:27:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.095+0000 7fadafd82700 1 -- 172.21.15.49:0/4010312274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 msgr2=0x7fada8109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.095+0000 7fadafd82700 1 --2- 172.21.15.49:0/4010312274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada8109c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fad980098f0 tx=0x7fad98009c00 comp rx=0 tx=0).stop 2023-10-08T09:27:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.095+0000 7fadafd82700 1 -- 172.21.15.49:0/4010312274 shutdown_connections 2023-10-08T09:27:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.095+0000 7fadafd82700 1 --2- 172.21.15.49:0/4010312274 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada810cb30 0x7fada810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.095+0000 7fadafd82700 1 --2- 172.21.15.49:0/4010312274 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada8109c20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.095+0000 7fadafd82700 1 --2- 172.21.15.49:0/4010312274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fada810a160 0x7fada810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.095+0000 7fadafd82700 1 -- 172.21.15.49:0/4010312274 >> 172.21.15.49:0/4010312274 conn(0x7fada806d770 msgr2=0x7fada806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.098+0000 7fadafd82700 1 -- 172.21.15.49:0/4010312274 shutdown_connections 2023-10-08T09:27:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.101+0000 7fadafd82700 1 -- 172.21.15.49:0/4010312274 wait complete. 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 Processor -- start 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 -- start start 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada81a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fada810cb30 0x7fada81a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada81aac20 0x7fada81ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada81ad5e0 con 0x7fada81aac20 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada81ad720 con 0x7fada81077f0 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadafd82700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada81ad8b0 con 0x7fada810cb30 2023-10-08T09:27:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada81a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada81a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43270/0 (socket says 172.21.15.49:43270) 2023-10-08T09:27:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.102+0000 7fadadb1e700 1 -- 172.21.15.49:0/1260160219 learned_addr learned my addr 172.21.15.49:0/1260160219 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.103+0000 7fadadb1e700 1 -- 172.21.15.49:0/1260160219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fada810cb30 msgr2=0x7fada81a5c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:27:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.103+0000 7fadadb1e700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fada810cb30 0x7fada81a5c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.103+0000 7fadadb1e700 1 -- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada81aac20 msgr2=0x7fada81ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.103+0000 7fadadb1e700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada81aac20 0x7fada81ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.103+0000 7fadadb1e700 1 -- 172.21.15.49:0/1260160219 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98009520 con 0x7fada81077f0 2023-10-08T09:27:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.104+0000 7fadadb1e700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada81a56d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad98000c00 tx=0x7fad9800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.109+0000 7fad9effd700 1 -- 172.21.15.49:0/1260160219 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9801c070 con 0x7fada81077f0 2023-10-08T09:27:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.109+0000 7fad9effd700 1 -- 172.21.15.49:0/1260160219 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad980054f0 con 0x7fada81077f0 2023-10-08T09:27:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.109+0000 7fad9effd700 1 -- 172.21.15.49:0/1260160219 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98020410 con 0x7fada81077f0 2023-10-08T09:27:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.109+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada81adb30 con 0x7fada81077f0 2023-10-08T09:27:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.109+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada81adf00 con 0x7fada81077f0 2023-10-08T09:27:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.114+0000 7fad9effd700 1 -- 172.21.15.49:0/1260160219 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fad980046d0 con 0x7fada81077f0 2023-10-08T09:27:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.114+0000 7fad9effd700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad940a1580 0x7fad940a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.115+0000 7fad9effd700 1 -- 172.21.15.49:0/1260160219 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fad980ce1b0 con 0x7fada81077f0 2023-10-08T09:27:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.115+0000 7fadad31d700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad940a1580 0x7fad940a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.116+0000 7fadad31d700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad940a1580 0x7fad940a3a40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fada8071c70 tx=0x7fada4006c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 Processor -- start 2023-10-08T09:27:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 -- start start 2023-10-08T09:27:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a824e40 0x561d6a825260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a825cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8262b0 0x561d6a826730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a78d400 con 0x561d6a8262b0 2023-10-08T09:27:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a77ebc0 con 0x561d6a824e40 2023-10-08T09:27:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.121+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a77c4e0 con 0x561d6a8257a0 2023-10-08T09:27:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8262b0 0x561d6a826730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8262b0 0x561d6a826730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53660/0 (socket says 172.21.15.49:53660) 2023-10-08T09:27:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 -- 172.21.15.49:0/2756293208 learned_addr learned my addr 172.21.15.49:0/2756293208 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 -- 172.21.15.49:0/2756293208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 msgr2=0x561d6a825cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 --2- 172.21.15.49:0/2756293208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a825cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 -- 172.21.15.49:0/2756293208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a824e40 msgr2=0x561d6a825260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 --2- 172.21.15.49:0/2756293208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a824e40 0x561d6a825260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.122+0000 7fadae31f700 1 -- 172.21.15.49:0/2756293208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d6a5a2d80 con 0x561d6a8262b0 2023-10-08T09:27:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.123+0000 7fadae31f700 1 --2- 172.21.15.49:0/2756293208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8262b0 0x561d6a826730 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fada0009c40 tx=0x7fada0009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7c40628ede5c1a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.123+0000 7fad93fff700 1 -- 172.21.15.49:0/2756293208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0017070 con 0x561d6a8262b0 2023-10-08T09:27:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.123+0000 7fad93fff700 1 -- 172.21.15.49:0/2756293208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fada000f040 con 0x561d6a8262b0 2023-10-08T09:27:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.123+0000 7fad93fff700 1 -- 172.21.15.49:0/2756293208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0005970 con 0x561d6a8262b0 2023-10-08T09:27:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.124+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2756293208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8262b0 msgr2=0x561d6a826730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.124+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2756293208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8262b0 0x561d6a826730 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fada0009c40 tx=0x7fada0009f50 comp rx=0 tx=0).stop 2023-10-08T09:27:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.127+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2756293208 shutdown_connections 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.127+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2756293208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8262b0 0x561d6a826730 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.127+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2756293208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a824e40 0x561d6a825260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.127+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2756293208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a825cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.127+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2756293208 >> 172.21.15.49:0/2756293208 conn(0x561d6a799200 msgr2=0x561d6a81f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.129+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2756293208 shutdown_connections 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.129+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2756293208 wait complete. 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.129+0000 7fadbbf6bb80 1 -- start start 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.129+0000 7fadbbf6bb80 1 Processor -- start 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a824e40 0x561d6a83f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8257a0 0x561d6a83ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a846ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a777dd0 con 0x561d6a8257a0 2023-10-08T09:27:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a7146b0 con 0x561d6a8262b0 2023-10-08T09:27:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a714350 con 0x561d6a824e40 2023-10-08T09:27:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadbbf6bb80 10 client.0 _ll_register_callbacks cb 0x561d6a5717a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a824e40 0x561d6a83f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a824e40 0x561d6a83f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40696/0 (socket says 172.21.15.49:40696) 2023-10-08T09:27:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadadb1e700 1 -- 172.21.15.49:0/2540823608 learned_addr learned my addr 172.21.15.49:0/2540823608 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.130+0000 7fadae31f700 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a846ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadad31d700 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8257a0 0x561d6a83ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadadb1e700 1 -- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 msgr2=0x561d6a846ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadadb1e700 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a846ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadadb1e700 1 -- 172.21.15.49:0/2540823608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8257a0 msgr2=0x561d6a83ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadadb1e700 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8257a0 0x561d6a83ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadadb1e700 1 -- 172.21.15.49:0/2540823608 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada00098a0 con 0x561d6a824e40 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadadb1e700 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a824e40 0x561d6a83f9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad9809aed0 tx=0x7fad9809b0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ad3e1db8dec8d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fad917fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fad917fa700 1 -- 172.21.15.49:0/2540823608 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9809a8c0 con 0x561d6a824e40 2023-10-08T09:27:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fad917fa700 1 -- 172.21.15.49:0/2540823608 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fad98005660 con 0x561d6a824e40 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d6a73fe50 con 0x561d6a824e40 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fad917fa700 1 -- 172.21.15.49:0/2540823608 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad9809f120 con 0x561d6a824e40 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadbbf6bb80 10 client.26629 fetch_fsmap learned FSMap version 41 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.131+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561d6a58bc00 con 0x561d6a824e40 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fad917fa700 1 -- 172.21.15.49:0/2540823608 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad980b7070 con 0x561d6a824e40 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fad917fa700 1 -- 172.21.15.49:0/2540823608 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fad980a3040 con 0x561d6a824e40 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fadbbf6bb80 10 client.26629 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fadbbf6bb80 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fadbbf6bb80 20 client.26629 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a84b360 0x561d6a84d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fadbbf6bb80 4 client.26629 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:27:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d6a770db0 con 0x561d6a84b360 2023-10-08T09:27:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.132+0000 7fadad31d700 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a84b360 0x561d6a84d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.133+0000 7fad917fa700 10 client.26629 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:49.138 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.133+0000 7fadad31d700 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a84b360 0x561d6a84d780 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467cb2672f5795ac server_cookie=81e6cb97a6e7b115 in_seq=0 out_seq=0 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.133+0000 7fad917fa700 1 -- 172.21.15.49:0/2540823608 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561d6a770db0 con 0x561d6a84b360 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.133+0000 7fad917fa700 10 client.26629 handle_command_reply: tid=0 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.133+0000 7fadbbf6bb80 1 client.26629 shutdown 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.134+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a824e40 msgr2=0x561d6a83f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.134+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a824e40 0x561d6a83f9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad9809aed0 tx=0x7fad9809b0b0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 shutdown_connections 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a84b360 0x561d6a84d780 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a824e40 0x561d6a83f9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a846ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/2540823608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a8257a0 0x561d6a83ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 >> 172.21.15.49:0/2540823608 conn(0x561d6a799200 msgr2=0x561d6a81f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:49.139 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fad917fa700 0 client.26629 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 shutdown_connections 2023-10-08T09:27:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.135+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/2540823608 wait complete. 2023-10-08T09:27:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.140+0000 7fadbbf6bb80 20 client.26629 trim_cache size 0 max 16384 2023-10-08T09:27:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 Processor -- start 2023-10-08T09:27:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 -- start start 2023-10-08T09:27:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a847550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a846e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a81f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a5f88c0 con 0x561d6a824e40 2023-10-08T09:27:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a81a2b0 con 0x561d6a8262b0 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a826130 con 0x561d6a8257a0 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a847550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a847550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53670/0 (socket says 172.21.15.49:53670) 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadadb1e700 1 -- 172.21.15.49:0/3643787641 learned_addr learned my addr 172.21.15.49:0/3643787641 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadad31d700 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a846e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.167+0000 7fadae31f700 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a81f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fadadb1e700 1 -- 172.21.15.49:0/3643787641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 msgr2=0x561d6a846e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fadadb1e700 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a846e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fadadb1e700 1 -- 172.21.15.49:0/3643787641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 msgr2=0x561d6a81f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fadadb1e700 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a81f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fadadb1e700 1 -- 172.21.15.49:0/3643787641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d6a5c91f0 con 0x561d6a824e40 2023-10-08T09:27:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fadad31d700 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a846e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:27:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fadadb1e700 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a847550 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fad98005950 tx=0x7fad9809ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbabb7465e6f7fb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fad9cff9700 1 -- 172.21.15.49:0/3643787641 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9809f120 con 0x561d6a824e40 2023-10-08T09:27:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fad9cff9700 1 -- 172.21.15.49:0/3643787641 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad980b7070 con 0x561d6a824e40 2023-10-08T09:27:49.171 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.168+0000 7fad9cff9700 1 -- 172.21.15.49:0/3643787641 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9809f320 con 0x561d6a824e40 2023-10-08T09:27:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/3643787641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 msgr2=0x561d6a847550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a847550 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fad98005950 tx=0x7fad9809ae40 comp rx=0 tx=0).stop 2023-10-08T09:27:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/3643787641 shutdown_connections 2023-10-08T09:27:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8262b0 0x561d6a81f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a847550 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/3643787641 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8257a0 0x561d6a846e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/3643787641 >> 172.21.15.49:0/3643787641 conn(0x561d6a799200 msgr2=0x561d6a8520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/3643787641 shutdown_connections 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/3643787641 wait complete. 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.169+0000 7fadbbf6bb80 1 -- start start 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 1 Processor -- start 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a842640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8257a0 0x561d6a842b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8262b0 0x561d6a845ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a8460e0 con 0x561d6a824e40 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a846250 con 0x561d6a8257a0 2023-10-08T09:27:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d6a846410 con 0x561d6a8262b0 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadbbf6bb80 10 client.0 _ll_register_callbacks cb 0x7fad98097510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a842640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadadb1e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a842640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53674/0 (socket says 172.21.15.49:53674) 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadadb1e700 1 -- 172.21.15.49:0/330723719 learned_addr learned my addr 172.21.15.49:0/330723719 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.170+0000 7fadae31f700 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8262b0 0x561d6a845ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadae31f700 1 -- 172.21.15.49:0/330723719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8257a0 msgr2=0x561d6a842b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadae31f700 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8257a0 0x561d6a842b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadae31f700 1 -- 172.21.15.49:0/330723719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 msgr2=0x561d6a842640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadae31f700 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a842640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadae31f700 1 -- 172.21.15.49:0/330723719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada40135d0 con 0x561d6a8262b0 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadadb1e700 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a842640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:27:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadae31f700 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8262b0 0x561d6a845ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fada0000f80 tx=0x7fada0003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86502663d6d25893 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fad917fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fad917fa700 1 -- 172.21.15.49:0/330723719 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0017040 con 0x561d6a8262b0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fad917fa700 1 -- 172.21.15.49:0/330723719 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fada0004bb0 con 0x561d6a8262b0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fada000f040 con 0x561d6a8262b0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fad917fa700 1 -- 172.21.15.49:0/330723719 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fada0005750 con 0x561d6a8262b0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadbbf6bb80 10 client.26635 fetch_fsmap learned FSMap version 41 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.171+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561d6a58bc00 con 0x561d6a8262b0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fad917fa700 1 -- 172.21.15.49:0/330723719 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fada000f040 con 0x561d6a8262b0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fad917fa700 1 -- 172.21.15.49:0/330723719 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fada0012070 con 0x561d6a8262b0 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fadbbf6bb80 10 client.26635 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fadbbf6bb80 10 client.26635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:49.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fadbbf6bb80 20 client.26635 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a89fb20 0x561d6a8a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fadbbf6bb80 4 client.26635 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:27:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d6a770ad0 con 0x561d6a89fb20 2023-10-08T09:27:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.172+0000 7fadadb1e700 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a89fb20 0x561d6a8a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.173+0000 7fad917fa700 10 client.26635 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.173+0000 7fadadb1e700 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a89fb20 0x561d6a8a1f40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=961bd194a04bf7f1 server_cookie=3b633a77f19b0ee7 in_seq=0 out_seq=0 2023-10-08T09:27:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.185+0000 7fad917fa700 1 -- 172.21.15.49:0/330723719 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d6a770ad0 con 0x561d6a89fb20 2023-10-08T09:27:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.185+0000 7fad917fa700 10 client.26635 handle_command_reply: tid=0 2023-10-08T09:27:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.185+0000 7fadbbf6bb80 1 client.26635 shutdown 2023-10-08T09:27:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.185+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8262b0 msgr2=0x561d6a845ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.185+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8262b0 0x561d6a845ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fada0000f80 tx=0x7fada0003f90 comp rx=0 tx=0).stop 2023-10-08T09:27:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.186+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 shutdown_connections 2023-10-08T09:27:49.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.186+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x561d6a89fb20 0x561d6a8a1f40 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.186+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d6a8262b0 0x561d6a845ba0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.186+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x561d6a824e40 0x561d6a842640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.186+0000 7fadbbf6bb80 1 --2- 172.21.15.49:0/330723719 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x561d6a8257a0 0x561d6a842b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.186+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 >> 172.21.15.49:0/330723719 conn(0x561d6a799200 msgr2=0x561d6a81bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.186+0000 7fad917fa700 0 client.26635 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 shutdown_connections 2023-10-08T09:27:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadbbf6bb80 1 -- 172.21.15.49:0/330723719 wait complete. 2023-10-08T09:27:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadbbf6bb80 20 client.26635 trim_cache size 0 max 16384 2023-10-08T09:27:49.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:27:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:27:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:27:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad940a1580 msgr2=0x7fad940a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad940a1580 0x7fad940a3a40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fada8071c70 tx=0x7fada4006c10 comp rx=0 tx=0).stop 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 msgr2=0x7fada81a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada81a56d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fad98000c00 tx=0x7fad9800b8d0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 shutdown_connections 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fad940a1580 0x7fad940a3a40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fada81aac20 0x7fada81ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fada81077f0 0x7fada81a56d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 --2- 172.21.15.49:0/1260160219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fada810cb30 0x7fada81a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.187+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 >> 172.21.15.49:0/1260160219 conn(0x7fada806d770 msgr2=0x7fada810a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.188+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 shutdown_connections 2023-10-08T09:27:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:27:49.188+0000 7fadafd82700 1 -- 172.21.15.49:0/1260160219 wait complete. 2023-10-08T09:27:49.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:48 smithi184 ceph-mon[103675]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2023-10-08T09:27:49.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:48 smithi132 ceph-mon[103389]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2023-10-08T09:27:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 745.0 s, 476.5 tps, lat 214.473 ms stddev 226.203 2023-10-08T09:27:49.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:48 smithi049 ceph-mon[99485]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.3 MiB/s wr, 493 op/s 2023-10-08T09:27:51.076 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:27:51.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 -- 172.21.15.49:0/346352041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1007c0 msgr2=0x7fd71c100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 --2- 172.21.15.49:0/346352041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1007c0 0x7fd71c100be0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd718009920 tx=0x7fd718009c30 comp rx=0 tx=0).stop 2023-10-08T09:27:51.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 -- 172.21.15.49:0/346352041 shutdown_connections 2023-10-08T09:27:51.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 --2- 172.21.15.49:0/346352041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71c102350 0x7fd71c108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.535 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 --2- 172.21.15.49:0/346352041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1007c0 0x7fd71c100be0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 --2- 172.21.15.49:0/346352041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd71c1019b0 0x7fd71c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 -- 172.21.15.49:0/346352041 >> 172.21.15.49:0/346352041 conn(0x7fd71c0fbd30 msgr2=0x7fd71c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 -- 172.21.15.49:0/346352041 shutdown_connections 2023-10-08T09:27:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.534+0000 7fd723cac700 1 -- 172.21.15.49:0/346352041 wait complete. 2023-10-08T09:27:51.536 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.535+0000 7fd723cac700 1 Processor -- start 2023-10-08T09:27:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.536+0000 7fd723cac700 1 -- start start 2023-10-08T09:27:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.536+0000 7fd723cac700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd71c1007c0 0x7fd71c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.536+0000 7fd723cac700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1019b0 0x7fd71c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.536+0000 7fd723cac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71c102350 0x7fd71c1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.536+0000 7fd723cac700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a49a0 con 0x7fd71c1019b0 2023-10-08T09:27:51.537 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.536+0000 7fd723cac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a4ae0 con 0x7fd71c102350 2023-10-08T09:27:51.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.536+0000 7fd723cac700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1a4c70 con 0x7fd71c1007c0 2023-10-08T09:27:51.538 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1019b0 0x7fd71c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1019b0 0x7fd71c19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53686/0 (socket says 172.21.15.49:53686) 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 -- 172.21.15.49:0/2193658228 learned_addr learned my addr 172.21.15.49:0/2193658228 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd722249700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71c102350 0x7fd71c1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 -- 172.21.15.49:0/2193658228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd71c1007c0 msgr2=0x7fd71c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd71c1007c0 0x7fd71c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 -- 172.21.15.49:0/2193658228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71c102350 msgr2=0x7fd71c1a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71c102350 0x7fd71c1a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd721247700 1 -- 172.21.15.49:0/2193658228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718009520 con 0x7fd71c1019b0 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.537+0000 7fd722249700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71c102350 0x7fd71c1a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:27:51.539 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.538+0000 7fd721247700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1019b0 0x7fd71c19d3b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00e9c0 tx=0x7fd70c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:51.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.538+0000 7fd712ffd700 1 -- 172.21.15.49:0/2193658228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00cbd0 con 0x7fd71c1019b0 2023-10-08T09:27:51.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.538+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c1a4f50 con 0x7fd71c1019b0 2023-10-08T09:27:51.543 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.538+0000 7fd712ffd700 1 -- 172.21.15.49:0/2193658228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70c00cd30 con 0x7fd71c1019b0 2023-10-08T09:27:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.538+0000 7fd712ffd700 1 -- 172.21.15.49:0/2193658228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c005590 con 0x7fd71c1019b0 2023-10-08T09:27:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.538+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c1a5420 con 0x7fd71c1019b0 2023-10-08T09:27:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.542+0000 7fd712ffd700 1 -- 172.21.15.49:0/2193658228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd70c004830 con 0x7fd71c1019b0 2023-10-08T09:27:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.542+0000 7fd72fe95b80 1 Processor -- start 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd72fe95b80 1 -- start start 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e4f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e4f9c0 0x558041e50270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e50800 0x558041e50c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041db7460 con 0x558041e50800 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041da8c20 con 0x558041e4f9c0 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd712ffd700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd7080a1550 0x7fd7080a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041da6540 con 0x558041e4f060 2023-10-08T09:27:51.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd721247700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e50800 0x558041e50c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd721a48700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e4f9c0 0x558041e50270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd721247700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e50800 0x558041e50c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53694/0 (socket says 172.21.15.49:53694) 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd721247700 1 -- 172.21.15.49:0/1184097427 learned_addr learned my addr 172.21.15.49:0/1184097427 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd722249700 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.543+0000 7fd722249700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd7080a1550 0x7fd7080a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd712ffd700 1 -- 172.21.15.49:0/2193658228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd70c014070 con 0x7fd71c1019b0 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd721247700 1 -- 172.21.15.49:0/1184097427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 msgr2=0x558041e4f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd721247700 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e4f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd721247700 1 -- 172.21.15.49:0/1184097427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e4f9c0 msgr2=0x558041e50270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd721247700 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e4f9c0 0x558041e50270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd721247700 1 -- 172.21.15.49:0/1184097427 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558041bccde0 con 0x558041e50800 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd722249700 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e4f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd721247700 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e50800 0x558041e50c90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd70c005950 tx=0x7fd70c01b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c70567af4e3b9a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd707fff700 1 -- 172.21.15.49:0/1184097427 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c09a010 con 0x558041e50800 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd722249700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd7080a1550 0x7fd7080a3a10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd7140076c0 tx=0x7fd714007650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd707fff700 1 -- 172.21.15.49:0/1184097427 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70c099d80 con 0x558041e50800 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.544+0000 7fd707fff700 1 -- 172.21.15.49:0/1184097427 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c09a2d0 con 0x558041e50800 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1184097427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e50800 msgr2=0x558041e50c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e50800 0x558041e50c90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd70c005950 tx=0x7fd70c01b4f0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1184097427 shutdown_connections 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e4f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e50800 0x558041e50c90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.552 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1184097427 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e4f9c0 0x558041e50270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1184097427 >> 172.21.15.49:0/1184097427 conn(0x558041dc3260 msgr2=0x558041e49e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1184097427 shutdown_connections 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.545+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1184097427 wait complete. 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.546+0000 7fd72fe95b80 1 -- start start 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.546+0000 7fd72fe95b80 1 Processor -- start 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.546+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f060 0x558041e58e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f9c0 0x558041e59370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041da1e30 con 0x558041e4f060 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041d3e710 con 0x558041e50800 2023-10-08T09:27:51.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041d3e3b0 con 0x558041e4f9c0 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd72fe95b80 10 client.0 _ll_register_callbacks cb 0x558041b9b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd721a48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f060 0x558041e58e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd721a48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f060 0x558041e58e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53700/0 (socket says 172.21.15.49:53700) 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd722249700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd721a48700 1 -- 172.21.15.49:0/1227993149 learned_addr learned my addr 172.21.15.49:0/1227993149 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd721247700 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f9c0 0x558041e59370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd722249700 1 -- 172.21.15.49:0/1227993149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f9c0 msgr2=0x558041e59370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd722249700 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f9c0 0x558041e59370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd722249700 1 -- 172.21.15.49:0/1227993149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f060 msgr2=0x558041e58e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd722249700 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f060 0x558041e58e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd722249700 1 -- 172.21.15.49:0/1227993149 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71400a8d0 con 0x558041e50800 2023-10-08T09:27:51.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd721a48700 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f060 0x558041e58e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd722249700 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e603b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd714017c40 tx=0x7fd714004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617ea86afecd1c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.547+0000 7fd7057fa700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd7057fa700 1 -- 172.21.15.49:0/1227993149 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714004bb0 con 0x558041e50800 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558041d69eb0 con 0x558041e50800 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd7057fa700 1 -- 172.21.15.49:0/1227993149 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd714004d10 con 0x558041e50800 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd7057fa700 1 -- 172.21.15.49:0/1227993149 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd71401da50 con 0x558041e50800 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 10 client.26600 fetch_fsmap learned FSMap version 41 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fd71c1a5420 con 0x558041e50800 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd7057fa700 1 -- 172.21.15.49:0/1227993149 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558041d69eb0 con 0x558041e50800 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd7057fa700 1 -- 172.21.15.49:0/1227993149 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd71401aa00 con 0x558041e50800 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 10 client.26600 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:27:51.555 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 10 client.26600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:51.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 20 client.26600 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:51.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041e64d30 0x558041e67150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 4 client.26600 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:27:51.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.548+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558041d9ae10 con 0x558041e64d30 2023-10-08T09:27:51.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.549+0000 7fd721a48700 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041e64d30 0x558041e67150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.549+0000 7fd721a48700 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041e64d30 0x558041e67150 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc7732a8198c834 server_cookie=9bc48076948c72b9 in_seq=0 out_seq=0 2023-10-08T09:27:51.556 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.549+0000 7fd7057fa700 10 client.26600 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:51.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd7057fa700 1 -- 172.21.15.49:0/1227993149 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558041d9ae10 con 0x558041e64d30 2023-10-08T09:27:51.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd7057fa700 10 client.26600 handle_command_reply: tid=0 2023-10-08T09:27:51.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 client.26600 shutdown 2023-10-08T09:27:51.557 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 msgr2=0x558041e603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e603b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd714017c40 tx=0x7fd714004b40 comp rx=0 tx=0).stop 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 shutdown_connections 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041e64d30 0x558041e67150 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e603b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f060 0x558041e58e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd7057fa700 0 client.26600 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1227993149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f9c0 0x558041e59370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.551+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 >> 172.21.15.49:0/1227993149 conn(0x558041dc3260 msgr2=0x558041e494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.552+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 shutdown_connections 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.552+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1227993149 wait complete. 2023-10-08T09:27:51.558 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.553+0000 7fd72fe95b80 20 client.26600 trim_cache size 0 max 16384 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.588+0000 7fd72fe95b80 1 Processor -- start 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.588+0000 7fd72fe95b80 1 -- start start 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e50be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e60370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e45f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041c22920 con 0x558041e4f9c0 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041e445d0 con 0x558041e50800 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041e5a070 con 0x558041e4f060 2023-10-08T09:27:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd721a48700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e50be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd721a48700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e50be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40742/0 (socket says 172.21.15.49:40742) 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.589+0000 7fd721a48700 1 -- 172.21.15.49:0/2823155124 learned_addr learned my addr 172.21.15.49:0/2823155124 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.590+0000 7fd721a48700 1 -- 172.21.15.49:0/2823155124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 msgr2=0x558041e45f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.590+0000 7fd721a48700 1 --2- 172.21.15.49:0/2823155124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e45f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.590+0000 7fd721a48700 1 -- 172.21.15.49:0/2823155124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 msgr2=0x558041e60370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.590+0000 7fd721a48700 1 --2- 172.21.15.49:0/2823155124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e60370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.590+0000 7fd721a48700 1 -- 172.21.15.49:0/2823155124 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558041bb5c60 con 0x558041e4f060 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.590+0000 7fd721a48700 1 --2- 172.21.15.49:0/2823155124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e50be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd71c101810 tx=0x7fd7180094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d3772f071ecbbb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.593+0000 7fd710d82700 1 -- 172.21.15.49:0/2823155124 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718047a10 con 0x558041e4f060 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.593+0000 7fd710d82700 1 -- 172.21.15.49:0/2823155124 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd71801d040 con 0x558041e4f060 2023-10-08T09:27:51.598 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.593+0000 7fd710d82700 1 -- 172.21.15.49:0/2823155124 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718005280 con 0x558041e4f060 2023-10-08T09:27:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.594+0000 7fd72fe95b80 1 -- 172.21.15.49:0/2823155124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 msgr2=0x558041e50be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.594+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/2823155124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e50be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd71c101810 tx=0x7fd7180094b0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.594+0000 7fd72fe95b80 1 -- 172.21.15.49:0/2823155124 shutdown_connections 2023-10-08T09:27:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.594+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/2823155124 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e50be0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.594+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/2823155124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e45f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.594+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/2823155124 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e60370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.599 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.594+0000 7fd72fe95b80 1 -- 172.21.15.49:0/2823155124 >> 172.21.15.49:0/2823155124 conn(0x558041dc3260 msgr2=0x558041e49dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:51.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.618+0000 7fd72fe95b80 1 -- 172.21.15.49:0/2823155124 shutdown_connections 2023-10-08T09:27:51.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.620+0000 7fd72fe95b80 1 -- 172.21.15.49:0/2823155124 wait complete. 2023-10-08T09:27:51.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.620+0000 7fd72fe95b80 1 -- start start 2023-10-08T09:27:51.621 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.620+0000 7fd72fe95b80 1 Processor -- start 2023-10-08T09:27:51.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.621+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e5fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.621+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.622 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.621+0000 7fd72fe95b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e5c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.621+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041e5ccf0 con 0x558041e4f9c0 2023-10-08T09:27:51.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.621+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041e5ce60 con 0x558041e50800 2023-10-08T09:27:51.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.621+0000 7fd72fe95b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x558041e5d020 con 0x558041e4f060 2023-10-08T09:27:51.623 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.621+0000 7fd72fe95b80 10 client.0 _ll_register_callbacks cb 0x7fd714007450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:27:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53710/0 (socket says 172.21.15.49:53710) 2023-10-08T09:27:51.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 -- 172.21.15.49:0/1642654142 learned_addr learned my addr 172.21.15.49:0/1642654142 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:27:51.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 -- 172.21.15.49:0/1642654142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 msgr2=0x558041e5fb90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:27:51.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e5fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 -- 172.21.15.49:0/1642654142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 msgr2=0x558041e5c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:27:51.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e5c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 -- 172.21.15.49:0/1642654142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c109550 con 0x558041e4f9c0 2023-10-08T09:27:51.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd721247700 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e600d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd70c004800 tx=0x7fd70c099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9860878ff1484778 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:27:51.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.624+0000 7fd7057fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:27:51.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.625+0000 7fd7057fa700 1 -- 172.21.15.49:0/1642654142 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c09e390 con 0x558041e4f9c0 2023-10-08T09:27:51.626 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.625+0000 7fd7057fa700 1 -- 172.21.15.49:0/1642654142 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd70c0a25d0 con 0x558041e4f9c0 2023-10-08T09:27:51.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.626+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd70c099d80 con 0x558041e4f9c0 2023-10-08T09:27:51.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.626+0000 7fd7057fa700 1 -- 172.21.15.49:0/1642654142 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd70c0056f0 con 0x558041e4f9c0 2023-10-08T09:27:51.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.626+0000 7fd72fe95b80 10 client.17025 fetch_fsmap learned FSMap version 41 2023-10-08T09:27:51.627 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.626+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558041bf3250 con 0x558041e4f9c0 2023-10-08T09:27:51.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.626+0000 7fd7057fa700 1 -- 172.21.15.49:0/1642654142 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70c099d80 con 0x558041e4f9c0 2023-10-08T09:27:51.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.626+0000 7fd7057fa700 1 -- 172.21.15.49:0/1642654142 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd70c09a5c0 con 0x558041e4f9c0 2023-10-08T09:27:51.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.635+0000 7fd72fe95b80 10 client.17025 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:27:51.638 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.635+0000 7fd72fe95b80 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:27:51.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.635+0000 7fd72fe95b80 20 client.17025 populate_metadata read hostname 'smithi049' 2023-10-08T09:27:51.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.635+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041eb9d90 0x558041ebc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:27:51.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.635+0000 7fd72fe95b80 4 client.17025 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:27:51.639 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.635+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558041d9ab30 con 0x558041eb9d90 2023-10-08T09:27:51.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.640+0000 7fd721a48700 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041eb9d90 0x558041ebc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:27:51.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.640+0000 7fd7057fa700 10 client.17025 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:51.641 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.640+0000 7fd721a48700 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041eb9d90 0x558041ebc1b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fda727c3dbc98e1 server_cookie=d279b7a9202f00e in_seq=0 out_seq=0 2023-10-08T09:27:51.654 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:27:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:27:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:27:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:27:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:27:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd7057fa700 1 -- 172.21.15.49:0/1642654142 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558041d9ab30 con 0x558041eb9d90 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd7057fa700 10 client.17025 handle_command_reply: tid=0 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 client.17025 shutdown 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 msgr2=0x558041e600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.655 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e600d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd70c004800 tx=0x7fd70c099c50 comp rx=0 tx=0).stop 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 shutdown_connections 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x558041eb9d90 0x558041ebc1b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x558041e4f060 0x558041e5fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x558041e50800 0x558041e5c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 --2- 172.21.15.49:0/1642654142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x558041e4f9c0 0x558041e600d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 >> 172.21.15.49:0/1642654142 conn(0x558041dc3260 msgr2=0x558041e49d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd7057fa700 0 client.17025 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 shutdown_connections 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.649+0000 7fd72fe95b80 1 -- 172.21.15.49:0/1642654142 wait complete. 2023-10-08T09:27:51.656 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.650+0000 7fd72fe95b80 20 client.17025 trim_cache size 0 max 16384 2023-10-08T09:27:51.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.654+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd7080a1550 msgr2=0x7fd7080a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.654+0000 7fd723cac700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd7080a1550 0x7fd7080a3a10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd7140076c0 tx=0x7fd714007650 comp rx=0 tx=0).stop 2023-10-08T09:27:51.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.654+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1019b0 msgr2=0x7fd71c19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:27:51.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.654+0000 7fd723cac700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1019b0 0x7fd71c19d3b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00e9c0 tx=0x7fd70c00ed80 comp rx=0 tx=0).stop 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.656+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 shutdown_connections 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.656+0000 7fd723cac700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd7080a1550 0x7fd7080a3a10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.656+0000 7fd723cac700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd71c1007c0 0x7fd71c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.656+0000 7fd723cac700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd71c102350 0x7fd71c1a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.656+0000 7fd723cac700 1 --2- 172.21.15.49:0/2193658228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd71c1019b0 0x7fd71c19d3b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.656+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 >> 172.21.15.49:0/2193658228 conn(0x7fd71c0fbd30 msgr2=0x7fd71c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.659+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 shutdown_connections 2023-10-08T09:27:51.661 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:27:51.659+0000 7fd723cac700 1 -- 172.21.15.49:0/2193658228 wait complete. 2023-10-08T09:27:51.679 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:27:52.141 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:51 smithi049 ceph-mon[99485]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 543 op/s 2023-10-08T09:27:52.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:51 smithi184 ceph-mon[103675]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 543 op/s 2023-10-08T09:27:52.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:51 smithi132 ceph-mon[103389]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.3 MiB/s wr, 543 op/s 2023-10-08T09:27:53.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:53 smithi049 ceph-mon[99485]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 535 op/s 2023-10-08T09:27:53.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:53 smithi184 ceph-mon[103675]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 535 op/s 2023-10-08T09:27:53.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:53 smithi132 ceph-mon[103389]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.7 MiB/s wr, 535 op/s 2023-10-08T09:27:54.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 750.0 s, 544.8 tps, lat 180.077 ms stddev 195.678 2023-10-08T09:27:56.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:55 smithi049 ceph-mon[99485]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2023-10-08T09:27:56.255 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:55 smithi184 ceph-mon[103675]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2023-10-08T09:27:56.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:55 smithi132 ceph-mon[103389]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2023-10-08T09:27:57.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:56 smithi184 ceph-mon[103675]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 535 op/s 2023-10-08T09:27:57.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:56 smithi132 ceph-mon[103389]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 535 op/s 2023-10-08T09:27:57.391 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:56 smithi049 ceph-mon[99485]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 535 op/s 2023-10-08T09:27:59.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 755.0 s, 472.9 tps, lat 207.693 ms stddev 217.976 2023-10-08T09:28:00.138 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:27:59 smithi049 ceph-mon[99485]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.3 MiB/s wr, 506 op/s 2023-10-08T09:28:00.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:27:59 smithi184 ceph-mon[103675]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.3 MiB/s wr, 506 op/s 2023-10-08T09:28:00.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:27:59 smithi132 ceph-mon[103389]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.3 MiB/s wr, 506 op/s 2023-10-08T09:28:01.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:01 smithi049 ceph-mon[99485]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.3 MiB/s wr, 553 op/s 2023-10-08T09:28:02.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:01 smithi184 ceph-mon[103675]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.3 MiB/s wr, 553 op/s 2023-10-08T09:28:02.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:01 smithi132 ceph-mon[103389]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.3 MiB/s wr, 553 op/s 2023-10-08T09:28:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.382+0000 7f7e3911c700 1 -- 172.21.15.49:0/800917565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410a160 msgr2=0x7f7e3410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.382+0000 7f7e3911c700 1 --2- 172.21.15.49:0/800917565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410a160 0x7f7e3410c5f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c01c0d0 tx=0x7f7e2c01c3e0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.386+0000 7f7e3911c700 1 -- 172.21.15.49:0/800917565 shutdown_connections 2023-10-08T09:28:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.386+0000 7f7e3911c700 1 --2- 172.21.15.49:0/800917565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e3410cb30 0x7f7e3410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.386+0000 7f7e3911c700 1 --2- 172.21.15.49:0/800917565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e341077f0 0x7f7e34109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.386+0000 7f7e3911c700 1 --2- 172.21.15.49:0/800917565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410a160 0x7f7e3410c5f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.386+0000 7f7e3911c700 1 -- 172.21.15.49:0/800917565 >> 172.21.15.49:0/800917565 conn(0x7f7e3406d770 msgr2=0x7f7e3406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.394+0000 7f7e3911c700 1 -- 172.21.15.49:0/800917565 shutdown_connections 2023-10-08T09:28:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.394+0000 7f7e3911c700 1 -- 172.21.15.49:0/800917565 wait complete. 2023-10-08T09:28:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.396+0000 7f7e3911c700 1 Processor -- start 2023-10-08T09:28:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3911c700 1 -- start start 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3911c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e341077f0 0x7f7e3419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3911c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3410a160 0x7f7e3419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3911c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410cb30 0x7f7e341a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3911c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e341a48e0 con 0x7f7e341077f0 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3911c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e341a4a50 con 0x7f7e3410a160 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3911c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e341a4c10 con 0x7f7e3410cb30 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e341077f0 0x7f7e3419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e341077f0 0x7f7e3419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53724/0 (socket says 172.21.15.49:53724) 2023-10-08T09:28:02.399 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2094784105 learned_addr learned my addr 172.21.15.49:0/2094784105 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3410a160 0x7f7e3419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.397+0000 7f7e3359e700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410cb30 0x7f7e341a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2094784105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410cb30 msgr2=0x7f7e341a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410cb30 0x7f7e341a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2094784105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3410a160 msgr2=0x7f7e3419d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3410a160 0x7f7e3419d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2094784105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c01ad10 con 0x7f7e341077f0 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e341077f0 0x7f7e3419cdf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800b550 tx=0x7f7e2800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e1bfff700 1 -- 172.21.15.49:0/2094784105 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28011830 con 0x7f7e341077f0 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e341a4ef0 con 0x7f7e341077f0 2023-10-08T09:28:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e341a52b0 con 0x7f7e341077f0 2023-10-08T09:28:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e1bfff700 1 -- 172.21.15.49:0/2094784105 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e28009340 con 0x7f7e341077f0 2023-10-08T09:28:02.401 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.398+0000 7f7e1bfff700 1 -- 172.21.15.49:0/2094784105 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2801ade0 con 0x7f7e341077f0 2023-10-08T09:28:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.401+0000 7f7e1bfff700 1 -- 172.21.15.49:0/2094784105 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7e280193b0 con 0x7f7e341077f0 2023-10-08T09:28:02.402 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.402+0000 7f7e1bfff700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7e1c0a1550 0x7f7e1c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.403 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.402+0000 7f7e1bfff700 1 -- 172.21.15.49:0/2094784105 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7e280cc970 con 0x7f7e341077f0 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7e1c0a1550 0x7f7e1c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 Processor -- start 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 -- start start 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed794e400 con 0x55bed79e72b0 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed793fbc0 con 0x55bed79e67a0 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e45305b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed793d4e0 con 0x55bed79e5e40 2023-10-08T09:28:02.405 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7e1c0a1550 0x7f7e1c0a3a10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000f80 tx=0x7f7e2c018e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e3359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e3359e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79e7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53734/0 (socket says 172.21.15.49:53734) 2023-10-08T09:28:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.403+0000 7f7e3359e700 1 -- 172.21.15.49:0/353645611 learned_addr learned my addr 172.21.15.49:0/353645611 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e3259c700 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e32d9d700 1 -- 172.21.15.49:0/353645611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 msgr2=0x55bed79e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.406 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e32d9d700 1 -- 172.21.15.49:0/353645611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 msgr2=0x55bed79e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e32d9d700 1 -- 172.21.15.49:0/353645611 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bed7763d80 con 0x55bed79e5e40 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e3259c700 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e3359e700 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79e6260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7e280ca190 tx=0x7f7e28095dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0588b23a8ce562 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e197fa700 1 -- 172.21.15.49:0/353645611 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2809dab0 con 0x55bed79e5e40 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e197fa700 1 -- 172.21.15.49:0/353645611 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e280a1cf0 con 0x55bed79e5e40 2023-10-08T09:28:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.404+0000 7f7e197fa700 1 -- 172.21.15.49:0/353645611 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280990b0 con 0x55bed79e5e40 2023-10-08T09:28:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 -- 172.21.15.49:0/353645611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 msgr2=0x55bed79e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.409 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79e6260 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7e280ca190 tx=0x7f7e28095dd0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 -- 172.21.15.49:0/353645611 shutdown_connections 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79e6260 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 --2- 172.21.15.49:0/353645611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 -- 172.21.15.49:0/353645611 >> 172.21.15.49:0/353645611 conn(0x55bed795a200 msgr2=0x55bed79e08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 -- 172.21.15.49:0/353645611 shutdown_connections 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.405+0000 7f7e45305b80 1 -- 172.21.15.49:0/353645611 wait complete. 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 -- start start 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 Processor -- start 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e5e40 0x55bed79ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.410 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 0x55bed79f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed7938dd0 con 0x55bed79e5e40 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed78d56b0 con 0x55bed79e67a0 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed78d5350 con 0x55bed79e72b0 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e45305b80 10 client.0 _ll_register_callbacks cb 0x55bed77327a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e5e40 0x55bed79ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e5e40 0x55bed79ef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53738/0 (socket says 172.21.15.49:53738) 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e32d9d700 1 -- 172.21.15.49:0/138486449 learned_addr learned my addr 172.21.15.49:0/138486449 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.406+0000 7f7e3359e700 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 0x55bed79f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.407+0000 7f7e32d9d700 1 -- 172.21.15.49:0/138486449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 msgr2=0x55bed79f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.407+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 0x55bed79f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.407+0000 7f7e32d9d700 1 -- 172.21.15.49:0/138486449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 msgr2=0x55bed79efdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.407+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.407+0000 7f7e32d9d700 1 -- 172.21.15.49:0/138486449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e24009680 con 0x55bed79e5e40 2023-10-08T09:28:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.407+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e5e40 0x55bed79ef890 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7e280ca4b0 tx=0x7f7e2809dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871b8bba2678e7e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:28:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e16ff5700 1 -- 172.21.15.49:0/138486449 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28097b20 con 0x55bed79e5e40 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e16ff5700 1 -- 172.21.15.49:0/138486449 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7e28098e50 con 0x55bed79e5e40 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bed7900e50 con 0x55bed79e5e40 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e16ff5700 1 -- 172.21.15.49:0/138486449 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e28099010 con 0x55bed79e5e40 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e45305b80 10 client.17034 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f7e341a52b0 con 0x55bed79e5e40 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.411+0000 7f7e16ff5700 1 -- 172.21.15.49:0/138486449 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bed7900e50 con 0x55bed79e5e40 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e16ff5700 1 -- 172.21.15.49:0/138486449 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7e28095750 con 0x55bed79e5e40 2023-10-08T09:28:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e45305b80 10 client.17034 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e45305b80 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e45305b80 20 client.17034 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e45305b80 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed79fb3b0 0x55bed79fd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e45305b80 4 client.17034 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:28:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bed7931db0 con 0x55bed79fb3b0 2023-10-08T09:28:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.412+0000 7f7e3259c700 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed79fb3b0 0x55bed79fd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.413+0000 7f7e3259c700 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed79fb3b0 0x55bed79fd7d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92998323ee7db653 server_cookie=11e7044263f596bd in_seq=0 out_seq=0 2023-10-08T09:28:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.413+0000 7f7e16ff5700 10 client.17034 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.414+0000 7f7e16ff5700 1 -- 172.21.15.49:0/138486449 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bed7931db0 con 0x55bed79fb3b0 2023-10-08T09:28:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.414+0000 7f7e16ff5700 10 client.17034 handle_command_reply: tid=0 2023-10-08T09:28:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.414+0000 7f7e45305b80 1 client.17034 shutdown 2023-10-08T09:28:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.415+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e5e40 msgr2=0x55bed79ef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.415+0000 7f7e45305b80 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e5e40 0x55bed79ef890 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7e280ca4b0 tx=0x7f7e2809dab0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.416+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 shutdown_connections 2023-10-08T09:28:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.416+0000 7f7e45305b80 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed79fb3b0 0x55bed79fd7d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.416+0000 7f7e45305b80 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 0x55bed79f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.416+0000 7f7e45305b80 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e5e40 0x55bed79ef890 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.416+0000 7f7e45305b80 1 --2- 172.21.15.49:0/138486449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79efdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.416+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 >> 172.21.15.49:0/138486449 conn(0x55bed795a200 msgr2=0x55bed79e0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.416+0000 7f7e16ff5700 0 client.17034 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.421+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 shutdown_connections 2023-10-08T09:28:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.421+0000 7f7e45305b80 1 -- 172.21.15.49:0/138486449 wait complete. 2023-10-08T09:28:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.421+0000 7f7e45305b80 20 client.17034 trim_cache size 0 max 16384 2023-10-08T09:28:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.442+0000 7f7e45305b80 1 Processor -- start 2023-10-08T09:28:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.442+0000 7f7e45305b80 1 -- start start 2023-10-08T09:28:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e5e40 0x55bed79f7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e67a0 0x55bed79f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 0x55bed79eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e45305b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed77b98c0 con 0x55bed79e67a0 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e45305b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed7900e80 con 0x55bed79e5e40 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e45305b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed79e1770 con 0x55bed79e72b0 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e67a0 0x55bed79f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e67a0 0x55bed79f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53748/0 (socket says 172.21.15.49:53748) 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 -- 172.21.15.49:0/2444404256 learned_addr learned my addr 172.21.15.49:0/2444404256 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e5e40 0x55bed79f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 -- 172.21.15.49:0/2444404256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 msgr2=0x55bed79eeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 0x55bed79eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 -- 172.21.15.49:0/2444404256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e5e40 msgr2=0x55bed79f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e5e40 0x55bed79f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e3259c700 1 -- 172.21.15.49:0/2444404256 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bed774cc00 con 0x55bed79e67a0 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.443+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e5e40 0x55bed79f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.444+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e67a0 0x55bed79f6d00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c018fd0 tx=0x7f7e2c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c77ae2158d8f7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.444+0000 7f7e19ffb700 1 -- 172.21.15.49:0/2444404256 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c01f040 con 0x55bed79e67a0 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.444+0000 7f7e19ffb700 1 -- 172.21.15.49:0/2444404256 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e2c03edd0 con 0x55bed79e67a0 2023-10-08T09:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.444+0000 7f7e19ffb700 1 -- 172.21.15.49:0/2444404256 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c02e6d0 con 0x55bed79e67a0 2023-10-08T09:28:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 -- 172.21.15.49:0/2444404256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e67a0 msgr2=0x55bed79f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e67a0 0x55bed79f6d00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c018fd0 tx=0x7f7e2c01c3e0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 -- 172.21.15.49:0/2444404256 shutdown_connections 2023-10-08T09:28:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e72b0 0x55bed79eeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e5e40 0x55bed79f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2444404256 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e67a0 0x55bed79f6d00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 -- 172.21.15.49:0/2444404256 >> 172.21.15.49:0/2444404256 conn(0x55bed795a200 msgr2=0x55bed79dcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 -- 172.21.15.49:0/2444404256 shutdown_connections 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 -- 172.21.15.49:0/2444404256 wait complete. 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 -- start start 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.445+0000 7f7e45305b80 1 Processor -- start 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e45305b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e45305b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed79f6500 con 0x55bed79e72b0 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e45305b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed79f6670 con 0x55bed79e67a0 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e45305b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bed79f6830 con 0x55bed79e5e40 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e45305b80 10 client.0 _ll_register_callbacks cb 0x7f7e24009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79f4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40786/0 (socket says 172.21.15.49:40786) 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2353020704 learned_addr learned my addr 172.21.15.49:0/2353020704 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e3359e700 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2353020704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 msgr2=0x55bed79f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2353020704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 msgr2=0x55bed79f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 -- 172.21.15.49:0/2353020704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c033040 con 0x55bed79e5e40 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e3359e700 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e32d9d700 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79f4b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e28095c20 tx=0x7f7e2809dbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e08acdcf6f74234b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e16ff5700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:28:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.446+0000 7f7e16ff5700 1 -- 172.21.15.49:0/2353020704 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280b2240 con 0x55bed79e5e40 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e280a1cf0 con 0x55bed79e5e40 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e16ff5700 1 -- 172.21.15.49:0/2353020704 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7e280840b0 con 0x55bed79e5e40 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e16ff5700 1 -- 172.21.15.49:0/2353020704 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e28099730 con 0x55bed79e5e40 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 10 client.26662 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55bed778a1f0 con 0x55bed79e5e40 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e16ff5700 1 -- 172.21.15.49:0/2353020704 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e280a1cf0 con 0x55bed79e5e40 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e16ff5700 1 -- 172.21.15.49:0/2353020704 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7e28098990 con 0x55bed79e5e40 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 10 client.26662 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 10 client.26662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 20 client.26662 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed7a4fca0 0x55bed7a520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 4 client.26662 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:28:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.447+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bed7931ad0 con 0x55bed7a4fca0 2023-10-08T09:28:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.448+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed7a4fca0 0x55bed7a520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.449+0000 7f7e3259c700 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed7a4fca0 0x55bed7a520b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe22f8c8db6e9beb server_cookie=e76f5db463283335 in_seq=0 out_seq=0 2023-10-08T09:28:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.449+0000 7f7e16ff5700 10 client.26662 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e16ff5700 1 -- 172.21.15.49:0/2353020704 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bed7931ad0 con 0x55bed7a4fca0 2023-10-08T09:28:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e16ff5700 10 client.26662 handle_command_reply: tid=0 2023-10-08T09:28:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 client.26662 shutdown 2023-10-08T09:28:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 msgr2=0x55bed79f4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79f4b10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e28095c20 tx=0x7f7e2809dbd0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 shutdown_connections 2023-10-08T09:28:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55bed7a4fca0 0x55bed7a520b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55bed79e5e40 0x55bed79f4b10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55bed79e72b0 0x55bed79f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 --2- 172.21.15.49:0/2353020704 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55bed79e67a0 0x55bed79f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 >> 172.21.15.49:0/2353020704 conn(0x55bed795a200 msgr2=0x55bed79eeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e16ff5700 0 client.26662 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 shutdown_connections 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 1 -- 172.21.15.49:0/2353020704 wait complete. 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.463+0000 7f7e45305b80 20 client.26662 trim_cache size 0 max 16384 2023-10-08T09:28:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:28:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:28:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:28:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:28:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7e1c0a1550 msgr2=0x7f7e1c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7e1c0a1550 0x7f7e1c0a3a10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c000f80 tx=0x7f7e2c018e70 comp rx=0 tx=0).stop 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e341077f0 msgr2=0x7f7e3419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e341077f0 0x7f7e3419cdf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800b550 tx=0x7f7e2800b860 comp rx=0 tx=0).stop 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 shutdown_connections 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7e1c0a1550 0x7f7e1c0a3a10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e3410cb30 0x7f7e341a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7e341077f0 0x7f7e3419cdf0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 --2- 172.21.15.49:0/2094784105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e3410a160 0x7f7e3419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 >> 172.21.15.49:0/2094784105 conn(0x7f7e3406d770 msgr2=0x7f7e3410a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 shutdown_connections 2023-10-08T09:28:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:02.464+0000 7f7e3911c700 1 -- 172.21.15.49:0/2094784105 wait complete. 2023-10-08T09:28:03.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:03 smithi184 ceph-mon[103675]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.5 MiB/s wr, 532 op/s 2023-10-08T09:28:03.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:03 smithi132 ceph-mon[103389]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.5 MiB/s wr, 532 op/s 2023-10-08T09:28:03.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:03 smithi049 ceph-mon[99485]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.5 MiB/s wr, 532 op/s 2023-10-08T09:28:04.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 760.0 s, 495.9 tps, lat 205.383 ms stddev 234.822 2023-10-08T09:28:06.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:05 smithi049 ceph-mon[99485]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2023-10-08T09:28:06.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:05 smithi184 ceph-mon[103675]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2023-10-08T09:28:06.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:05 smithi132 ceph-mon[103389]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 497 op/s 2023-10-08T09:28:06.880 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:06 smithi049 ceph-mon[99485]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2023-10-08T09:28:07.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:06 smithi184 ceph-mon[103675]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2023-10-08T09:28:07.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:06 smithi132 ceph-mon[103389]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.3 MiB/s wr, 533 op/s 2023-10-08T09:28:09.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:08 smithi049 ceph-mon[99485]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2023-10-08T09:28:09.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:08 smithi184 ceph-mon[103675]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2023-10-08T09:28:09.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:08 smithi132 ceph-mon[103389]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2023-10-08T09:28:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 765.0 s, 502.9 tps, lat 200.044 ms stddev 239.005 2023-10-08T09:28:11.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:10 smithi184 ceph-mon[103675]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 515 op/s 2023-10-08T09:28:11.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:10 smithi132 ceph-mon[103389]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 515 op/s 2023-10-08T09:28:11.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:10 smithi049 ceph-mon[99485]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 515 op/s 2023-10-08T09:28:13.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:13 smithi049 ceph-mon[99485]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.2 MiB/s wr, 502 op/s 2023-10-08T09:28:13.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:13 smithi184 ceph-mon[103675]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.2 MiB/s wr, 502 op/s 2023-10-08T09:28:13.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:13 smithi132 ceph-mon[103389]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 5.2 MiB/s wr, 502 op/s 2023-10-08T09:28:14.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 770.0 s, 495.1 tps, lat 202.164 ms stddev 239.524 2023-10-08T09:28:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.645+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2984268918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0106d70 msgr2=0x7f3fc0107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.645+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2984268918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0106d70 0x7f3fc0107190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00099c0 tx=0x7f3fb0009cd0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.646+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2984268918 shutdown_connections 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.646+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2984268918 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0108900 0x7f3fc010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.646+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2984268918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0106d70 0x7f3fc0107190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.646+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2984268918 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 0x7f3fc01083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.646+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2984268918 >> 172.21.15.49:0/2984268918 conn(0x7f3fc0075d10 msgr2=0x7f3fc0078150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.646+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2984268918 shutdown_connections 2023-10-08T09:28:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.646+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2984268918 wait complete. 2023-10-08T09:28:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 Processor -- start 2023-10-08T09:28:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 -- start start 2023-10-08T09:28:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0106d70 0x7f3fc019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 0x7f3fc019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0108900 0x7f3fc01a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01a47f0 con 0x7f3fc0106d70 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01a4960 con 0x7f3fc0108900 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.647+0000 7f3fc5b82700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01a4b20 con 0x7f3fc0107f60 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 0x7f3fc019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0106d70 0x7f3fc019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 0x7f3fc019d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40800/0 (socket says 172.21.15.49:40800) 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0106d70 0x7f3fc019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53764/0 (socket says 172.21.15.49:53764) 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0108900 0x7f3fc01a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 -- 172.21.15.49:0/2078621066 learned_addr learned my addr 172.21.15.49:0/2078621066 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 -- 172.21.15.49:0/2078621066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0108900 msgr2=0x7f3fc01a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0108900 0x7f3fc01a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 -- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0106d70 msgr2=0x7f3fc019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0106d70 0x7f3fc019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 -- 172.21.15.49:0/2078621066 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00095f0 con 0x7f3fc0107f60 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0106d70 0x7f3fc019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 0x7f3fc019d210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e9a0 tx=0x7f3fb400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbcff9700 1 -- 172.21.15.49:0/2078621066 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4009790 con 0x7f3fc0107f60 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbcff9700 1 -- 172.21.15.49:0/2078621066 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb4010430 con 0x7f3fc0107f60 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc01a4e00 con 0x7f3fc0107f60 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fbcff9700 1 -- 172.21.15.49:0/2078621066 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4018680 con 0x7f3fc0107f60 2023-10-08T09:28:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.648+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc01a5280 con 0x7f3fc0107f60 2023-10-08T09:28:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.651+0000 7f3fbcff9700 1 -- 172.21.15.49:0/2078621066 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3fb40105a0 con 0x7f3fc0107f60 2023-10-08T09:28:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.651+0000 7f3fbcff9700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3fa80a1480 0x7f3fa80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.652+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3fa80a1480 0x7f3fa80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.652+0000 7f3fbcff9700 1 -- 172.21.15.49:0/2078621066 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3fb4014070 con 0x7f3fc0107f60 2023-10-08T09:28:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.652+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3fa80a1480 0x7f3fa80a3940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0003730 tx=0x7f3fb001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.652+0000 7f3fd1d6bb80 1 Processor -- start 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fd1d6bb80 1 -- start start 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd2e40 0x556673fd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673fd3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd42b0 0x556673fd4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673f3b400 con 0x556673fd37a0 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673f2cbc0 con 0x556673fd42b0 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673f2a4e0 con 0x556673fd2e40 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd2e40 0x556673fd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd42b0 0x556673fd4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd2e40 0x556673fd3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40806/0 (socket says 172.21.15.49:40806) 2023-10-08T09:28:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbf7fe700 1 -- 172.21.15.49:0/3069868724 learned_addr learned my addr 172.21.15.49:0/3069868724 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbffff700 1 -- 172.21.15.49:0/3069868724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd2e40 msgr2=0x556673fd3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbffff700 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd2e40 0x556673fd3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbffff700 1 -- 172.21.15.49:0/3069868724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 msgr2=0x556673fd3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbffff700 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673fd3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbffff700 1 -- 172.21.15.49:0/3069868724 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556673d50d80 con 0x556673fd42b0 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd2e40 0x556673fd3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.653+0000 7f3fbffff700 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd42b0 0x556673fd4730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8011580 tx=0x7f3fb8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b502de94cabd5cef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.654+0000 7f3fa5ffb700 1 -- 172.21.15.49:0/3069868724 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800e040 con 0x556673fd42b0 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.654+0000 7f3fa5ffb700 1 -- 172.21.15.49:0/3069868724 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb800f040 con 0x556673fd42b0 2023-10-08T09:28:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.654+0000 7f3fa5ffb700 1 -- 172.21.15.49:0/3069868724 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8004430 con 0x556673fd42b0 2023-10-08T09:28:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3069868724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd42b0 msgr2=0x556673fd4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd42b0 0x556673fd4730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8011580 tx=0x7f3fb8009920 comp rx=0 tx=0).stop 2023-10-08T09:28:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3069868724 shutdown_connections 2023-10-08T09:28:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd2e40 0x556673fd3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd42b0 0x556673fd4730 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3069868724 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673fd3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3069868724 >> 172.21.15.49:0/3069868724 conn(0x556673f47200 msgr2=0x556673fcd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3069868724 shutdown_connections 2023-10-08T09:28:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3069868724 wait complete. 2023-10-08T09:28:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 -- start start 2023-10-08T09:28:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.655+0000 7f3fd1d6bb80 1 Processor -- start 2023-10-08T09:28:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673fed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd37a0 0x556673fedde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd42b0 0x556673ff4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673f25dd0 con 0x556673fd42b0 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673ec26b0 con 0x556673fd2e40 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673ec2350 con 0x556673fd37a0 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fd1d6bb80 10 client.0 _ll_register_callbacks cb 0x556673d1f7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673fed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673fed8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43392/0 (socket says 172.21.15.49:43392) 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 -- 172.21.15.49:0/3829532191 learned_addr learned my addr 172.21.15.49:0/3829532191 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbffff700 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd42b0 0x556673ff4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 -- 172.21.15.49:0/3829532191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd37a0 msgr2=0x556673fedde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:28:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd37a0 0x556673fedde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 -- 172.21.15.49:0/3829532191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd42b0 msgr2=0x556673ff4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd42b0 0x556673ff4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.656+0000 7f3fbf7fe700 1 -- 172.21.15.49:0/3829532191 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc010fb00 con 0x556673fd2e40 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673fed8a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009960 tx=0x7f3fb0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857c72b9c4b06a96 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3f9effd700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3f9effd700 1 -- 172.21.15.49:0/3829532191 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb002a070 con 0x556673fd2e40 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556673eede50 con 0x556673fd2e40 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3f9effd700 1 -- 172.21.15.49:0/3829532191 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3fb000f460 con 0x556673fd2e40 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3f9effd700 1 -- 172.21.15.49:0/3829532191 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb000f620 con 0x556673fd2e40 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3fd1d6bb80 10 client.26618 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f3fb8011230 con 0x556673fd2e40 2023-10-08T09:28:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3f9effd700 1 -- 172.21.15.49:0/3829532191 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556673eede50 con 0x556673fd2e40 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.657+0000 7f3f9effd700 1 -- 172.21.15.49:0/3829532191 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3fb0024720 con 0x556673fd2e40 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.658+0000 7f3fd1d6bb80 10 client.26618 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.658+0000 7f3fd1d6bb80 10 client.26618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.658+0000 7f3fd1d6bb80 20 client.26618 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.658+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556673fdf650 0x556673fe1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.658+0000 7f3fd1d6bb80 4 client.26618 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.658+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556673f1edb0 con 0x556673fdf650 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.659+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556673fdf650 0x556673fe1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.660+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556673fdf650 0x556673fe1a70 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71076ae3f7d94e17 server_cookie=ef350aad49cfa628 in_seq=0 out_seq=0 2023-10-08T09:28:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.660+0000 7f3f9effd700 10 client.26618 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3f9effd700 1 -- 172.21.15.49:0/3829532191 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556673f1edb0 con 0x556673fdf650 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3f9effd700 10 client.26618 handle_command_reply: tid=0 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 client.26618 shutdown 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 msgr2=0x556673fed8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673fed8a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009960 tx=0x7f3fb0000bc0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 shutdown_connections 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556673fdf650 0x556673fe1a70 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd42b0 0x556673ff4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3f9effd700 0 client.26618 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673fed8a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/3829532191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd37a0 0x556673fedde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 >> 172.21.15.49:0/3829532191 conn(0x556673f47200 msgr2=0x556673fcd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 shutdown_connections 2023-10-08T09:28:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/3829532191 wait complete. 2023-10-08T09:28:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.661+0000 7f3fd1d6bb80 20 client.26618 trim_cache size 0 max 16384 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.680+0000 7f3fd1d6bb80 1 Processor -- start 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.680+0000 7f3fd1d6bb80 1 -- start start 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd2e40 0x556673ff5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd37a0 0x556673ff4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673fd4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673da68c0 con 0x556673fd2e40 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673eede50 con 0x556673fd37a0 2023-10-08T09:28:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673fdb3b0 con 0x556673fd42b0 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd2e40 0x556673ff5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673fd4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbeffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd37a0 0x556673ff4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd2e40 0x556673ff5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53784/0 (socket says 172.21.15.49:53784) 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbf7fe700 1 -- 172.21.15.49:0/1080559207 learned_addr learned my addr 172.21.15.49:0/1080559207 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbeffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd37a0 0x556673ff4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43400/0 (socket says 172.21.15.49:43400) 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673fd4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40822/0 (socket says 172.21.15.49:40822) 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 -- 172.21.15.49:0/1080559207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd37a0 msgr2=0x556673ff4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 --2- 172.21.15.49:0/1080559207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd37a0 0x556673ff4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 -- 172.21.15.49:0/1080559207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd2e40 msgr2=0x556673ff5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 --2- 172.21.15.49:0/1080559207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd2e40 0x556673ff5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 -- 172.21.15.49:0/1080559207 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556673d39c00 con 0x556673fd42b0 2023-10-08T09:28:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.681+0000 7f3fbffff700 1 --2- 172.21.15.49:0/1080559207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673fd4690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009b00 tx=0x7f3fb80092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2cf2cdc73db35f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.682+0000 7f3fa67fc700 1 -- 172.21.15.49:0/1080559207 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800e040 con 0x556673fd42b0 2023-10-08T09:28:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.682+0000 7f3fa67fc700 1 -- 172.21.15.49:0/1080559207 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb8003c00 con 0x556673fd42b0 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.682+0000 7f3fa67fc700 1 -- 172.21.15.49:0/1080559207 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80276c0 con 0x556673fd42b0 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.683+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/1080559207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 msgr2=0x556673fd4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.683+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/1080559207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673fd4690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009b00 tx=0x7f3fb80092e0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.683+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/1080559207 shutdown_connections 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.683+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/1080559207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673fd4690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.683+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/1080559207 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd2e40 0x556673ff5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.683+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/1080559207 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd37a0 0x556673ff4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.683+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/1080559207 >> 172.21.15.49:0/1080559207 conn(0x556673f47200 msgr2=0x556673fe6410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/1080559207 shutdown_connections 2023-10-08T09:28:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/1080559207 wait complete. 2023-10-08T09:28:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 -- start start 2023-10-08T09:28:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 Processor -- start 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673ff29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673ff2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673ff3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673ff4410 con 0x556673fd37a0 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673ff4580 con 0x556673fd2e40 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556673ff4740 con 0x556673fd42b0 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fd1d6bb80 10 client.0 _ll_register_callbacks cb 0x7f3fb00049c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fbeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673ff2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673ff29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fbf7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673ff29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43404/0 (socket says 172.21.15.49:43404) 2023-10-08T09:28:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.685+0000 7f3fbeffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673ff2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53792/0 (socket says 172.21.15.49:53792) 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbf7fe700 1 -- 172.21.15.49:0/525776928 learned_addr learned my addr 172.21.15.49:0/525776928 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbffff700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673ff3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbeffd700 1 -- 172.21.15.49:0/525776928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 msgr2=0x556673ff3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673ff3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbeffd700 1 -- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 msgr2=0x556673ff29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673ff29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbeffd700 1 -- 172.21.15.49:0/525776928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556673d50d80 con 0x556673fd37a0 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673ff29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbffff700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673ff3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3fbeffd700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673ff2f10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3fb409a150 tx=0x7f3fb4010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26248576b6dae22d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3f9effd700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:28:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3f9effd700 1 -- 172.21.15.49:0/525776928 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb409a1c0 con 0x556673fd37a0 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.686+0000 7f3f9effd700 1 -- 172.21.15.49:0/525776928 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3fb409e310 con 0x556673fd37a0 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fb800f040 con 0x556673fd37a0 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3f9effd700 1 -- 172.21.15.49:0/525776928 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb40a2550 con 0x556673fd37a0 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 10 client.17055 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556673d771f0 con 0x556673fd37a0 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3f9effd700 1 -- 172.21.15.49:0/525776928 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb800f040 con 0x556673fd37a0 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3f9effd700 1 -- 172.21.15.49:0/525776928 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3fb40996b0 con 0x556673fd37a0 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 10 client.17055 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 20 client.17055 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55667405f580 0x5566740619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 4 client.17055 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:28:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556673f1ead0 con 0x55667405f580 2023-10-08T09:28:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.687+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55667405f580 0x5566740619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.688+0000 7f3f9effd700 10 client.17055 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.688+0000 7f3fbf7fe700 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55667405f580 0x5566740619a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab6f317b7489e87 server_cookie=3b497ad33d187bc0 in_seq=0 out_seq=0 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3f9effd700 1 -- 172.21.15.49:0/525776928 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556673f1ead0 con 0x55667405f580 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3f9effd700 10 client.17055 handle_command_reply: tid=0 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 client.17055 shutdown 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 msgr2=0x556673ff2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673ff2f10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3fb409a150 tx=0x7f3fb4010f80 comp rx=0 tx=0).stop 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 shutdown_connections 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55667405f580 0x5566740619a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556673fd42b0 0x556673ff3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556673fd2e40 0x556673ff29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 --2- 172.21.15.49:0/525776928 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556673fd37a0 0x556673ff2f10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3f9effd700 0 client.17055 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 >> 172.21.15.49:0/525776928 conn(0x556673f47200 msgr2=0x556673fed650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 shutdown_connections 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 1 -- 172.21.15.49:0/525776928 wait complete. 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.700+0000 7f3fd1d6bb80 20 client.17055 trim_cache size 0 max 16384 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:28:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:28:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:28:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:28:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3fa80a1480 msgr2=0x7f3fa80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3fa80a1480 0x7f3fa80a3940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0003730 tx=0x7f3fb001a040 comp rx=0 tx=0).stop 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 msgr2=0x7f3fc019d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 0x7f3fc019d210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e9a0 tx=0x7f3fb400ecb0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 shutdown_connections 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3fa80a1480 0x7f3fa80a3940 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3fc0108900 0x7f3fc01a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fc0106d70 0x7f3fc019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 --2- 172.21.15.49:0/2078621066 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3fc0107f60 0x7f3fc019d210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 >> 172.21.15.49:0/2078621066 conn(0x7f3fc0075d10 msgr2=0x7f3fc010b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 shutdown_connections 2023-10-08T09:28:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:15.701+0000 7f3fc5b82700 1 -- 172.21.15.49:0/2078621066 wait complete. 2023-10-08T09:28:16.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:15 smithi049 ceph-mon[99485]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.7 MiB/s wr, 481 op/s 2023-10-08T09:28:16.252 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:15 smithi184 ceph-mon[103675]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.7 MiB/s wr, 481 op/s 2023-10-08T09:28:16.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:15 smithi132 ceph-mon[103389]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.7 MiB/s wr, 481 op/s 2023-10-08T09:28:17.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:16 smithi184 ceph-mon[103675]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.9 MiB/s wr, 494 op/s 2023-10-08T09:28:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:16 smithi132 ceph-mon[103389]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.9 MiB/s wr, 494 op/s 2023-10-08T09:28:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:16 smithi049 ceph-mon[99485]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.9 MiB/s wr, 494 op/s 2023-10-08T09:28:19.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 775.0 s, 467.9 tps, lat 213.353 ms stddev 226.481 2023-10-08T09:28:20.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:19 smithi049 ceph-mon[99485]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.7 MiB/s wr, 470 op/s 2023-10-08T09:28:20.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:19 smithi184 ceph-mon[103675]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.7 MiB/s wr, 470 op/s 2023-10-08T09:28:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:19 smithi132 ceph-mon[103389]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.7 MiB/s wr, 470 op/s 2023-10-08T09:28:21.681 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:28:21.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.945+0000 7f53808b6700 1 -- 172.21.15.49:0/3206914891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c10cb30 msgr2=0x7f537c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:21.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.945+0000 7f53808b6700 1 --2- 172.21.15.49:0/3206914891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c10cb30 0x7f537c10cfb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f536c009990 tx=0x7f536c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.958+0000 7f53808b6700 1 -- 172.21.15.49:0/3206914891 shutdown_connections 2023-10-08T09:28:21.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.958+0000 7f53808b6700 1 --2- 172.21.15.49:0/3206914891 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c10cb30 0x7f537c10cfb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.958+0000 7f53808b6700 1 --2- 172.21.15.49:0/3206914891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f537c1077f0 0x7f537c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.958+0000 7f53808b6700 1 --2- 172.21.15.49:0/3206914891 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c10a160 0x7f537c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.958+0000 7f53808b6700 1 -- 172.21.15.49:0/3206914891 >> 172.21.15.49:0/3206914891 conn(0x7f537c06d790 msgr2=0x7f537c06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:21.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.961+0000 7f53808b6700 1 -- 172.21.15.49:0/3206914891 shutdown_connections 2023-10-08T09:28:21.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.967+0000 7f53808b6700 1 -- 172.21.15.49:0/3206914891 wait complete. 2023-10-08T09:28:21.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.967+0000 7f53808b6700 1 Processor -- start 2023-10-08T09:28:21.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f53808b6700 1 -- start start 2023-10-08T09:28:21.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f53808b6700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c1077f0 0x7f537c1a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f53808b6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f537c10a160 0x7f537c1a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f53808b6700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c10cb30 0x7f537c1acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f53808b6700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1ad130 con 0x7f537c1077f0 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f53808b6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1ad270 con 0x7f537c10a160 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f53808b6700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1ad400 con 0x7f537c10cb30 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f537a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c1077f0 0x7f537c1a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f537a59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c1077f0 0x7f537c1a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53808/0 (socket says 172.21.15.49:53808) 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f537ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c10cb30 0x7f537c1acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:21.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.968+0000 7f537a59c700 1 -- 172.21.15.49:0/2575453295 learned_addr learned my addr 172.21.15.49:0/2575453295 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:21.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.969+0000 7f537ad9d700 1 -- 172.21.15.49:0/2575453295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f537c10a160 msgr2=0x7f537c1a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:21.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.969+0000 7f537ad9d700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f537c10a160 0x7f537c1a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.969+0000 7f537ad9d700 1 -- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c1077f0 msgr2=0x7f537c1a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:21.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.969+0000 7f537ad9d700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c1077f0 0x7f537c1a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.969+0000 7f537ad9d700 1 -- 172.21.15.49:0/2575453295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c0095f0 con 0x7f537c10cb30 2023-10-08T09:28:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.974+0000 7f537ad9d700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c10cb30 0x7f537c1acb60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f536c0098e0 tx=0x7f536c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:21.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.981+0000 7f53737fe700 1 -- 172.21.15.49:0/2575453295 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c01d070 con 0x7f537c10cb30 2023-10-08T09:28:21.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.981+0000 7f53737fe700 1 -- 172.21.15.49:0/2575453295 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f536c00f460 con 0x7f537c10cb30 2023-10-08T09:28:21.983 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.981+0000 7f53737fe700 1 -- 172.21.15.49:0/2575453295 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c017650 con 0x7f537c10cb30 2023-10-08T09:28:21.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.981+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c1ad680 con 0x7f537c10cb30 2023-10-08T09:28:21.984 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.981+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1ada50 con 0x7f537c10cb30 2023-10-08T09:28:21.988 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.987+0000 7f538ca9fb80 1 Processor -- start 2023-10-08T09:28:21.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.987+0000 7f538ca9fb80 1 -- start start 2023-10-08T09:28:21.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.988+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075198c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:21.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.988+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075198d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:21.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.988+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x56075198dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:21.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.988+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607518f4460 con 0x56075198c060 2023-10-08T09:28:21.989 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.988+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607518e5c20 con 0x56075198d800 2023-10-08T09:28:21.990 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.988+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607518e3540 con 0x56075198c9c0 2023-10-08T09:28:21.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.989+0000 7f53737fe700 1 -- 172.21.15.49:0/2575453295 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f536c00f890 con 0x7f537c10cb30 2023-10-08T09:28:21.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.990+0000 7f53737fe700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53680a1600 0x7f53680a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:21.992 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.991+0000 7f53737fe700 1 -- 172.21.15.49:0/2575453295 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f536c0ce110 con 0x7f537c10cb30 2023-10-08T09:28:21.993 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.992+0000 7f537ad9d700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53680a1600 0x7f53680a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:21.994 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.992+0000 7f537ad9d700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53680a1600 0x7f53680a3ac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f536c005b40 tx=0x7f536c00fc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:21.996 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.995+0000 7f537a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075198d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:21.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.995+0000 7f537a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075198d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40844/0 (socket says 172.21.15.49:40844) 2023-10-08T09:28:21.997 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.995+0000 7f537a59c700 1 -- 172.21.15.49:0/152612946 learned_addr learned my addr 172.21.15.49:0/152612946 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.995+0000 7f5379d9b700 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x56075198dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f5379d9b700 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075198c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f537a59c700 1 -- 172.21.15.49:0/152612946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 msgr2=0x56075198dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f537a59c700 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x56075198dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f537a59c700 1 -- 172.21.15.49:0/152612946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 msgr2=0x56075198c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f537a59c700 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075198c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f537a59c700 1 -- 172.21.15.49:0/152612946 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560751709de0 con 0x56075198c9c0 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f537a59c700 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075198d270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f537400e2d0 tx=0x7f5374009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4deb41f168c0fad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f5370ff9700 1 -- 172.21.15.49:0/152612946 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400eea0 con 0x56075198c9c0 2023-10-08T09:28:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f5370ff9700 1 -- 172.21.15.49:0/152612946 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537400f040 con 0x56075198c9c0 2023-10-08T09:28:21.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.996+0000 7f5370ff9700 1 -- 172.21.15.49:0/152612946 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53740138a0 con 0x56075198c9c0 2023-10-08T09:28:21.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.997+0000 7f538ca9fb80 1 -- 172.21.15.49:0/152612946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 msgr2=0x56075198d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:21.999 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:21.997+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075198d270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f537400e2d0 tx=0x7f5374009790 comp rx=0 tx=0).stop 2023-10-08T09:28:22.007 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.002+0000 7f538ca9fb80 1 -- 172.21.15.49:0/152612946 shutdown_connections 2023-10-08T09:28:22.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.002+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x56075198dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.002+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075198c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.002+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/152612946 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075198d270 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.008 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.002+0000 7f538ca9fb80 1 -- 172.21.15.49:0/152612946 >> 172.21.15.49:0/152612946 conn(0x560751900260 msgr2=0x560751986e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:22.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.013+0000 7f538ca9fb80 1 -- 172.21.15.49:0/152612946 shutdown_connections 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.015+0000 7f538ca9fb80 1 -- 172.21.15.49:0/152612946 wait complete. 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.016+0000 7f538ca9fb80 1 -- start start 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.016+0000 7f538ca9fb80 1 Processor -- start 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198c060 0x560751995e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x560751996390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198d800 0x56075199d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607518dee30 con 0x56075198d800 2023-10-08T09:28:22.101 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075187b710 con 0x56075198c060 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075187b3b0 con 0x56075198c9c0 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f538ca9fb80 10 client.0 _ll_register_callbacks cb 0x5607516d8800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.020+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198c060 0x560751995e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198c060 0x560751995e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43434/0 (socket says 172.21.15.49:43434) 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 -- 172.21.15.49:0/2804785853 learned_addr learned my addr 172.21.15.49:0/2804785853 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198d800 0x56075199d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 -- 172.21.15.49:0/2804785853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 msgr2=0x560751996390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x560751996390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 -- 172.21.15.49:0/2804785853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198c060 msgr2=0x560751995e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198c060 0x560751995e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.102 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 -- 172.21.15.49:0/2804785853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5374009e40 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5379d9b700 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198d800 0x56075199d3d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5364015f60 tx=0x7f53640069f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63234ef9b9a02e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5361ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5361ffb700 1 -- 172.21.15.49:0/2804785853 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364004020 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f5361ffb700 1 -- 172.21.15.49:0/2804785853 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5364004180 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.021+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607518a6eb0 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f5361ffb700 1 -- 172.21.15.49:0/2804785853 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53640156c0 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 10 client.17067 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5607516f2c60 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f5361ffb700 1 -- 172.21.15.49:0/2804785853 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607518a6eb0 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f5361ffb700 1 -- 172.21.15.49:0/2804785853 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5364006270 con 0x56075198d800 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 10 client.17067 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:22.103 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 20 client.17067 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5607519b2dc0 0x5607519b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 4 client.17067 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.022+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607518d7e10 con 0x5607519b2dc0 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.024+0000 7f537a59c700 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5607519b2dc0 0x5607519b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.025+0000 7f537a59c700 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5607519b2dc0 0x5607519b51e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab426ecc800e16 server_cookie=3660d7c8c66af62e in_seq=0 out_seq=0 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.025+0000 7f5361ffb700 10 client.17067 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.026+0000 7f5361ffb700 1 -- 172.21.15.49:0/2804785853 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5607518d7e10 con 0x5607519b2dc0 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.026+0000 7f5361ffb700 10 client.17067 handle_command_reply: tid=0 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.029+0000 7f538ca9fb80 1 client.17067 shutdown 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.040+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198d800 msgr2=0x56075199d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:22.104 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.040+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198d800 0x56075199d3d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5364015f60 tx=0x7f53640069f0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.044+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 shutdown_connections 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.044+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5607519b2dc0 0x5607519b51e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.044+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198d800 0x56075199d3d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.044+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198c060 0x560751995e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.044+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/2804785853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x560751996390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.044+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 >> 172.21.15.49:0/2804785853 conn(0x560751900260 msgr2=0x5607519864d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.044+0000 7f5361ffb700 0 client.17067 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:22.105 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.045+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 shutdown_connections 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.051+0000 7f538ca9fb80 1 -- 172.21.15.49:0/2804785853 wait complete. 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.051+0000 7f538ca9fb80 20 client.17067 trim_cache size 0 max 16384 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.083+0000 7f538ca9fb80 1 Processor -- start 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.083+0000 7f538ca9fb80 1 -- start start 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.083+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x560751985a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.084+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.084+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x560751995e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.084+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075175f920 con 0x56075198c060 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.084+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075198d060 con 0x56075198d800 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.084+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075198ce30 con 0x56075198c9c0 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.095+0000 7f537a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.106 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.096+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x560751995e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.096+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x560751995e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43444/0 (socket says 172.21.15.49:43444) 2023-10-08T09:28:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.096+0000 7f5379d9b700 1 -- 172.21.15.49:0/1262898370 learned_addr learned my addr 172.21.15.49:0/1262898370 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:22.107 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.096+0000 7f5379d9b700 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x560751985a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.102+0000 7f537a59c700 1 -- 172.21.15.49:0/1262898370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 msgr2=0x560751995e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.102+0000 7f537a59c700 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x560751995e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.102+0000 7f537a59c700 1 -- 172.21.15.49:0/1262898370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 msgr2=0x560751985a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.102+0000 7f537a59c700 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x560751985a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.102+0000 7f537a59c700 1 -- 172.21.15.49:0/1262898370 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560751730250 con 0x56075198c9c0 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.102+0000 7f537a59c700 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199d2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5374009b80 tx=0x7f5374009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0802afb1b0d9fca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.102+0000 7f53717fa700 1 -- 172.21.15.49:0/1262898370 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374024070 con 0x56075198c9c0 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.103+0000 7f53717fa700 1 -- 172.21.15.49:0/1262898370 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537402b070 con 0x56075198c9c0 2023-10-08T09:28:22.108 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.103+0000 7f53717fa700 1 -- 172.21.15.49:0/1262898370 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374027a80 con 0x56075198c9c0 2023-10-08T09:28:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.107+0000 7f538ca9fb80 1 -- 172.21.15.49:0/1262898370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 msgr2=0x56075199d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:22.111 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.107+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199d2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5374009b80 tx=0x7f5374009790 comp rx=0 tx=0).stop 2023-10-08T09:28:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.119+0000 7f538ca9fb80 1 -- 172.21.15.49:0/1262898370 shutdown_connections 2023-10-08T09:28:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.119+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x560751995e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.119+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x560751985a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.119+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/1262898370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199d2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.121 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.119+0000 7f538ca9fb80 1 -- 172.21.15.49:0/1262898370 >> 172.21.15.49:0/1262898370 conn(0x560751900260 msgr2=0x560751995360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:22.126 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.125+0000 7f538ca9fb80 1 -- 172.21.15.49:0/1262898370 shutdown_connections 2023-10-08T09:28:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.125+0000 7f538ca9fb80 1 -- 172.21.15.49:0/1262898370 wait complete. 2023-10-08T09:28:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.125+0000 7f538ca9fb80 1 -- start start 2023-10-08T09:28:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.125+0000 7f538ca9fb80 1 Processor -- start 2023-10-08T09:28:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.125+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075199ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.127 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f538ca9fb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x56075199b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075199d690 con 0x56075198c060 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075199bf00 con 0x56075198d800 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f538ca9fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075199c090 con 0x56075198c9c0 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f538ca9fb80 10 client.0 _ll_register_callbacks cb 0x7f5364015c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075199ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f5379d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075199ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53832/0 (socket says 172.21.15.49:53832) 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f5379d9b700 1 -- 172.21.15.49:0/16808288 learned_addr learned my addr 172.21.15.49:0/16808288 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:22.128 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f537a59c700 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f537a59c700 1 -- 172.21.15.49:0/16808288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 msgr2=0x56075199b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f537a59c700 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x56075199b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f537a59c700 1 -- 172.21.15.49:0/16808288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 msgr2=0x56075199ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f537a59c700 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075199ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.126+0000 7f537a59c700 1 -- 172.21.15.49:0/16808288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560751709de0 con 0x56075198c9c0 2023-10-08T09:28:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.127+0000 7f537a59c700 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199b480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5374009fd0 tx=0x7f5374003aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555e3a4d6513e111 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:22.129 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.127+0000 7f5361ffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:28:22.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.127+0000 7f5361ffb700 1 -- 172.21.15.49:0/16808288 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374024040 con 0x56075198c9c0 2023-10-08T09:28:22.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.127+0000 7f5361ffb700 1 -- 172.21.15.49:0/16808288 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5374027820 con 0x56075198c9c0 2023-10-08T09:28:22.130 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.127+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f537400f040 con 0x56075198c9c0 2023-10-08T09:28:22.136 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:21 smithi049 ceph-mon[99485]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2023-10-08T09:28:22.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.136+0000 7f5361ffb700 1 -- 172.21.15.49:0/16808288 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53740275a0 con 0x56075198c9c0 2023-10-08T09:28:22.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.136+0000 7f538ca9fb80 10 client.26689 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:22.137 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.136+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5607517082b0 con 0x56075198c9c0 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f5361ffb700 1 -- 172.21.15.49:0/16808288 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f537400f040 con 0x56075198c9c0 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f5361ffb700 1 -- 172.21.15.49:0/16808288 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5374014bd0 con 0x56075198c9c0 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f538ca9fb80 10 client.26689 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f538ca9fb80 10 client.26689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f538ca9fb80 20 client.26689 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560751a07ec0 0x560751a0a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f538ca9fb80 4 client.26689 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:28:22.145 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.143+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5607518d7b30 con 0x560751a07ec0 2023-10-08T09:28:22.146 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.145+0000 7f5379d9b700 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560751a07ec0 0x560751a0a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:22.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.150+0000 7f5361ffb700 10 client.26689 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:22.157 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.150+0000 7f5379d9b700 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560751a07ec0 0x560751a0a2e0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30c012d8615f836 server_cookie=d38f4bedfb4a264b in_seq=0 out_seq=0 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.150+0000 7f5361ffb700 1 -- 172.21.15.49:0/16808288 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5607518d7b30 con 0x560751a07ec0 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.150+0000 7f5361ffb700 10 client.26689 handle_command_reply: tid=0 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.150+0000 7f538ca9fb80 1 client.26689 shutdown 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 msgr2=0x56075199b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199b480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5374009fd0 tx=0x7f5374003aa0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 shutdown_connections 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x560751a07ec0 0x560751a0a2e0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56075198d800 0x56075199b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56075198c060 0x56075199ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 --2- 172.21.15.49:0/16808288 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56075198c9c0 0x56075199b480 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.158 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 >> 172.21.15.49:0/16808288 conn(0x560751900260 msgr2=0x560751985a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f5361ffb700 0 client.26689 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 shutdown_connections 2023-10-08T09:28:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 1 -- 172.21.15.49:0/16808288 wait complete. 2023-10-08T09:28:22.159 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.151+0000 7f538ca9fb80 20 client.26689 trim_cache size 0 max 16384 2023-10-08T09:28:22.160 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:28:22.160 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:28:22.160 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:28:22.160 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:28:22.160 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:28:22.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:28:22.160 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:28:22.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:28:22.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:28:22.161 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:28:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53680a1600 msgr2=0x7f53680a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53680a1600 0x7f53680a3ac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f536c005b40 tx=0x7f536c00fc30 comp rx=0 tx=0).stop 2023-10-08T09:28:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c10cb30 msgr2=0x7f537c1acb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:22.162 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c10cb30 0x7f537c1acb60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f536c0098e0 tx=0x7f536c003c10 comp rx=0 tx=0).stop 2023-10-08T09:28:22.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 shutdown_connections 2023-10-08T09:28:22.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f53680a1600 0x7f53680a3ac0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f537c10cb30 0x7f537c1acb60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.163 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.160+0000 7f53808b6700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f537c1077f0 0x7f537c1a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.161+0000 7f53808b6700 1 --2- 172.21.15.49:0/2575453295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f537c10a160 0x7f537c1a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:22.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.161+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 >> 172.21.15.49:0/2575453295 conn(0x7f537c06d790 msgr2=0x7f537c1099e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:22.164 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.161+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 shutdown_connections 2023-10-08T09:28:22.165 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:22.161+0000 7f53808b6700 1 -- 172.21.15.49:0/2575453295 wait complete. 2023-10-08T09:28:22.211 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:28:22.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:21 smithi184 ceph-mon[103675]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2023-10-08T09:28:22.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:21 smithi132 ceph-mon[103389]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2023-10-08T09:28:23.510 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:23 smithi184 ceph-mon[103675]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.9 MiB/s wr, 501 op/s 2023-10-08T09:28:23.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:23 smithi132 ceph-mon[103389]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.9 MiB/s wr, 501 op/s 2023-10-08T09:28:23.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:23 smithi049 ceph-mon[99485]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.9 MiB/s wr, 501 op/s 2023-10-08T09:28:24.317 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 780.0 s, 455.6 tps, lat 220.478 ms stddev 249.460 2023-10-08T09:28:26.135 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:25 smithi049 ceph-mon[99485]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2023-10-08T09:28:26.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:25 smithi184 ceph-mon[103675]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2023-10-08T09:28:26.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:25 smithi132 ceph-mon[103389]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2023-10-08T09:28:28.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:27 smithi049 ceph-mon[99485]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2023-10-08T09:28:28.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:27 smithi184 ceph-mon[103675]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2023-10-08T09:28:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:27 smithi132 ceph-mon[103389]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2023-10-08T09:28:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 -- 172.21.15.49:0/3631885236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69900ff640 msgr2=0x7f69900ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 --2- 172.21.15.49:0/3631885236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69900ff640 0x7f69900ffa60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f69800099c0 tx=0x7f6980009cd0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 -- 172.21.15.49:0/3631885236 shutdown_connections 2023-10-08T09:28:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 --2- 172.21.15.49:0/3631885236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69901011d0 0x7f6990107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 --2- 172.21.15.49:0/3631885236 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69900ff640 0x7f69900ffa60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 --2- 172.21.15.49:0/3631885236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6990100830 0x7f6990100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 -- 172.21.15.49:0/3631885236 >> 172.21.15.49:0/3631885236 conn(0x7f69900fabb0 msgr2=0x7f69900fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 -- 172.21.15.49:0/3631885236 shutdown_connections 2023-10-08T09:28:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.855+0000 7f699664f700 1 -- 172.21.15.49:0/3631885236 wait complete. 2023-10-08T09:28:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 Processor -- start 2023-10-08T09:28:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 -- start start 2023-10-08T09:28:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69900ff640 0x7f6990103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6990100830 0x7f6990101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69901011d0 0x7f6990102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69901028f0 con 0x7f69901011d0 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6990102a60 con 0x7f69900ff640 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f699664f700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69901a0710 con 0x7f6990100830 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f698ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69900ff640 0x7f6990103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f698ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69900ff640 0x7f6990103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43460/0 (socket says 172.21.15.49:43460) 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f698ffff700 1 -- 172.21.15.49:0/2259998000 learned_addr learned my addr 172.21.15.49:0/2259998000 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.856+0000 7f698f7fe700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6990100830 0x7f6990101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698ffff700 1 -- 172.21.15.49:0/2259998000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6990100830 msgr2=0x7f6990101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698ffff700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6990100830 0x7f6990101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698ffff700 1 -- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69901011d0 msgr2=0x7f6990102150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698ffff700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69901011d0 0x7f6990102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698ffff700 1 -- 172.21.15.49:0/2259998000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69800095f0 con 0x7f69900ff640 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698f7fe700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6990100830 0x7f6990101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698ffff700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69900ff640 0x7f6990103580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6980003730 tx=0x7f698000f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698d7fa700 1 -- 172.21.15.49:0/2259998000 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698001d070 con 0x7f69900ff640 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69901a0940 con 0x7f69900ff640 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698d7fa700 1 -- 172.21.15.49:0/2259998000 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69800059a0 con 0x7f69900ff640 2023-10-08T09:28:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f698d7fa700 1 -- 172.21.15.49:0/2259998000 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6980004a00 con 0x7f69900ff640 2023-10-08T09:28:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.857+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69901a0de0 con 0x7f69900ff640 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.860+0000 7f698d7fa700 1 -- 172.21.15.49:0/2259998000 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6980004b60 con 0x7f69900ff640 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.861+0000 7f698d7fa700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69780a1480 0x7f69780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.861+0000 7f698d7fa700 1 -- 172.21.15.49:0/2259998000 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f69800cdc40 con 0x7f69900ff640 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.861+0000 7f698f7fe700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69780a1480 0x7f69780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.861+0000 7f698f7fe700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69780a1480 0x7f69780a3940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6984011490 tx=0x7f6984009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 Processor -- start 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 -- start start 2023-10-08T09:28:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d6361e40 0x5596d6362260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d6362cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6363730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d62ca400 con 0x5596d63632b0 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d62bbbc0 con 0x5596d63627a0 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f69a2838b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d62b94e0 con 0x5596d6361e40 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f698ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d6361e40 0x5596d6362260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f6994bec700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6363730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f698ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d6361e40 0x5596d6362260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40888/0 (socket says 172.21.15.49:40888) 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f698ffff700 1 -- 172.21.15.49:0/3427986147 learned_addr learned my addr 172.21.15.49:0/3427986147 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f698f7fe700 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d6362cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f6994bec700 1 -- 172.21.15.49:0/3427986147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d6361e40 msgr2=0x5596d6362260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f6994bec700 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d6361e40 0x5596d6362260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.862+0000 7f6994bec700 1 -- 172.21.15.49:0/3427986147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 msgr2=0x5596d6362cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f6994bec700 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d6362cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f6994bec700 1 -- 172.21.15.49:0/3427986147 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596d60dfd80 con 0x5596d63632b0 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f698ffff700 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d6361e40 0x5596d6362260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f698f7fe700 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d6362cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f6994bec700 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6363730 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f698800e490 tx=0x7f698800e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d96476c53ff87a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f69767fc700 1 -- 172.21.15.49:0/3427986147 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988004db0 con 0x5596d63632b0 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f69767fc700 1 -- 172.21.15.49:0/3427986147 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f698800f040 con 0x5596d63632b0 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.863+0000 7f69767fc700 1 -- 172.21.15.49:0/3427986147 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988004480 con 0x5596d63632b0 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 -- 172.21.15.49:0/3427986147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 msgr2=0x5596d6363730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6363730 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f698800e490 tx=0x7f698800e7a0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 -- 172.21.15.49:0/3427986147 shutdown_connections 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d6361e40 0x5596d6362260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6363730 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 --2- 172.21.15.49:0/3427986147 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d6362cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 -- 172.21.15.49:0/3427986147 >> 172.21.15.49:0/3427986147 conn(0x5596d62d6200 msgr2=0x5596d635c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 -- 172.21.15.49:0/3427986147 shutdown_connections 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.864+0000 7f69a2838b80 1 -- 172.21.15.49:0/3427986147 wait complete. 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 -- start start 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 Processor -- start 2023-10-08T09:28:28.872 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63627a0 0x5596d636be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6372dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d62b4dd0 con 0x5596d63632b0 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d62516b0 con 0x5596d6361e40 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d6251350 con 0x5596d63627a0 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.865+0000 7f69a2838b80 10 client.0 _ll_register_callbacks cb 0x5596d60ae7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43474/0 (socket says 172.21.15.49:43474) 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698ffff700 1 -- 172.21.15.49:0/1754987148 learned_addr learned my addr 172.21.15.49:0/1754987148 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f6994bec700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6372dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698f7fe700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63627a0 0x5596d636be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698f7fe700 1 -- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 msgr2=0x5596d636b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698f7fe700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698f7fe700 1 -- 172.21.15.49:0/1754987148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 msgr2=0x5596d6372dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698f7fe700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6372dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698f7fe700 1 -- 172.21.15.49:0/1754987148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69901a0de0 con 0x5596d63627a0 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698ffff700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f6994bec700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6372dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f698f7fe700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63627a0 0x5596d636be20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6984003aa0 tx=0x7f69840047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85ce68734d57881 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.866+0000 7f696f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.867+0000 7f696f7fe700 1 -- 172.21.15.49:0/1754987148 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698401fa90 con 0x5596d63627a0 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.867+0000 7f696f7fe700 1 -- 172.21.15.49:0/1754987148 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f698401fbf0 con 0x5596d63627a0 2023-10-08T09:28:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.867+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596d627ce50 con 0x5596d63627a0 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.873+0000 7f696f7fe700 1 -- 172.21.15.49:0/1754987148 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f698401fdb0 con 0x5596d63627a0 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 10 client.26701 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f6984017070 con 0x5596d63627a0 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f696f7fe700 1 -- 172.21.15.49:0/1754987148 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596d627ce50 con 0x5596d63627a0 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f696f7fe700 1 -- 172.21.15.49:0/1754987148 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6984027470 con 0x5596d63627a0 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 10 client.26701 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 10 client.26701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 20 client.26701 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d6377750 0x5596d6379b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 4 client.26701 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:28:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596d62addb0 con 0x5596d6377750 2023-10-08T09:28:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.874+0000 7f698ffff700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d6377750 0x5596d6379b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.877+0000 7f698ffff700 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d6377750 0x5596d6379b70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf03e332349d7e4 server_cookie=e60eb27905974018 in_seq=0 out_seq=0 2023-10-08T09:28:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.877+0000 7f696f7fe700 10 client.26701 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.877+0000 7f696f7fe700 1 -- 172.21.15.49:0/1754987148 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5596d62addb0 con 0x5596d6377750 2023-10-08T09:28:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.877+0000 7f696f7fe700 10 client.26701 handle_command_reply: tid=0 2023-10-08T09:28:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 client.26701 shutdown 2023-10-08T09:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63627a0 msgr2=0x5596d636be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63627a0 0x5596d636be20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6984003aa0 tx=0x7f69840047c0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 shutdown_connections 2023-10-08T09:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d6377750 0x5596d6379b70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63632b0 0x5596d6372dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f696f7fe700 0 client.26701 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1754987148 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63627a0 0x5596d636be20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 >> 172.21.15.49:0/1754987148 conn(0x5596d62d6200 msgr2=0x5596d635c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 shutdown_connections 2023-10-08T09:28:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 1 -- 172.21.15.49:0/1754987148 wait complete. 2023-10-08T09:28:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.878+0000 7f69a2838b80 20 client.26701 trim_cache size 0 max 16384 2023-10-08T09:28:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 Processor -- start 2023-10-08T09:28:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 -- start start 2023-10-08T09:28:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63627a0 0x5596d6373470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6372ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d61358c0 con 0x5596d63627a0 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d6373220 con 0x5596d6361e40 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f69a2838b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d63625e0 con 0x5596d63632b0 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.896+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63627a0 0x5596d6373470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f698ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f698ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43482/0 (socket says 172.21.15.49:43482) 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63627a0 0x5596d6373470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53870/0 (socket says 172.21.15.49:53870) 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f698ffff700 1 -- 172.21.15.49:0/1791447939 learned_addr learned my addr 172.21.15.49:0/1791447939 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6994bec700 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6372ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6994bec700 1 -- 172.21.15.49:0/1791447939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 msgr2=0x5596d636ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6994bec700 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6994bec700 1 -- 172.21.15.49:0/1791447939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63627a0 msgr2=0x5596d6373470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6994bec700 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63627a0 0x5596d6373470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6994bec700 1 -- 172.21.15.49:0/1791447939 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698800e140 con 0x5596d63632b0 2023-10-08T09:28:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f698f7fe700 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63627a0 0x5596d6373470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6994bec700 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6372ce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6988009220 tx=0x7f698800ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f039db3b0fd955 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6976ffd700 1 -- 172.21.15.49:0/1791447939 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988004db0 con 0x5596d63632b0 2023-10-08T09:28:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6976ffd700 1 -- 172.21.15.49:0/1791447939 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6988009300 con 0x5596d63632b0 2023-10-08T09:28:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.897+0000 7f6976ffd700 1 -- 172.21.15.49:0/1791447939 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988004820 con 0x5596d63632b0 2023-10-08T09:28:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.898+0000 7f69a2838b80 1 -- 172.21.15.49:0/1791447939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 msgr2=0x5596d6372ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.898+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6372ce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6988009220 tx=0x7f698800ef50 comp rx=0 tx=0).stop 2023-10-08T09:28:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.898+0000 7f69a2838b80 1 -- 172.21.15.49:0/1791447939 shutdown_connections 2023-10-08T09:28:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.898+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6372ce0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.898+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d6361e40 0x5596d636ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.898+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1791447939 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d63627a0 0x5596d6373470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.898+0000 7f69a2838b80 1 -- 172.21.15.49:0/1791447939 >> 172.21.15.49:0/1791447939 conn(0x5596d62d6200 msgr2=0x5596d637e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.899+0000 7f69a2838b80 1 -- 172.21.15.49:0/1791447939 shutdown_connections 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.899+0000 7f69a2838b80 1 -- 172.21.15.49:0/1791447939 wait complete. 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.899+0000 7f69a2838b80 1 -- start start 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.899+0000 7f69a2838b80 1 Processor -- start 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.899+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d6361e40 0x5596d636e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.900+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d636ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.900+0000 7f69a2838b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6371c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.900+0000 7f69a2838b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d63721a0 con 0x5596d6361e40 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.900+0000 7f69a2838b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d6372310 con 0x5596d63627a0 2023-10-08T09:28:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.900+0000 7f69a2838b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596d63724d0 con 0x5596d63632b0 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.900+0000 7f69a2838b80 10 client.0 _ll_register_callbacks cb 0x7f6984003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.900+0000 7f6994bec700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6371c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6371c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40908/0 (socket says 172.21.15.49:40908) 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 -- 172.21.15.49:0/1883144020 learned_addr learned my addr 172.21.15.49:0/1883144020 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f698f7fe700 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d636ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 -- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 msgr2=0x5596d636ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d636ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 -- 172.21.15.49:0/1883144020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d6361e40 msgr2=0x5596d636e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d6361e40 0x5596d636e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 -- 172.21.15.49:0/1883144020 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6984017070 con 0x5596d63632b0 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f6994bec700 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6371c60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f698800eaf0 tx=0x7f6988004280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baef0603592e5473 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f696f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f696f7fe700 1 -- 172.21.15.49:0/1883144020 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988003c00 con 0x5596d63632b0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f696f7fe700 1 -- 172.21.15.49:0/1883144020 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6988003d60 con 0x5596d63632b0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.901+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f698800f040 con 0x5596d63632b0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f696f7fe700 1 -- 172.21.15.49:0/1883144020 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6988007500 con 0x5596d63632b0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 10 client.26707 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5596d60dfd80 con 0x5596d63632b0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f696f7fe700 1 -- 172.21.15.49:0/1883144020 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f698800f040 con 0x5596d63632b0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f696f7fe700 1 -- 172.21.15.49:0/1883144020 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f6988014530 con 0x5596d63632b0 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 10 client.26707 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 10 client.26707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:28.906 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 20 client.26707 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d63d44a0 0x5596d63d6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 4 client.26707 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:28:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596d62adad0 con 0x5596d63d44a0 2023-10-08T09:28:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.902+0000 7f698ffff700 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d63d44a0 0x5596d63d6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.903+0000 7f696f7fe700 10 client.26707 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:28.907 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.903+0000 7f698ffff700 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d63d44a0 0x5596d63d6890 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4fdc171ab5accbe server_cookie=98a4f0c306e3c97c in_seq=0 out_seq=0 2023-10-08T09:28:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.912+0000 7f696f7fe700 1 -- 172.21.15.49:0/1883144020 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596d62adad0 con 0x5596d63d44a0 2023-10-08T09:28:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.912+0000 7f696f7fe700 10 client.26707 handle_command_reply: tid=0 2023-10-08T09:28:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.912+0000 7f69a2838b80 1 client.26707 shutdown 2023-10-08T09:28:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 msgr2=0x5596d6371c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6371c60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f698800eaf0 tx=0x7f6988004280 comp rx=0 tx=0).stop 2023-10-08T09:28:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 shutdown_connections 2023-10-08T09:28:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5596d63d44a0 0x5596d63d6890 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5596d63632b0 0x5596d6371c60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5596d6361e40 0x5596d636e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f696f7fe700 0 client.26707 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 --2- 172.21.15.49:0/1883144020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5596d63627a0 0x5596d636ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 >> 172.21.15.49:0/1883144020 conn(0x5596d62d6200 msgr2=0x5596d636b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 shutdown_connections 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.916+0000 7f69a2838b80 1 -- 172.21.15.49:0/1883144020 wait complete. 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f69a2838b80 20 client.26707 trim_cache size 0 max 16384 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:28:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:28:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:28:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:28:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69780a1480 msgr2=0x7f69780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69780a1480 0x7f69780a3940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6984011490 tx=0x7f6984009260 comp rx=0 tx=0).stop 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69900ff640 msgr2=0x7f6990103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69900ff640 0x7f6990103580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6980003730 tx=0x7f698000f670 comp rx=0 tx=0).stop 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 shutdown_connections 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f69780a1480 0x7f69780a3940 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f69901011d0 0x7f6990102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69900ff640 0x7f6990103580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 --2- 172.21.15.49:0/2259998000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6990100830 0x7f6990101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 >> 172.21.15.49:0/2259998000 conn(0x7f69900fabb0 msgr2=0x7f6990067af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 shutdown_connections 2023-10-08T09:28:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:28.917+0000 7f699664f700 1 -- 172.21.15.49:0/2259998000 wait complete. 2023-10-08T09:28:29.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:28 smithi049 ceph-mon[99485]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.6 MiB/s wr, 477 op/s 2023-10-08T09:28:29.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:28 smithi184 ceph-mon[103675]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.6 MiB/s wr, 477 op/s 2023-10-08T09:28:29.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:28 smithi132 ceph-mon[103389]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.6 MiB/s wr, 477 op/s 2023-10-08T09:28:29.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 785.0 s, 447.6 tps, lat 222.489 ms stddev 264.466 2023-10-08T09:28:31.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:30 smithi184 ceph-mon[103675]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.7 MiB/s wr, 488 op/s 2023-10-08T09:28:31.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:30 smithi132 ceph-mon[103389]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.7 MiB/s wr, 488 op/s 2023-10-08T09:28:31.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:30 smithi049 ceph-mon[99485]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 4.7 MiB/s wr, 488 op/s 2023-10-08T09:28:33.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:33 smithi184 ceph-mon[103675]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 490 op/s 2023-10-08T09:28:33.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:33 smithi132 ceph-mon[103389]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 490 op/s 2023-10-08T09:28:33.636 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:33 smithi049 ceph-mon[99485]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 490 op/s 2023-10-08T09:28:34.313 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 790.0 s, 506.3 tps, lat 196.986 ms stddev 211.896 2023-10-08T09:28:35.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:34 smithi184 ceph-mon[103675]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2023-10-08T09:28:35.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:34 smithi132 ceph-mon[103389]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2023-10-08T09:28:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:34 smithi049 ceph-mon[99485]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.4 MiB/s wr, 464 op/s 2023-10-08T09:28:38.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:37 smithi049 ceph-mon[99485]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 486 op/s 2023-10-08T09:28:38.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:37 smithi184 ceph-mon[103675]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 486 op/s 2023-10-08T09:28:38.302 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:37 smithi132 ceph-mon[103389]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 486 op/s 2023-10-08T09:28:39.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:38 smithi184 ceph-mon[103675]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 472 op/s 2023-10-08T09:28:39.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:38 smithi132 ceph-mon[103389]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 472 op/s 2023-10-08T09:28:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 795.0 s, 494.2 tps, lat 201.491 ms stddev 238.330 2023-10-08T09:28:39.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:38 smithi049 ceph-mon[99485]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 472 op/s 2023-10-08T09:28:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.160+0000 7fec68a8c700 1 -- 172.21.15.49:0/1593331578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 msgr2=0x7fec640ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.160+0000 7fec68a8c700 1 --2- 172.21.15.49:0/1593331578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 0x7fec640ffa90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec5400dda0 tx=0x7fec540074a0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.161+0000 7fec68a8c700 1 -- 172.21.15.49:0/1593331578 shutdown_connections 2023-10-08T09:28:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.161+0000 7fec68a8c700 1 --2- 172.21.15.49:0/1593331578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec64101200 0x7fec64068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.161+0000 7fec68a8c700 1 --2- 172.21.15.49:0/1593331578 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 0x7fec640ffa90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.161+0000 7fec68a8c700 1 --2- 172.21.15.49:0/1593331578 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 0x7fec64100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.161+0000 7fec68a8c700 1 -- 172.21.15.49:0/1593331578 >> 172.21.15.49:0/1593331578 conn(0x7fec640fac00 msgr2=0x7fec640fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.164+0000 7fec68a8c700 1 -- 172.21.15.49:0/1593331578 shutdown_connections 2023-10-08T09:28:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.164+0000 7fec68a8c700 1 -- 172.21.15.49:0/1593331578 wait complete. 2023-10-08T09:28:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 Processor -- start 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 -- start start 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 0x7fec64194570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 0x7fec64194ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec64101200 0x7fec6419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6419c0a0 con 0x7fec64100860 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec641ad1f0 con 0x7fec640ff670 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec68a8c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec641ad380 con 0x7fec64101200 2023-10-08T09:28:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec61d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 0x7fec64194ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec61d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 0x7fec64194ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53896/0 (socket says 172.21.15.49:53896) 2023-10-08T09:28:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.165+0000 7fec61d9b700 1 -- 172.21.15.49:0/3966402465 learned_addr learned my addr 172.21.15.49:0/3966402465 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.166+0000 7fec6259c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 0x7fec64194570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.166+0000 7fec61d9b700 1 -- 172.21.15.49:0/3966402465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec64101200 msgr2=0x7fec6419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.166+0000 7fec61d9b700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec64101200 0x7fec6419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.166+0000 7fec61d9b700 1 -- 172.21.15.49:0/3966402465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 msgr2=0x7fec64194570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.166+0000 7fec61d9b700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 0x7fec64194570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.166+0000 7fec61d9b700 1 -- 172.21.15.49:0/3966402465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5400da00 con 0x7fec64100860 2023-10-08T09:28:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.166+0000 7fec6259c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 0x7fec64194570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:28:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.172+0000 7fec61d9b700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 0x7fec64194ab0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fec5800b530 tx=0x7fec5800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.172+0000 7fec537fe700 1 -- 172.21.15.49:0/3966402465 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58011830 con 0x7fec64100860 2023-10-08T09:28:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.172+0000 7fec537fe700 1 -- 172.21.15.49:0/3966402465 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec58011e50 con 0x7fec64100860 2023-10-08T09:28:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.172+0000 7fec537fe700 1 -- 172.21.15.49:0/3966402465 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58010e40 con 0x7fec64100860 2023-10-08T09:28:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.172+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec641ad660 con 0x7fec64100860 2023-10-08T09:28:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.172+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec641adb20 con 0x7fec64100860 2023-10-08T09:28:42.177 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.176+0000 7fec537fe700 1 -- 172.21.15.49:0/3966402465 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fec58011990 con 0x7fec64100860 2023-10-08T09:28:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec537fe700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fec4c0a1550 0x7fec4c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec537fe700 1 -- 172.21.15.49:0/3966402465 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fec580cca70 con 0x7fec64100860 2023-10-08T09:28:42.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec6259c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fec4c0a1550 0x7fec4c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec74c75b80 1 Processor -- start 2023-10-08T09:28:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec74c75b80 1 -- start start 2023-10-08T09:28:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec6259c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fec4c0a1550 0x7fec4c0a3a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fec5400dcf0 tx=0x7fec5400f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:42.180 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edcbdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.177+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edcbe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec74c75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edc25400 con 0x5589edcbce40 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec74c75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edc16bc0 con 0x5589edcbd7a0 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec74c75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edc144e0 con 0x5589edcbe2b0 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53904/0 (socket says 172.21.15.49:53904) 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 -- 172.21.15.49:0/2686171606 learned_addr learned my addr 172.21.15.49:0/2686171606 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec62d9d700 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edcbe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec61d9b700 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edcbdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 -- 172.21.15.49:0/2686171606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 msgr2=0x5589edcbe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edcbe730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 -- 172.21.15.49:0/2686171606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 msgr2=0x5589edcbdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edcbdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 -- 172.21.15.49:0/2686171606 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589eda3ad80 con 0x5589edcbce40 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec62d9d700 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edcbe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec61d9b700 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edcbdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec6259c700 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbd260 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fec54017d20 tx=0x7fec54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14410523a06cfd30 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.178+0000 7fec50ff9700 1 -- 172.21.15.49:0/2686171606 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540172e0 con 0x5589edcbce40 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec50ff9700 1 -- 172.21.15.49:0/2686171606 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec54017900 con 0x5589edcbce40 2023-10-08T09:28:42.182 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec50ff9700 1 -- 172.21.15.49:0/2686171606 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54003e80 con 0x5589edcbce40 2023-10-08T09:28:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 -- 172.21.15.49:0/2686171606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 msgr2=0x5589edcbd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbd260 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fec54017d20 tx=0x7fec54017000 comp rx=0 tx=0).stop 2023-10-08T09:28:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 -- 172.21.15.49:0/2686171606 shutdown_connections 2023-10-08T09:28:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edcbe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbd260 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 --2- 172.21.15.49:0/2686171606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edcbdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 -- 172.21.15.49:0/2686171606 >> 172.21.15.49:0/2686171606 conn(0x5589edc31200 msgr2=0x5589edcb78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 -- 172.21.15.49:0/2686171606 shutdown_connections 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.179+0000 7fec74c75b80 1 -- 172.21.15.49:0/2686171606 wait complete. 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 -- start start 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 Processor -- start 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbce40 0x5589edcc6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbd7a0 0x5589edcc6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edc0fdd0 con 0x5589edcbd7a0 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edbac6b0 con 0x5589edcbce40 2023-10-08T09:28:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edbac350 con 0x5589edcbe2b0 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec74c75b80 10 client.0 _ll_register_callbacks cb 0x5589eda097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec6259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbce40 0x5589edcc6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec61d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbd7a0 0x5589edcc6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec6259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbce40 0x5589edcc6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43524/0 (socket says 172.21.15.49:43524) 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec61d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbd7a0 0x5589edcc6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53912/0 (socket says 172.21.15.49:53912) 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.180+0000 7fec6259c700 1 -- 172.21.15.49:0/3245108294 learned_addr learned my addr 172.21.15.49:0/3245108294 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec6259c700 1 -- 172.21.15.49:0/3245108294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 msgr2=0x5589edccdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec6259c700 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec6259c700 1 -- 172.21.15.49:0/3245108294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbd7a0 msgr2=0x5589edcc6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:42.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec6259c700 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbd7a0 0x5589edcc6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec6259c700 1 -- 172.21.15.49:0/3245108294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec641adb20 con 0x5589edcbce40 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec6259c700 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbce40 0x5589edcc6820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec5400dda0 tx=0x7fec54017ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8473fae622ec8be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec49ffb700 1 -- 172.21.15.49:0/3245108294 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540176f0 con 0x5589edcbce40 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589edbd7e50 con 0x5589edcbce40 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec49ffb700 1 -- 172.21.15.49:0/3245108294 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fec5403d460 con 0x5589edcbce40 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec49ffb700 1 -- 172.21.15.49:0/3245108294 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec5403d620 con 0x5589edcbce40 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec74c75b80 10 client.26660 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fec5c00e140 con 0x5589edcbce40 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.181+0000 7fec49ffb700 1 -- 172.21.15.49:0/3245108294 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589edbd7e50 con 0x5589edcbce40 2023-10-08T09:28:42.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec49ffb700 1 -- 172.21.15.49:0/3245108294 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fec5400c720 con 0x5589edcbce40 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec74c75b80 10 client.26660 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec74c75b80 10 client.26660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec74c75b80 20 client.26660 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edcd2170 0x5589edcd4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec74c75b80 4 client.26660 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589edc08db0 con 0x5589edcd2170 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.182+0000 7fec61d9b700 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edcd2170 0x5589edcd4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.183+0000 7fec61d9b700 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edcd2170 0x5589edcd4560 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ec139efa49b8ba server_cookie=76d1b4c5282113b7 in_seq=0 out_seq=0 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.183+0000 7fec49ffb700 10 client.26660 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.184+0000 7fec49ffb700 1 -- 172.21.15.49:0/3245108294 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5589edc08db0 con 0x5589edcd2170 2023-10-08T09:28:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.184+0000 7fec49ffb700 10 client.26660 handle_command_reply: tid=0 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.184+0000 7fec74c75b80 1 client.26660 shutdown 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.184+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbce40 msgr2=0x5589edcc6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.184+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbce40 0x5589edcc6820 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fec5400dda0 tx=0x7fec54017ae0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.184+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 shutdown_connections 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edcd2170 0x5589edcd4560 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbce40 0x5589edcc6820 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3245108294 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbd7a0 0x5589edcc6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 >> 172.21.15.49:0/3245108294 conn(0x5589edc31200 msgr2=0x5589edcb7010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec49ffb700 0 client.26660 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 shutdown_connections 2023-10-08T09:28:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 1 -- 172.21.15.49:0/3245108294 wait complete. 2023-10-08T09:28:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.185+0000 7fec74c75b80 20 client.26660 trim_cache size 0 max 16384 2023-10-08T09:28:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 Processor -- start 2023-10-08T09:28:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 -- start start 2023-10-08T09:28:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbd7a0 0x5589edccdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbe2b0 0x5589edcb3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589eda908c0 con 0x5589edcbce40 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edcb9390 con 0x5589edcbe2b0 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec74c75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edcbdb80 con 0x5589edcbd7a0 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.205+0000 7fec6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53918/0 (socket says 172.21.15.49:53918) 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbe2b0 0x5589edcb3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 -- 172.21.15.49:0/3820005437 learned_addr learned my addr 172.21.15.49:0/3820005437 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec61d9b700 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbd7a0 0x5589edccdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 -- 172.21.15.49:0/3820005437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbd7a0 msgr2=0x5589edccdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbd7a0 0x5589edccdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 -- 172.21.15.49:0/3820005437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbe2b0 msgr2=0x5589edcb3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbe2b0 0x5589edcb3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 -- 172.21.15.49:0/3820005437 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec54015040 con 0x5589edcbce40 2023-10-08T09:28:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec6259c700 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbe690 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fec5400df50 tx=0x7fec54007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab1aa8741a222e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec517fa700 1 -- 172.21.15.49:0/3820005437 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400cc50 con 0x5589edcbce40 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec517fa700 1 -- 172.21.15.49:0/3820005437 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec5400cdb0 con 0x5589edcbce40 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.206+0000 7fec517fa700 1 -- 172.21.15.49:0/3820005437 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400c900 con 0x5589edcbce40 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 -- 172.21.15.49:0/3820005437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 msgr2=0x5589edcbe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbe690 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fec5400df50 tx=0x7fec54007a80 comp rx=0 tx=0).stop 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 -- 172.21.15.49:0/3820005437 shutdown_connections 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbe2b0 0x5589edcb3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edcbe690 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 --2- 172.21.15.49:0/3820005437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbd7a0 0x5589edccdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 -- 172.21.15.49:0/3820005437 >> 172.21.15.49:0/3820005437 conn(0x5589edc31200 msgr2=0x5589edcb7870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 -- 172.21.15.49:0/3820005437 shutdown_connections 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.207+0000 7fec74c75b80 1 -- 172.21.15.49:0/3820005437 wait complete. 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 -- start start 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 Processor -- start 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edccb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edccbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edccd390 con 0x5589edcbce40 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edccd500 con 0x5589edcbd7a0 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589edccd6c0 con 0x5589edcbe2b0 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec74c75b80 10 client.0 _ll_register_callbacks cb 0x7fec58010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edccb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edccb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53924/0 (socket says 172.21.15.49:53924) 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 -- 172.21.15.49:0/4174493112 learned_addr learned my addr 172.21.15.49:0/4174493112 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec62d9d700 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 -- 172.21.15.49:0/4174493112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 msgr2=0x5589edccce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 -- 172.21.15.49:0/4174493112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 msgr2=0x5589edccbee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edccbee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.208+0000 7fec6259c700 1 -- 172.21.15.49:0/4174493112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c00e140 con 0x5589edcbce40 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec6259c700 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edccb9a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fec54007b90 tx=0x7fec54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b751871fb86ea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec49ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec49ffb700 1 -- 172.21.15.49:0/4174493112 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400cc50 con 0x5589edcbce40 2023-10-08T09:28:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec49ffb700 1 -- 172.21.15.49:0/4174493112 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fec54003d40 con 0x5589edcbce40 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec54017900 con 0x5589edcbce40 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec49ffb700 1 -- 172.21.15.49:0/4174493112 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec54004040 con 0x5589edcbce40 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 10 client.17100 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5589eda23c00 con 0x5589edcbce40 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec49ffb700 1 -- 172.21.15.49:0/4174493112 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec54017900 con 0x5589edcbce40 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec49ffb700 1 -- 172.21.15.49:0/4174493112 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fec5403d460 con 0x5589edcbce40 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 10 client.17100 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 20 client.17100 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edd267e0 0x5589edd28c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 4 client.17100 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:28:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.209+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5589edc08ad0 con 0x5589edd267e0 2023-10-08T09:28:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.210+0000 7fec61d9b700 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edd267e0 0x5589edd28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.211+0000 7fec49ffb700 10 client.17100 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.211+0000 7fec61d9b700 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edd267e0 0x5589edd28c00 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e018668d439ebf7 server_cookie=71f8655f0daea79c in_seq=0 out_seq=0 2023-10-08T09:28:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec49ffb700 1 -- 172.21.15.49:0/4174493112 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5589edc08ad0 con 0x5589edd267e0 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec49ffb700 10 client.17100 handle_command_reply: tid=0 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 client.17100 shutdown 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 msgr2=0x5589edccb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edccb9a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fec54007b90 tx=0x7fec54000bc0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 shutdown_connections 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5589edd267e0 0x5589edd28c00 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5589edcbe2b0 0x5589edccce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec49ffb700 0 client.17100 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5589edcbce40 0x5589edccb9a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 --2- 172.21.15.49:0/4174493112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5589edcbd7a0 0x5589edccbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.231+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 >> 172.21.15.49:0/4174493112 conn(0x5589edc31200 msgr2=0x5589edcc5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.232+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 shutdown_connections 2023-10-08T09:28:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.232+0000 7fec74c75b80 1 -- 172.21.15.49:0/4174493112 wait complete. 2023-10-08T09:28:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.234+0000 7fec74c75b80 20 client.17100 trim_cache size 0 max 16384 2023-10-08T09:28:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:28:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:28:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:28:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:28:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.234+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fec4c0a1550 msgr2=0x7fec4c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.234+0000 7fec68a8c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fec4c0a1550 0x7fec4c0a3a10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fec5400dcf0 tx=0x7fec5400f040 comp rx=0 tx=0).stop 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.234+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 msgr2=0x7fec64194ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.234+0000 7fec68a8c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 0x7fec64194ab0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fec5800b530 tx=0x7fec5800b8f0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 shutdown_connections 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fec4c0a1550 0x7fec4c0a3a10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec64101200 0x7fec6419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec640ff670 0x7fec64194570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 --2- 172.21.15.49:0/3966402465 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec64100860 0x7fec64194ab0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 >> 172.21.15.49:0/3966402465 conn(0x7fec640fac00 msgr2=0x7fec640fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 shutdown_connections 2023-10-08T09:28:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:42.235+0000 7fec68a8c700 1 -- 172.21.15.49:0/3966402465 wait complete. 2023-10-08T09:28:42.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:41 smithi184 ceph-mon[103675]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2023-10-08T09:28:42.302 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:41 smithi132 ceph-mon[103389]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2023-10-08T09:28:42.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:41 smithi049 ceph-mon[99485]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2023-10-08T09:28:43.499 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:43 smithi184 ceph-mon[103675]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.9 MiB/s wr, 516 op/s 2023-10-08T09:28:43.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:43 smithi132 ceph-mon[103389]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.9 MiB/s wr, 516 op/s 2023-10-08T09:28:43.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:43 smithi049 ceph-mon[99485]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 4.9 MiB/s wr, 516 op/s 2023-10-08T09:28:44.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 800.0 s, 477.9 tps, lat 211.714 ms stddev 223.293 2023-10-08T09:28:46.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:45 smithi049 ceph-mon[99485]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.8 MiB/s wr, 500 op/s 2023-10-08T09:28:46.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:45 smithi184 ceph-mon[103675]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.8 MiB/s wr, 500 op/s 2023-10-08T09:28:46.302 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:45 smithi132 ceph-mon[103389]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.8 MiB/s wr, 500 op/s 2023-10-08T09:28:47.140 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:46 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:28:47.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:46 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:28:47.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:46 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:28:48.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:47 smithi049 ceph-mon[99485]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2023-10-08T09:28:48.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:28:48.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:28:48.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:28:48.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:47 smithi184 ceph-mon[103675]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2023-10-08T09:28:48.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:28:48.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:28:48.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:28:48.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:47 smithi132 ceph-mon[103389]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 4.6 MiB/s wr, 495 op/s 2023-10-08T09:28:48.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:28:48.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:28:48.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:28:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 805.0 s, 479.2 tps, lat 205.223 ms stddev 223.151 2023-10-08T09:28:50.134 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:49 smithi049 ceph-mon[99485]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2023-10-08T09:28:50.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:49 smithi184 ceph-mon[103675]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2023-10-08T09:28:50.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:49 smithi132 ceph-mon[103389]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2023-10-08T09:28:52.213 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:28:52.243 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:51 smithi049 ceph-mon[99485]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.0 MiB/s wr, 518 op/s 2023-10-08T09:28:52.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:51 smithi184 ceph-mon[103675]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.0 MiB/s wr, 518 op/s 2023-10-08T09:28:52.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:51 smithi132 ceph-mon[103389]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.0 MiB/s wr, 518 op/s 2023-10-08T09:28:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.404+0000 7f5076735700 1 -- 172.21.15.49:0/1164530522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5070100180 msgr2=0x7f5070104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.404+0000 7f5076735700 1 --2- 172.21.15.49:0/1164530522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5070100180 0x7f5070104a20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f50600098f0 tx=0x7f5060009c00 comp rx=0 tx=0).stop 2023-10-08T09:28:52.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.420+0000 7f5076735700 1 -- 172.21.15.49:0/1164530522 shutdown_connections 2023-10-08T09:28:52.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.420+0000 7f5076735700 1 --2- 172.21.15.49:0/1164530522 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50700fea60 0x7f50700fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.420+0000 7f5076735700 1 --2- 172.21.15.49:0/1164530522 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5070100180 0x7f5070104a20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.421 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.420+0000 7f5076735700 1 --2- 172.21.15.49:0/1164530522 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50700ff7c0 0x7f50700ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.422 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.420+0000 7f5076735700 1 -- 172.21.15.49:0/1164530522 >> 172.21.15.49:0/1164530522 conn(0x7f50700fa390 msgr2=0x7f50700fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:52.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.435+0000 7f5076735700 1 -- 172.21.15.49:0/1164530522 shutdown_connections 2023-10-08T09:28:52.444 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.443+0000 7f5076735700 1 -- 172.21.15.49:0/1164530522 wait complete. 2023-10-08T09:28:52.445 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.444+0000 7f5076735700 1 Processor -- start 2023-10-08T09:28:52.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.444+0000 7f5076735700 1 -- start start 2023-10-08T09:28:52.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.444+0000 7f5076735700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50700fea60 0x7f5070194540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.444+0000 7f5076735700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50700ff7c0 0x7f5070194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.446 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.444+0000 7f5076735700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5070100180 0x7f507019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.444+0000 7f5076735700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507019c070 con 0x7f5070100180 2023-10-08T09:28:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5076735700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507006a210 con 0x7f50700fea60 2023-10-08T09:28:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5076735700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507006a3a0 con 0x7f50700ff7c0 2023-10-08T09:28:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5070100180 0x7f507019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5070100180 0x7f507019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53940/0 (socket says 172.21.15.49:53940) 2023-10-08T09:28:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 -- 172.21.15.49:0/985461012 learned_addr learned my addr 172.21.15.49:0/985461012 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:52.447 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 -- 172.21.15.49:0/985461012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50700ff7c0 msgr2=0x7f5070194a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:28:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50700ff7c0 0x7f5070194a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 -- 172.21.15.49:0/985461012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50700fea60 msgr2=0x7f5070194540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50700fea60 0x7f5070194540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.445+0000 7f5074cd2700 1 -- 172.21.15.49:0/985461012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5060009520 con 0x7f5070100180 2023-10-08T09:28:52.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.446+0000 7f5074cd2700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5070100180 0x7f507019baa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f506000fba0 tx=0x7f506000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.447+0000 7f506d7fa700 1 -- 172.21.15.49:0/985461012 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506001c070 con 0x7f5070100180 2023-10-08T09:28:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.447+0000 7f506d7fa700 1 -- 172.21.15.49:0/985461012 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5060005500 con 0x7f5070100180 2023-10-08T09:28:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.447+0000 7f506d7fa700 1 -- 172.21.15.49:0/985461012 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5060004c10 con 0x7f5070100180 2023-10-08T09:28:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.447+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507006a620 con 0x7f5070100180 2023-10-08T09:28:52.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.447+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507006aa80 con 0x7f5070100180 2023-10-08T09:28:52.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.450+0000 7f506d7fa700 1 -- 172.21.15.49:0/985461012 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5060020070 con 0x7f5070100180 2023-10-08T09:28:52.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.451+0000 7f506d7fa700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f50580a1600 0x7f50580a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.451+0000 7f506d7fa700 1 -- 172.21.15.49:0/985461012 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f50600cd930 con 0x7f5070100180 2023-10-08T09:28:52.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.452+0000 7f506ffff700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f50580a1600 0x7f50580a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.453+0000 7f506ffff700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f50580a1600 0x7f50580a3ac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f506800abb0 tx=0x7f5068009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:52.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.454+0000 7f508291eb80 1 Processor -- start 2023-10-08T09:28:52.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.454+0000 7f508291eb80 1 -- start start 2023-10-08T09:28:52.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f508291eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f078480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f508291eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f079270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.456 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f508291eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f079c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f508291eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58efe0460 con 0x55a58f0789c0 2023-10-08T09:28:52.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f508291eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58efd1c20 con 0x55a58f079800 2023-10-08T09:28:52.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f508291eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58efcf540 con 0x55a58f078060 2023-10-08T09:28:52.457 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f506ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f078480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f5074cd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f5074cd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f079c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43560/0 (socket says 172.21.15.49:43560) 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f506ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f078480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40978/0 (socket says 172.21.15.49:40978) 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f5074cd2700 1 -- 172.21.15.49:0/37291937 learned_addr learned my addr 172.21.15.49:0/37291937 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f5074cd2700 1 -- 172.21.15.49:0/37291937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 msgr2=0x55a58f078480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f5074cd2700 1 --2- 172.21.15.49:0/37291937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f078480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.455+0000 7f5074cd2700 1 -- 172.21.15.49:0/37291937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 msgr2=0x55a58f079270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.456+0000 7f5074cd2700 1 --2- 172.21.15.49:0/37291937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f079270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.456+0000 7f5074cd2700 1 -- 172.21.15.49:0/37291937 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a58edf5de0 con 0x55a58f079800 2023-10-08T09:28:52.458 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.456+0000 7f5074cd2700 1 --2- 172.21.15.49:0/37291937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f079c90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50600098c0 tx=0x7f506009a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f1e76d675a07bda server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:52.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.460+0000 7f50567fc700 1 -- 172.21.15.49:0/37291937 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506009a9b0 con 0x55a58f079800 2023-10-08T09:28:52.464 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.460+0000 7f50567fc700 1 -- 172.21.15.49:0/37291937 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5060096cf0 con 0x55a58f079800 2023-10-08T09:28:52.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.460+0000 7f50567fc700 1 -- 172.21.15.49:0/37291937 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50600ba0b0 con 0x55a58f079800 2023-10-08T09:28:52.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.461+0000 7f508291eb80 1 -- 172.21.15.49:0/37291937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 msgr2=0x55a58f079c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.465 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.461+0000 7f508291eb80 1 --2- 172.21.15.49:0/37291937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f079c90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50600098c0 tx=0x7f506009a5c0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.470+0000 7f508291eb80 1 -- 172.21.15.49:0/37291937 shutdown_connections 2023-10-08T09:28:52.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.470+0000 7f508291eb80 1 --2- 172.21.15.49:0/37291937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f078480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.470+0000 7f508291eb80 1 --2- 172.21.15.49:0/37291937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f079c90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.470+0000 7f508291eb80 1 --2- 172.21.15.49:0/37291937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f079270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.470+0000 7f508291eb80 1 -- 172.21.15.49:0/37291937 >> 172.21.15.49:0/37291937 conn(0x55a58efec260 msgr2=0x55a58f072e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:52.473 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.471+0000 7f508291eb80 1 -- 172.21.15.49:0/37291937 shutdown_connections 2023-10-08T09:28:52.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.471+0000 7f508291eb80 1 -- 172.21.15.49:0/37291937 wait complete. 2023-10-08T09:28:52.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.472+0000 7f508291eb80 1 -- start start 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.472+0000 7f508291eb80 1 Processor -- start 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.473+0000 7f508291eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f078060 0x55a58f081b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.473+0000 7f508291eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f0820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.473+0000 7f508291eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f079800 0x55a58f0890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.473+0000 7f508291eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58efcae30 con 0x55a58f079800 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.473+0000 7f508291eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ef67710 con 0x55a58f078060 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.473+0000 7f508291eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ef673b0 con 0x55a58f0789c0 2023-10-08T09:28:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.473+0000 7f508291eb80 10 client.0 _ll_register_callbacks cb 0x55a58edc4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f078060 0x55a58f081b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f078060 0x55a58f081b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43562/0 (socket says 172.21.15.49:43562) 2023-10-08T09:28:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 -- 172.21.15.49:0/121623238 learned_addr learned my addr 172.21.15.49:0/121623238 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 -- 172.21.15.49:0/121623238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 msgr2=0x55a58f0820b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:28:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f0820b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 -- 172.21.15.49:0/121623238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f079800 msgr2=0x55a58f0890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:52.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f079800 0x55a58f0890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.474+0000 7f506ffff700 1 -- 172.21.15.49:0/121623238 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507006aa80 con 0x55a58f078060 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f506ffff700 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f078060 0x55a58f081b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f506801ae00 tx=0x7f5068004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=722dc9b36f74e5c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f504f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f504f7fe700 1 -- 172.21.15.49:0/121623238 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068018bd0 con 0x55a58f078060 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a58ef92eb0 con 0x55a58f078060 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f504f7fe700 1 -- 172.21.15.49:0/121623238 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f5068018d30 con 0x55a58f078060 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f504f7fe700 1 -- 172.21.15.49:0/121623238 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50680284c0 con 0x55a58f078060 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f508291eb80 10 client.26672 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.475+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a58eddec60 con 0x55a58f078060 2023-10-08T09:28:52.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.476+0000 7f504f7fe700 1 -- 172.21.15.49:0/121623238 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a58ef92eb0 con 0x55a58f078060 2023-10-08T09:28:52.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f504f7fe700 1 -- 172.21.15.49:0/121623238 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5068028620 con 0x55a58f078060 2023-10-08T09:28:52.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f508291eb80 10 client.26672 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:52.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f508291eb80 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:52.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f508291eb80 20 client.26672 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:52.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f508291eb80 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f08d950 0x55a58f08fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f508291eb80 4 client.26672 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:28:52.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a58efc3e10 con 0x55a58f08d950 2023-10-08T09:28:52.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.477+0000 7f506f7fe700 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f08d950 0x55a58f08fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.478+0000 7f504f7fe700 10 client.26672 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:52.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.478+0000 7f506f7fe700 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f08d950 0x55a58f08fdb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c7d9f0a44ec07e server_cookie=7afde8577b4bb632 in_seq=0 out_seq=0 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.478+0000 7f504f7fe700 1 -- 172.21.15.49:0/121623238 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a58efc3e10 con 0x55a58f08d950 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.478+0000 7f504f7fe700 10 client.26672 handle_command_reply: tid=0 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.478+0000 7f508291eb80 1 client.26672 shutdown 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f078060 msgr2=0x55a58f081b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f078060 0x55a58f081b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f506801ae00 tx=0x7f5068004bd0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 shutdown_connections 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f08d950 0x55a58f08fdb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f079800 0x55a58f0890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f078060 0x55a58f081b70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 --2- 172.21.15.49:0/121623238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f0820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.481 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 >> 172.21.15.49:0/121623238 conn(0x55a58efec260 msgr2=0x55a58f0724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:52.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f504f7fe700 0 client.26672 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:52.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 shutdown_connections 2023-10-08T09:28:52.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.479+0000 7f508291eb80 1 -- 172.21.15.49:0/121623238 wait complete. 2023-10-08T09:28:52.482 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.480+0000 7f508291eb80 20 client.26672 trim_cache size 0 max 16384 2023-10-08T09:28:52.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 Processor -- start 2023-10-08T09:28:52.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 -- start start 2023-10-08T09:28:52.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f078060 0x55a58f07c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f07cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f089060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ee4b920 con 0x55a58f078060 2023-10-08T09:28:52.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58f06fae0 con 0x55a58f079800 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.500+0000 7f508291eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58efd1c20 con 0x55a58f0789c0 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f078060 0x55a58f07c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f07cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f5074cd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f089060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f078060 0x55a58f07c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53956/0 (socket says 172.21.15.49:53956) 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f5074cd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f089060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43574/0 (socket says 172.21.15.49:43574) 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f07cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:40990/0 (socket says 172.21.15.49:40990) 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 -- 172.21.15.49:0/2533575204 learned_addr learned my addr 172.21.15.49:0/2533575204 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 -- 172.21.15.49:0/2533575204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 msgr2=0x55a58f07cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 --2- 172.21.15.49:0/2533575204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f07cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 -- 172.21.15.49:0/2533575204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 msgr2=0x55a58f089060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 --2- 172.21.15.49:0/2533575204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f089060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 -- 172.21.15.49:0/2533575204 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a58ee1c250 con 0x55a58f078060 2023-10-08T09:28:52.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f506ffff700 1 --2- 172.21.15.49:0/2533575204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f078060 0x55a58f07c7f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5068011040 tx=0x7f506800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecceb396b9a9b20b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:52.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f5056ffd700 1 -- 172.21.15.49:0/2533575204 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068035070 con 0x55a58f078060 2023-10-08T09:28:52.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f5056ffd700 1 -- 172.21.15.49:0/2533575204 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5068018a20 con 0x55a58f078060 2023-10-08T09:28:52.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.501+0000 7f5056ffd700 1 -- 172.21.15.49:0/2533575204 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50680287a0 con 0x55a58f078060 2023-10-08T09:28:52.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 -- 172.21.15.49:0/2533575204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f078060 msgr2=0x55a58f07c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 --2- 172.21.15.49:0/2533575204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f078060 0x55a58f07c7f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5068011040 tx=0x7f506800cc40 comp rx=0 tx=0).stop 2023-10-08T09:28:52.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 -- 172.21.15.49:0/2533575204 shutdown_connections 2023-10-08T09:28:52.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 --2- 172.21.15.49:0/2533575204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f089060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 --2- 172.21.15.49:0/2533575204 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f078060 0x55a58f07c7f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.507 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 --2- 172.21.15.49:0/2533575204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f0789c0 0x55a58f07cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 -- 172.21.15.49:0/2533575204 >> 172.21.15.49:0/2533575204 conn(0x55a58efec260 msgr2=0x55a58f081080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 -- 172.21.15.49:0/2533575204 shutdown_connections 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.502+0000 7f508291eb80 1 -- 172.21.15.49:0/2533575204 wait complete. 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 -- start start 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 Processor -- start 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f088ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f088ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f087150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58f073b00 con 0x55a58f0789c0 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58f087890 con 0x55a58f079800 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58f087a50 con 0x55a58f078060 2023-10-08T09:28:52.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.503+0000 7f508291eb80 10 client.0 _ll_register_callbacks cb 0x55a58edc4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f506ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f088ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f506f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f088ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f506f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f088ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53964/0 (socket says 172.21.15.49:53964) 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f506f7fe700 1 -- 172.21.15.49:0/26002795 learned_addr learned my addr 172.21.15.49:0/26002795 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f5074cd2700 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f087150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f5074cd2700 1 -- 172.21.15.49:0/26002795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 msgr2=0x55a58f088ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f5074cd2700 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f088ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f5074cd2700 1 -- 172.21.15.49:0/26002795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 msgr2=0x55a58f088ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f5074cd2700 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f088ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f5074cd2700 1 -- 172.21.15.49:0/26002795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a58eddec60 con 0x55a58f079800 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f5074cd2700 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f087150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55a58f080600 tx=0x7f50600057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af181951f1fb57a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.504+0000 7f504f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.505+0000 7f504f7fe700 1 -- 172.21.15.49:0/26002795 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50600a2cb0 con 0x55a58f079800 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.505+0000 7f504f7fe700 1 -- 172.21.15.49:0/26002795 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f506009eb00 con 0x55a58f079800 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.505+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5060096cf0 con 0x55a58f079800 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.505+0000 7f504f7fe700 1 -- 172.21.15.49:0/26002795 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5060004d70 con 0x55a58f079800 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.505+0000 7f508291eb80 10 client.26678 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.505+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a58edf42b0 con 0x55a58f079800 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.505+0000 7f504f7fe700 1 -- 172.21.15.49:0/26002795 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50600b8020 con 0x55a58f079800 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f504f7fe700 1 -- 172.21.15.49:0/26002795 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5060099ea0 con 0x55a58f079800 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f508291eb80 10 client.26678 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f508291eb80 10 client.26678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f508291eb80 20 client.26678 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:52.510 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f508291eb80 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f0e26a0 0x55a58f0e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:52.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f508291eb80 4 client.26678 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:28:52.511 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a58efc3b30 con 0x55a58f0e26a0 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.506+0000 7f506ffff700 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f0e26a0 0x55a58f0e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f504f7fe700 10 client.26678 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f506ffff700 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f0e26a0 0x55a58f0e4ac0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff10f9956abda5a server_cookie=ab16991684e180b0 in_seq=0 out_seq=0 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f504f7fe700 1 -- 172.21.15.49:0/26002795 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a58efc3b30 con 0x55a58f0e26a0 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f504f7fe700 10 client.26678 handle_command_reply: tid=0 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 client.26678 shutdown 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 msgr2=0x55a58f087150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f087150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55a58f080600 tx=0x7f50600057e0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 shutdown_connections 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a58f0e26a0 0x55a58f0e4ac0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a58f078060 0x55a58f088ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f504f7fe700 0 client.26678 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a58f079800 0x55a58f087150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 --2- 172.21.15.49:0/26002795 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a58f0789c0 0x55a58f088ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 >> 172.21.15.49:0/26002795 conn(0x55a58efec260 msgr2=0x55a58f07c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 shutdown_connections 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 1 -- 172.21.15.49:0/26002795 wait complete. 2023-10-08T09:28:52.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.507+0000 7f508291eb80 20 client.26678 trim_cache size 0 max 16384 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:28:52.514 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:28:52.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f50580a1600 msgr2=0x7f50580a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f50580a1600 0x7f50580a3ac0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f506800abb0 tx=0x7f5068009250 comp rx=0 tx=0).stop 2023-10-08T09:28:52.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5070100180 msgr2=0x7f507019baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:52.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5070100180 0x7f507019baa0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f506000fba0 tx=0x7f506000b840 comp rx=0 tx=0).stop 2023-10-08T09:28:52.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 shutdown_connections 2023-10-08T09:28:52.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f50580a1600 0x7f50580a3ac0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.515 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5070100180 0x7f507019baa0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50700fea60 0x7f5070194540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 --2- 172.21.15.49:0/985461012 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50700ff7c0 0x7f5070194a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:52.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 >> 172.21.15.49:0/985461012 conn(0x7f50700fa390 msgr2=0x7f5070102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:52.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 shutdown_connections 2023-10-08T09:28:52.516 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:28:52.508+0000 7f5076735700 1 -- 172.21.15.49:0/985461012 wait complete. 2023-10-08T09:28:52.526 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:28:53.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:53 smithi184 ceph-mon[103675]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 521 op/s 2023-10-08T09:28:53.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:53 smithi132 ceph-mon[103389]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 521 op/s 2023-10-08T09:28:53.636 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:53 smithi049 ceph-mon[99485]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 521 op/s 2023-10-08T09:28:54.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 810.0 s, 494.0 tps, lat 202.554 ms stddev 217.650 2023-10-08T09:28:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.486+0000 7f8a94f15700 1 -- 172.21.15.49:0/777300382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 msgr2=0x7f8a90109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.486+0000 7f8a94f15700 1 --2- 172.21.15.49:0/777300382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 0x7f8a90109c20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009990 tx=0x7f8a84009ca0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.496+0000 7f8a94f15700 1 -- 172.21.15.49:0/777300382 shutdown_connections 2023-10-08T09:28:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.496+0000 7f8a94f15700 1 --2- 172.21.15.49:0/777300382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 0x7f8a90109c20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.496+0000 7f8a94f15700 1 --2- 172.21.15.49:0/777300382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 0x7f8a9010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.496+0000 7f8a94f15700 1 --2- 172.21.15.49:0/777300382 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a9010a160 0x7f8a9010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.496+0000 7f8a94f15700 1 -- 172.21.15.49:0/777300382 >> 172.21.15.49:0/777300382 conn(0x7f8a9006d770 msgr2=0x7f8a9006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.510+0000 7f8a94f15700 1 -- 172.21.15.49:0/777300382 shutdown_connections 2023-10-08T09:28:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.514+0000 7f8a94f15700 1 -- 172.21.15.49:0/777300382 wait complete. 2023-10-08T09:28:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.515+0000 7f8a94f15700 1 Processor -- start 2023-10-08T09:28:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.515+0000 7f8a94f15700 1 -- start start 2023-10-08T09:28:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a94f15700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 0x7f8a90116fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a94f15700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a9010a160 0x7f8a90117500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a94f15700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 0x7f8a90118530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a94f15700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90118b30 con 0x7f8a9010a160 2023-10-08T09:28:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a94f15700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90118ca0 con 0x7f8a9010cb30 2023-10-08T09:28:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a94f15700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90118e10 con 0x7f8a901077f0 2023-10-08T09:28:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a8ed9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 0x7f8a90118530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a8ed9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 0x7f8a90118530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43590/0 (socket says 172.21.15.49:43590) 2023-10-08T09:28:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.516+0000 7f8a8ed9d700 1 -- 172.21.15.49:0/3780780727 learned_addr learned my addr 172.21.15.49:0/3780780727 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 0x7f8a90116fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8ed9d700 1 -- 172.21.15.49:0/3780780727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 msgr2=0x7f8a90116fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8ed9d700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 0x7f8a90116fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a9010a160 0x7f8a90117500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8ed9d700 1 -- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a9010a160 msgr2=0x7f8a90117500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8ed9d700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a9010a160 0x7f8a90117500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8ed9d700 1 -- 172.21.15.49:0/3780780727 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840095f0 con 0x7f8a9010cb30 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a9010a160 0x7f8a90117500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.517+0000 7f8a8ed9d700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 0x7f8a90118530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800ef30 tx=0x7f8a8800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.518+0000 7f8a7f7fe700 1 -- 172.21.15.49:0/3780780727 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88018070 con 0x7f8a9010cb30 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.518+0000 7f8a7f7fe700 1 -- 172.21.15.49:0/3780780727 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8800f040 con 0x7f8a9010cb30 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.518+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a90119090 con 0x7f8a9010cb30 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.518+0000 7f8a7f7fe700 1 -- 172.21.15.49:0/3780780727 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800cdb0 con 0x7f8a9010cb30 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.518+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a901195a0 con 0x7f8a9010cb30 2023-10-08T09:28:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.521+0000 7f8a7f7fe700 1 -- 172.21.15.49:0/3780780727 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8a88004000 con 0x7f8a9010cb30 2023-10-08T09:28:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.522+0000 7f8a7f7fe700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a780a1600 0x7f8a780a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.522+0000 7f8a7f7fe700 1 -- 172.21.15.49:0/3780780727 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8a880cd800 con 0x7f8a9010cb30 2023-10-08T09:28:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.522+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a780a1600 0x7f8a780a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a780a1600 0x7f8a780a3ac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8a901184c0 tx=0x7f8a8400b6b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 Processor -- start 2023-10-08T09:28:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 -- start start 2023-10-08T09:28:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa05e40 0x55f49aa06260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa067a0 0x55f49aa06cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa07730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a96e400 con 0x55f49aa067a0 2023-10-08T09:28:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a95fbc0 con 0x55f49aa05e40 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a95d4e0 con 0x55f49aa072b0 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8a8e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa05e40 0x55f49aa06260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8a8e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa05e40 0x55f49aa06260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43598/0 (socket says 172.21.15.49:43598) 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.524+0000 7f8a8e59c700 1 -- 172.21.15.49:0/4057256677 learned_addr learned my addr 172.21.15.49:0/4057256677 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa067a0 0x55f49aa06cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8ed9d700 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa07730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8e59c700 1 -- 172.21.15.49:0/4057256677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 msgr2=0x55f49aa07730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa07730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8e59c700 1 -- 172.21.15.49:0/4057256677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa067a0 msgr2=0x55f49aa06cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa067a0 0x55f49aa06cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8e59c700 1 -- 172.21.15.49:0/4057256677 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f49a783d80 con 0x55f49aa05e40 2023-10-08T09:28:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa067a0 0x55f49aa06cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa05e40 0x55f49aa06260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009990 tx=0x7f8a84020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e2353ff2a7dbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a7cff9700 1 -- 172.21.15.49:0/4057256677 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400bc10 con 0x55f49aa05e40 2023-10-08T09:28:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a7cff9700 1 -- 172.21.15.49:0/4057256677 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8401e040 con 0x55f49aa05e40 2023-10-08T09:28:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.525+0000 7f8a7cff9700 1 -- 172.21.15.49:0/4057256677 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840048a0 con 0x55f49aa05e40 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.526+0000 7f8aa10feb80 1 -- 172.21.15.49:0/4057256677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa05e40 msgr2=0x55f49aa06260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.526+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa05e40 0x55f49aa06260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009990 tx=0x7f8a84020000 comp rx=0 tx=0).stop 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.526+0000 7f8aa10feb80 1 -- 172.21.15.49:0/4057256677 shutdown_connections 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.526+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa07730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.526+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa05e40 0x55f49aa06260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.526+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/4057256677 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa067a0 0x55f49aa06cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.526+0000 7f8aa10feb80 1 -- 172.21.15.49:0/4057256677 >> 172.21.15.49:0/4057256677 conn(0x55f49a97a200 msgr2=0x55f49aa008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.527+0000 7f8aa10feb80 1 -- 172.21.15.49:0/4057256677 shutdown_connections 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.527+0000 7f8aa10feb80 1 -- 172.21.15.49:0/4057256677 wait complete. 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.527+0000 7f8aa10feb80 1 -- start start 2023-10-08T09:28:55.531 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.527+0000 7f8aa10feb80 1 Processor -- start 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.527+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa067a0 0x55f49aa0fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa16cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a958dd0 con 0x55f49aa05e40 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a8f56b0 con 0x55f49aa067a0 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a8f5350 con 0x55f49aa072b0 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8aa10feb80 10 client.0 _ll_register_callbacks cb 0x55f49a7527a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa16cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa16cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41026/0 (socket says 172.21.15.49:41026) 2023-10-08T09:28:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8ed9d700 1 -- 172.21.15.49:0/3404343536 learned_addr learned my addr 172.21.15.49:0/3404343536 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8e59c700 1 -- 172.21.15.49:0/3404343536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 msgr2=0x55f49aa16cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa16cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8e59c700 1 -- 172.21.15.49:0/3404343536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa067a0 msgr2=0x55f49aa0fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa067a0 0x55f49aa0fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.534 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.528+0000 7f8a8e59c700 1 -- 172.21.15.49:0/3404343536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a80009900 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0f7c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8a84031040 tx=0x7f8a84004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53429b6526fdd70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8a75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8a75ffb700 1 -- 172.21.15.49:0/3404343536 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84005600 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f49a920e50 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8a75ffb700 1 -- 172.21.15.49:0/3404343536 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8a84004d70 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8a75ffb700 1 -- 172.21.15.49:0/3404343536 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a84003680 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 10 client.17121 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f8a9010da70 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8a75ffb700 1 -- 172.21.15.49:0/3404343536 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f49a920e50 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8a75ffb700 1 -- 172.21.15.49:0/3404343536 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8a84042070 con 0x55f49aa05e40 2023-10-08T09:28:55.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 20 client.17121 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa1b1f0 0x55f49aa1d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 4 client.17121 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.529+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f49a951db0 con 0x55f49aa1b1f0 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.530+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa1b1f0 0x55f49aa1d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.530+0000 7f8a75ffb700 10 client.17121 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.530+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa1b1f0 0x55f49aa1d610 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd4d271d3367f56 server_cookie=d83d899bff380964 in_seq=0 out_seq=0 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8a75ffb700 1 -- 172.21.15.49:0/3404343536 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f49a951db0 con 0x55f49aa1b1f0 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8a75ffb700 10 client.17121 handle_command_reply: tid=0 2023-10-08T09:28:55.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 client.17121 shutdown 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 msgr2=0x55f49aa0f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0f7c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8a84031040 tx=0x7f8a84004920 comp rx=0 tx=0).stop 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 shutdown_connections 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa1b1f0 0x55f49aa1d610 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa072b0 0x55f49aa16cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0f7c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/3404343536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa067a0 0x55f49aa0fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 >> 172.21.15.49:0/3404343536 conn(0x55f49a97a200 msgr2=0x55f49aa00010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8a75ffb700 0 client.17121 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 shutdown_connections 2023-10-08T09:28:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.532+0000 7f8aa10feb80 1 -- 172.21.15.49:0/3404343536 wait complete. 2023-10-08T09:28:55.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.533+0000 7f8aa10feb80 20 client.17121 trim_cache size 0 max 16384 2023-10-08T09:28:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.550+0000 7f8aa10feb80 1 Processor -- start 2023-10-08T09:28:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.550+0000 7f8aa10feb80 1 -- start start 2023-10-08T09:28:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa16c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa0ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49a7d98c0 con 0x55f49aa05e40 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49aa0e350 con 0x55f49aa072b0 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49aa014f0 con 0x55f49aa067a0 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53998/0 (socket says 172.21.15.49:53998) 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 -- 172.21.15.49:0/198903039 learned_addr learned my addr 172.21.15.49:0/198903039 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8ed9d700 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa0ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa16c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 -- 172.21.15.49:0/198903039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 msgr2=0x55f49aa16c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa16c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 -- 172.21.15.49:0/198903039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 msgr2=0x55f49aa0ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa0ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 -- 172.21.15.49:0/198903039 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f49a76cc00 con 0x55f49aa05e40 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.551+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0e530 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400b2e0 tx=0x7f8a84004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41af0b890f81ba93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8a7d7fa700 1 -- 172.21.15.49:0/198903039 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84004d70 con 0x55f49aa05e40 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8a7d7fa700 1 -- 172.21.15.49:0/198903039 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a84004ed0 con 0x55f49aa05e40 2023-10-08T09:28:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8a7d7fa700 1 -- 172.21.15.49:0/198903039 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8402f440 con 0x55f49aa05e40 2023-10-08T09:28:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 -- 172.21.15.49:0/198903039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 msgr2=0x55f49aa0e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0e530 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400b2e0 tx=0x7f8a84004500 comp rx=0 tx=0).stop 2023-10-08T09:28:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 -- 172.21.15.49:0/198903039 shutdown_connections 2023-10-08T09:28:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa0ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa0e530 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/198903039 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa16c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 -- 172.21.15.49:0/198903039 >> 172.21.15.49:0/198903039 conn(0x55f49a97a200 msgr2=0x55f49aa21f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 -- 172.21.15.49:0/198903039 shutdown_connections 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.552+0000 7f8aa10feb80 1 -- 172.21.15.49:0/198903039 wait complete. 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 -- start start 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 Processor -- start 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa14920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49aa15d30 con 0x55f49aa05e40 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49aa15ea0 con 0x55f49aa072b0 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f49aa16060 con 0x55f49aa067a0 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8aa10feb80 10 client.0 _ll_register_callbacks cb 0x7f8a80009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8a8e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8a8ed9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8a8e59c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa143e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54004/0 (socket says 172.21.15.49:54004) 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8a8dd9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa14920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8a8e59c700 1 -- 172.21.15.49:0/2343204555 learned_addr learned my addr 172.21.15.49:0/2343204555 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8a8e59c700 1 -- 172.21.15.49:0/2343204555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 msgr2=0x55f49aa14920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.553+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa14920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a8e59c700 1 -- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 msgr2=0x55f49aa157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a8e59c700 1 -- 172.21.15.49:0/2343204555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9010da70 con 0x55f49aa05e40 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a8ed9d700 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a8e59c700 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa143e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009ce0 tx=0x7f8a8400b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2839319664e90590 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a75ffb700 1 -- 172.21.15.49:0/2343204555 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84003880 con 0x55f49aa05e40 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a75ffb700 1 -- 172.21.15.49:0/2343204555 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8a840039e0 con 0x55f49aa05e40 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a8401e040 con 0x55f49aa05e40 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8a75ffb700 1 -- 172.21.15.49:0/2343204555 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a84003c50 con 0x55f49aa05e40 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8aa10feb80 10 client.17127 fetch_fsmap learned FSMap version 41 2023-10-08T09:28:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.554+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55f49a7aa1f0 con 0x55f49aa05e40 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8a75ffb700 1 -- 172.21.15.49:0/2343204555 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8401e040 con 0x55f49aa05e40 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8a75ffb700 1 -- 172.21.15.49:0/2343204555 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f8a84042070 con 0x55f49aa05e40 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8aa10feb80 10 client.17127 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8aa10feb80 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8aa10feb80 20 client.17127 populate_metadata read hostname 'smithi049' 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa6f340 0x55f49aa71730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8aa10feb80 4 client.17127 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f49a951ad0 con 0x55f49aa6f340 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa6f340 0x55f49aa71730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8a8dd9b700 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa6f340 0x55f49aa71730 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af56fc4e3268870 server_cookie=1ba7a971ca2caef9 in_seq=0 out_seq=0 2023-10-08T09:28:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.555+0000 7f8a75ffb700 10 client.17127 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8a75ffb700 1 -- 172.21.15.49:0/2343204555 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f49a951ad0 con 0x55f49aa6f340 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8a75ffb700 10 client.17127 handle_command_reply: tid=0 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 client.17127 shutdown 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 msgr2=0x55f49aa143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa143e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009ce0 tx=0x7f8a8400b310 comp rx=0 tx=0).stop 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 shutdown_connections 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55f49aa6f340 0x55f49aa71730 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55f49aa072b0 0x55f49aa157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8a75ffb700 0 client.17127 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55f49aa05e40 0x55f49aa143e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 --2- 172.21.15.49:0/2343204555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55f49aa067a0 0x55f49aa14920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 >> 172.21.15.49:0/2343204555 conn(0x55f49a97a200 msgr2=0x55f49aa0e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 shutdown_connections 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 1 -- 172.21.15.49:0/2343204555 wait complete. 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.567+0000 7f8aa10feb80 20 client.17127 trim_cache size 0 max 16384 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:28:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:28:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:28:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a780a1600 msgr2=0x7f8a780a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a780a1600 0x7f8a780a3ac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8a901184c0 tx=0x7f8a8400b6b0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 msgr2=0x7f8a90118530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 0x7f8a90118530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800ef30 tx=0x7f8a8800c3a0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 shutdown_connections 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a780a1600 0x7f8a780a3ac0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a901077f0 0x7f8a90116fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8a9010cb30 0x7f8a90118530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 --2- 172.21.15.49:0/3780780727 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8a9010a160 0x7f8a90117500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 >> 172.21.15.49:0/3780780727 conn(0x7f8a9006d770 msgr2=0x7f8a90107fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 shutdown_connections 2023-10-08T09:28:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:28:55.568+0000 7f8a94f15700 1 -- 172.21.15.49:0/3780780727 wait complete. 2023-10-08T09:28:56.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:55 smithi184 ceph-mon[103675]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.4 MiB/s wr, 476 op/s 2023-10-08T09:28:56.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:55 smithi132 ceph-mon[103389]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.4 MiB/s wr, 476 op/s 2023-10-08T09:28:56.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:55 smithi049 ceph-mon[99485]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.4 MiB/s wr, 476 op/s 2023-10-08T09:28:58.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:57 smithi184 ceph-mon[103675]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2023-10-08T09:28:58.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:57 smithi132 ceph-mon[103389]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2023-10-08T09:28:58.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:57 smithi049 ceph-mon[99485]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2023-10-08T09:28:59.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 815.0 s, 528.6 tps, lat 186.191 ms stddev 199.697 2023-10-08T09:29:00.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:28:59 smithi184 ceph-mon[103675]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2023-10-08T09:29:00.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:28:59 smithi132 ceph-mon[103389]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2023-10-08T09:29:00.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:28:59 smithi049 ceph-mon[99485]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2023-10-08T09:29:02.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:01 smithi184 ceph-mon[103675]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 495 op/s 2023-10-08T09:29:02.306 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:01 smithi132 ceph-mon[103389]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 495 op/s 2023-10-08T09:29:02.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:01 smithi049 ceph-mon[99485]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 495 op/s 2023-10-08T09:29:03.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:03 smithi184 ceph-mon[103675]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2023-10-08T09:29:03.552 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:03 smithi132 ceph-mon[103389]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2023-10-08T09:29:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:03 smithi049 ceph-mon[99485]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.9 MiB/s wr, 508 op/s 2023-10-08T09:29:04.321 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 820.0 s, 375.0 tps, lat 261.898 ms stddev 276.732 2023-10-08T09:29:06.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:05 smithi184 ceph-mon[103675]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.2 MiB/s wr, 478 op/s 2023-10-08T09:29:06.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:05 smithi049 ceph-mon[99485]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.2 MiB/s wr, 478 op/s 2023-10-08T09:29:06.433 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:05 smithi132 ceph-mon[103389]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.2 MiB/s wr, 478 op/s 2023-10-08T09:29:08.256 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:07 smithi184 ceph-mon[103675]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 494 op/s 2023-10-08T09:29:08.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:07 smithi132 ceph-mon[103389]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 494 op/s 2023-10-08T09:29:08.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:07 smithi049 ceph-mon[99485]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 5.4 MiB/s wr, 494 op/s 2023-10-08T09:29:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.792+0000 7fd8c3aff700 1 -- 172.21.15.49:0/3283289390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 msgr2=0x7fd8bc109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.792+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/3283289390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 0x7fd8bc109c20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0098f0 tx=0x7fd8ac009c00 comp rx=0 tx=0).stop 2023-10-08T09:29:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.800+0000 7fd8c3aff700 1 -- 172.21.15.49:0/3283289390 shutdown_connections 2023-10-08T09:29:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.800+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/3283289390 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 0x7fd8bc109c20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.800+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/3283289390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 0x7fd8bc10cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.800+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/3283289390 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8bc10a160 0x7fd8bc10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.800+0000 7fd8c3aff700 1 -- 172.21.15.49:0/3283289390 >> 172.21.15.49:0/3283289390 conn(0x7fd8bc06d770 msgr2=0x7fd8bc06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.821+0000 7fd8c3aff700 1 -- 172.21.15.49:0/3283289390 shutdown_connections 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.821+0000 7fd8c3aff700 1 -- 172.21.15.49:0/3283289390 wait complete. 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 Processor -- start 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 -- start start 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 0x7fd8bc1a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8bc10a160 0x7fd8bc1a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 0x7fd8bc078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc079020 con 0x7fd8bc10a160 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc079190 con 0x7fd8bc10cb30 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.822+0000 7fd8c3aff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc079350 con 0x7fd8bc1077f0 2023-10-08T09:29:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.823+0000 7fd8c209c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 0x7fd8bc078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.823+0000 7fd8c209c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 0x7fd8bc078a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43632/0 (socket says 172.21.15.49:43632) 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.823+0000 7fd8c209c700 1 -- 172.21.15.49:0/2889743183 learned_addr learned my addr 172.21.15.49:0/2889743183 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.823+0000 7fd8c189b700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 0x7fd8bc1a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.824+0000 7fd8c209c700 1 -- 172.21.15.49:0/2889743183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 msgr2=0x7fd8bc1a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.824+0000 7fd8c209c700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 0x7fd8bc1a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.824+0000 7fd8c209c700 1 -- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8bc10a160 msgr2=0x7fd8bc1a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.824+0000 7fd8c209c700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8bc10a160 0x7fd8bc1a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.824+0000 7fd8c209c700 1 -- 172.21.15.49:0/2889743183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac009520 con 0x7fd8bc10cb30 2023-10-08T09:29:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.824+0000 7fd8c209c700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 0x7fd8bc078a50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800b4a0 tx=0x7fd8b800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.825+0000 7fd8b2ffd700 1 -- 172.21.15.49:0/2889743183 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8011830 con 0x7fd8bc10cb30 2023-10-08T09:29:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.825+0000 7fd8b2ffd700 1 -- 172.21.15.49:0/2889743183 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8b8011e50 con 0x7fd8bc10cb30 2023-10-08T09:29:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.825+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc0795d0 con 0x7fd8bc10cb30 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.825+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc079ae0 con 0x7fd8bc10cb30 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.825+0000 7fd8b2ffd700 1 -- 172.21.15.49:0/2889743183 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8010e70 con 0x7fd8bc10cb30 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.830+0000 7fd8cfce8b80 1 Processor -- start 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.831+0000 7fd8cfce8b80 1 -- start start 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.831+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.831+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb58ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.831+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb59740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.831+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81faaf400 con 0x55e81fb592c0 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.832+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81faa0bc0 con 0x55e81fb57e50 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.832+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fa9e4e0 con 0x55e81fb587b0 2023-10-08T09:29:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.832+0000 7fd8b2ffd700 1 -- 172.21.15.49:0/2889743183 <== mon.2 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd8b801a440 con 0x7fd8bc10cb30 2023-10-08T09:29:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.832+0000 7fd8b2ffd700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd8a80a1680 0x7fd8a80a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.833+0000 7fd8b2ffd700 1 -- 172.21.15.49:0/2889743183 <== mon.2 v2:172.21.15.132:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd8b80ccef0 con 0x7fd8bc10cb30 2023-10-08T09:29:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb58270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43640/0 (socket says 172.21.15.49:43640) 2023-10-08T09:29:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c109a700 1 -- 172.21.15.49:0/3728619451 learned_addr learned my addr 172.21.15.49:0/3728619451 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c189b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb58ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c189b700 1 -- 172.21.15.49:0/3728619451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 msgr2=0x55e81fb58270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c189b700 1 --2- 172.21.15.49:0/3728619451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb58270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c189b700 1 -- 172.21.15.49:0/3728619451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 msgr2=0x55e81fb59740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c189b700 1 --2- 172.21.15.49:0/3728619451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb59740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c189b700 1 -- 172.21.15.49:0/3728619451 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81f8c4d80 con 0x55e81fb587b0 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c189b700 1 --2- 172.21.15.49:0/3728619451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb58ce0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0098c0 tx=0x7fd8ac005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a3d7074264e8c28 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.836+0000 7fd8c209c700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd8a80a1680 0x7fd8a80a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.837+0000 7fd8a7fff700 1 -- 172.21.15.49:0/3728619451 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac01c070 con 0x55e81fb587b0 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.837+0000 7fd8a7fff700 1 -- 172.21.15.49:0/3728619451 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8ac005110 con 0x55e81fb587b0 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.837+0000 7fd8a7fff700 1 -- 172.21.15.49:0/3728619451 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac003680 con 0x55e81fb587b0 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.838+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3728619451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 msgr2=0x55e81fb58ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.838+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3728619451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb58ce0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0098c0 tx=0x7fd8ac005800 comp rx=0 tx=0).stop 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.846+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3728619451 shutdown_connections 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.846+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3728619451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb59740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.846+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3728619451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb58270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.846+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3728619451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb58ce0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.846+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3728619451 >> 172.21.15.49:0/3728619451 conn(0x55e81fabb200 msgr2=0x55e81fb418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.847+0000 7fd8c209c700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd8a80a1680 0x7fd8a80a3b40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8011040 tx=0x7fd8b80092e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.864+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3728619451 shutdown_connections 2023-10-08T09:29:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.880+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3728619451 wait complete. 2023-10-08T09:29:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.880+0000 7fd8cfce8b80 1 -- start start 2023-10-08T09:29:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.880+0000 7fd8cfce8b80 1 Processor -- start 2023-10-08T09:29:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb57e50 0x55e81fb4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb587b0 0x55e81fb4e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb55a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fa99dd0 con 0x55e81fb592c0 2023-10-08T09:29:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fa366b0 con 0x55e81fb587b0 2023-10-08T09:29:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fa36350 con 0x55e81fb57e50 2023-10-08T09:29:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8cfce8b80 10 client.0 _ll_register_callbacks cb 0x55e81f8937a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb57e50 0x55e81fb4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb57e50 0x55e81fb4e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41066/0 (socket says 172.21.15.49:41066) 2023-10-08T09:29:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.892+0000 7fd8c109a700 1 -- 172.21.15.49:0/3373010324 learned_addr learned my addr 172.21.15.49:0/3373010324 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.893+0000 7fd8c189b700 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb587b0 0x55e81fb4e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.894+0000 7fd8c109a700 1 -- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb587b0 msgr2=0x55e81fb4e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.894+0000 7fd8c109a700 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb587b0 0x55e81fb4e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.894+0000 7fd8c109a700 1 -- 172.21.15.49:0/3373010324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 msgr2=0x55e81fb55a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.894+0000 7fd8c109a700 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb55a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.894+0000 7fd8c109a700 1 -- 172.21.15.49:0/3373010324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc10da70 con 0x55e81fb57e50 2023-10-08T09:29:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.895+0000 7fd8c109a700 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb57e50 0x55e81fb4e4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009eb0 tx=0x7fd8b4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8b417f78c6f453 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.895+0000 7fd8a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:29:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.895+0000 7fd8a57fa700 1 -- 172.21.15.49:0/3373010324 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4004bb0 con 0x55e81fb57e50 2023-10-08T09:29:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.895+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e81fa61e50 con 0x55e81fb57e50 2023-10-08T09:29:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.895+0000 7fd8a57fa700 1 -- 172.21.15.49:0/3373010324 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd8b4004d10 con 0x55e81fb57e50 2023-10-08T09:29:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.895+0000 7fd8a57fa700 1 -- 172.21.15.49:0/3373010324 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b4009d00 con 0x55e81fb57e50 2023-10-08T09:29:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.900+0000 7fd8cfce8b80 10 client.26761 fetch_fsmap learned FSMap version 41 2023-10-08T09:29:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.900+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e81f8adc00 con 0x55e81fb57e50 2023-10-08T09:29:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8a57fa700 1 -- 172.21.15.49:0/3373010324 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e81fa61e50 con 0x55e81fb57e50 2023-10-08T09:29:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8a57fa700 1 -- 172.21.15.49:0/3373010324 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd8b4014070 con 0x55e81fb57e50 2023-10-08T09:29:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8cfce8b80 10 client.26761 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:29:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8cfce8b80 10 client.26761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8cfce8b80 20 client.26761 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fb7e720 0x55e81fb80b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8cfce8b80 4 client.26761 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:29:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e81fa92db0 con 0x55e81fb7e720 2023-10-08T09:29:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.917+0000 7fd8c189b700 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fb7e720 0x55e81fb80b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.918+0000 7fd8c189b700 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fb7e720 0x55e81fb80b40 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3930a0646a228883 server_cookie=7cb67e2d6f5389d2 in_seq=0 out_seq=0 2023-10-08T09:29:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.928+0000 7fd8a57fa700 10 client.26761 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.938+0000 7fd8a57fa700 1 -- 172.21.15.49:0/3373010324 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e81fa92db0 con 0x55e81fb7e720 2023-10-08T09:29:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.938+0000 7fd8a57fa700 10 client.26761 handle_command_reply: tid=0 2023-10-08T09:29:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.938+0000 7fd8cfce8b80 1 client.26761 shutdown 2023-10-08T09:29:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.951+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb57e50 msgr2=0x55e81fb4e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.951+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb57e50 0x55e81fb4e4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009eb0 tx=0x7fd8b4009f90 comp rx=0 tx=0).stop 2023-10-08T09:29:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.968+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 shutdown_connections 2023-10-08T09:29:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.968+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fb7e720 0x55e81fb80b40 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.968+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb57e50 0x55e81fb4e4a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.968+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb55a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.972 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.968+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/3373010324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb587b0 0x55e81fb4e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.968+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 >> 172.21.15.49:0/3373010324 conn(0x55e81fabb200 msgr2=0x55e81fb41010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:08.973 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.968+0000 7fd8a57fa700 0 client.26761 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.983+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 shutdown_connections 2023-10-08T09:29:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.985+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/3373010324 wait complete. 2023-10-08T09:29:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:08.998+0000 7fd8cfce8b80 20 client.26761 trim_cache size 0 max 16384 2023-10-08T09:29:09.022 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.021+0000 7fd8cfce8b80 1 Processor -- start 2023-10-08T09:29:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.033+0000 7fd8cfce8b80 1 -- start start 2023-10-08T09:29:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.033+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.033+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb47b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.033+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.033+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81f91a8c0 con 0x55e81fb592c0 2023-10-08T09:29:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.033+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fb58b90 con 0x55e81fb57e50 2023-10-08T09:29:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.033+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fa61e50 con 0x55e81fb587b0 2023-10-08T09:29:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43654/0 (socket says 172.21.15.49:43654) 2023-10-08T09:29:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 -- 172.21.15.49:0/786789589 learned_addr learned my addr 172.21.15.49:0/786789589 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 -- 172.21.15.49:0/786789589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 msgr2=0x55e81fb47b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb47b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 -- 172.21.15.49:0/786789589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 msgr2=0x55e81fb41790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb41790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.034+0000 7fd8c109a700 1 -- 172.21.15.49:0/786789589 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e81f8eb1f0 con 0x55e81fb57e50 2023-10-08T09:29:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.035+0000 7fd8c109a700 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb596a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009040 tx=0x7fd8b4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6acf5904faab5d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.037+0000 7fd8b0d82700 1 -- 172.21.15.49:0/786789589 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4007ba0 con 0x55e81fb57e50 2023-10-08T09:29:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.037+0000 7fd8b0d82700 1 -- 172.21.15.49:0/786789589 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8b4007d00 con 0x55e81fb57e50 2023-10-08T09:29:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.037+0000 7fd8b0d82700 1 -- 172.21.15.49:0/786789589 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400f9c0 con 0x55e81fb57e50 2023-10-08T09:29:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.038+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/786789589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 msgr2=0x55e81fb596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.038+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb596a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009040 tx=0x7fd8b4004ad0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.045+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/786789589 shutdown_connections 2023-10-08T09:29:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.045+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb592c0 0x55e81fb41790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.045+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fb596a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.045+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/786789589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb587b0 0x55e81fb47b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.045+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/786789589 >> 172.21.15.49:0/786789589 conn(0x55e81fabb200 msgr2=0x55e81fb4d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.047+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/786789589 shutdown_connections 2023-10-08T09:29:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.047+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/786789589 wait complete. 2023-10-08T09:29:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.047+0000 7fd8cfce8b80 1 -- start start 2023-10-08T09:29:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.047+0000 7fd8cfce8b80 1 Processor -- start 2023-10-08T09:29:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fbcc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb587b0 0x55e81fbcea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8cfce8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb592c0 0x55e81fbd0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fb480e0 con 0x55e81fb587b0 2023-10-08T09:29:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fbd14f0 con 0x55e81fb57e50 2023-10-08T09:29:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8cfce8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e81fbd1660 con 0x55e81fb592c0 2023-10-08T09:29:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8cfce8b80 10 client.0 _ll_register_callbacks cb 0x7fd8b4011570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fbcc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8c109a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fbcc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43660/0 (socket says 172.21.15.49:43660) 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8c109a700 1 -- 172.21.15.49:0/2925755002 learned_addr learned my addr 172.21.15.49:0/2925755002 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8c109a700 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb592c0 0x55e81fbd0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.048+0000 7fd8c189b700 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb587b0 0x55e81fbcea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8c189b700 1 -- 172.21.15.49:0/2925755002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb592c0 msgr2=0x55e81fbd0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8c189b700 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb592c0 0x55e81fbd0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8c189b700 1 -- 172.21.15.49:0/2925755002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 msgr2=0x55e81fbcc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8c189b700 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fbcc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8c189b700 1 -- 172.21.15.49:0/2925755002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e81f8adc00 con 0x55e81fb587b0 2023-10-08T09:29:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8c189b700 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb587b0 0x55e81fbcea60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac000c00 tx=0x7fd8ac005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7a7166816e0622 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8a57fa700 1 -- 172.21.15.49:0/2925755002 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac026040 con 0x55e81fb587b0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.049+0000 7fd8a57fa700 1 -- 172.21.15.49:0/2925755002 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fd8ac00b810 con 0x55e81fb587b0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.050+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8ac005110 con 0x55e81fb587b0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.050+0000 7fd8a57fa700 1 -- 172.21.15.49:0/2925755002 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8ac01c040 con 0x55e81fb587b0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.050+0000 7fd8cfce8b80 10 client.17136 fetch_fsmap learned FSMap version 41 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.050+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e81f8c3250 con 0x55e81fb587b0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8a57fa700 1 -- 172.21.15.49:0/2925755002 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8ac005110 con 0x55e81fb587b0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8a57fa700 1 -- 172.21.15.49:0/2925755002 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd8ac015020 con 0x55e81fb587b0 2023-10-08T09:29:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8cfce8b80 10 client.17136 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:29:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8cfce8b80 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8cfce8b80 20 client.17136 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fbd2be0 0x55e81fbd5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8cfce8b80 4 client.17136 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:29:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.051+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e81fa92ad0 con 0x55e81fbd2be0 2023-10-08T09:29:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.056+0000 7fd8c109a700 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fbd2be0 0x55e81fbd5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.096+0000 7fd8c109a700 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fbd2be0 0x55e81fbd5000 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d371a8424be280 server_cookie=66050aa291e7eb5d in_seq=0 out_seq=0 2023-10-08T09:29:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.096+0000 7fd8a57fa700 10 client.17136 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:09.186 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.186+0000 7fd8a57fa700 1 -- 172.21.15.49:0/2925755002 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e81fa92ad0 con 0x55e81fbd2be0 2023-10-08T09:29:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.186+0000 7fd8a57fa700 10 client.17136 handle_command_reply: tid=0 2023-10-08T09:29:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.186+0000 7fd8cfce8b80 1 client.17136 shutdown 2023-10-08T09:29:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.186+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb587b0 msgr2=0x55e81fbcea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.186+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb587b0 0x55e81fbcea60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac000c00 tx=0x7fd8ac005070 comp rx=0 tx=0).stop 2023-10-08T09:29:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.188+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 shutdown_connections 2023-10-08T09:29:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.188+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55e81fbd2be0 0x55e81fbd5000 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.188+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55e81fb592c0 0x55e81fbd0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.188+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55e81fb57e50 0x55e81fbcc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.188+0000 7fd8a57fa700 0 client.17136 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.188+0000 7fd8cfce8b80 1 --2- 172.21.15.49:0/2925755002 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55e81fb587b0 0x55e81fbcea60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.188+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 >> 172.21.15.49:0/2925755002 conn(0x55e81fabb200 msgr2=0x55e81fb48e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.189+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 shutdown_connections 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.189+0000 7fd8cfce8b80 1 -- 172.21.15.49:0/2925755002 wait complete. 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.189+0000 7fd8cfce8b80 20 client.17136 trim_cache size 0 max 16384 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:29:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:29:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:29:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.189+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd8a80a1680 msgr2=0x7fd8a80a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.189+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd8a80a1680 0x7fd8a80a3b40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8011040 tx=0x7fd8b80092e0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.189+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 msgr2=0x7fd8bc078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.189+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 0x7fd8bc078a50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800b4a0 tx=0x7fd8b800b860 comp rx=0 tx=0).stop 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 shutdown_connections 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fd8a80a1680 0x7fd8a80a3b40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8bc1077f0 0x7fd8bc1a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8bc10cb30 0x7fd8bc078a50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 --2- 172.21.15.49:0/2889743183 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd8bc10a160 0x7fd8bc1a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 >> 172.21.15.49:0/2889743183 conn(0x7fd8bc06d770 msgr2=0x7fd8bc10b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 shutdown_connections 2023-10-08T09:29:09.192 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:09.190+0000 7fd8c3aff700 1 -- 172.21.15.49:0/2889743183 wait complete. 2023-10-08T09:29:09.304 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:09 smithi132 ceph-mon[103389]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 482 op/s 2023-10-08T09:29:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 825.0 s, 450.4 tps, lat 233.514 ms stddev 306.031 2023-10-08T09:29:09.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:09 smithi049 ceph-mon[99485]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 482 op/s 2023-10-08T09:29:09.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:09 smithi184 ceph-mon[103675]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.0 MiB/s wr, 482 op/s 2023-10-08T09:29:12.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:11 smithi184 ceph-mon[103675]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.6 MiB/s wr, 513 op/s 2023-10-08T09:29:12.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:11 smithi132 ceph-mon[103389]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.6 MiB/s wr, 513 op/s 2023-10-08T09:29:12.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:11 smithi049 ceph-mon[99485]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.6 MiB/s wr, 513 op/s 2023-10-08T09:29:13.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:13 smithi184 ceph-mon[103675]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 531 op/s 2023-10-08T09:29:13.552 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:13 smithi132 ceph-mon[103389]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 531 op/s 2023-10-08T09:29:13.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:13 smithi049 ceph-mon[99485]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 531 op/s 2023-10-08T09:29:14.313 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 830.0 s, 477.1 tps, lat 210.167 ms stddev 254.421 2023-10-08T09:29:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:15 smithi184 ceph-mon[103675]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.6 MiB/s wr, 484 op/s 2023-10-08T09:29:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:15 smithi132 ceph-mon[103389]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.6 MiB/s wr, 484 op/s 2023-10-08T09:29:16.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:15 smithi049 ceph-mon[99485]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.6 MiB/s wr, 484 op/s 2023-10-08T09:29:17.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:16 smithi184 ceph-mon[103675]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 MiB/s wr, 499 op/s 2023-10-08T09:29:17.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:16 smithi132 ceph-mon[103389]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 MiB/s wr, 499 op/s 2023-10-08T09:29:17.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:16 smithi049 ceph-mon[99485]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.4 MiB/s wr, 499 op/s 2023-10-08T09:29:19.313 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 835.0 s, 565.0 tps, lat 175.396 ms stddev 188.317 2023-10-08T09:29:20.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:19 smithi184 ceph-mon[103675]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 500 op/s 2023-10-08T09:29:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:19 smithi132 ceph-mon[103389]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 500 op/s 2023-10-08T09:29:20.387 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:19 smithi049 ceph-mon[99485]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.9 MiB/s wr, 500 op/s 2023-10-08T09:29:22.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:21 smithi184 ceph-mon[103675]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.0 MiB/s wr, 505 op/s 2023-10-08T09:29:22.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:21 smithi132 ceph-mon[103389]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.0 MiB/s wr, 505 op/s 2023-10-08T09:29:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 -- 172.21.15.49:0/1014425220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f60100840 msgr2=0x7f3f60100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 --2- 172.21.15.49:0/1014425220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f60100840 0x7f3f60100ca0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3f540099c0 tx=0x7f3f54009cd0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 -- 172.21.15.49:0/1014425220 shutdown_connections 2023-10-08T09:29:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 --2- 172.21.15.49:0/1014425220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f600ff650 0x7f3f600ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 --2- 172.21.15.49:0/1014425220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f601011e0 0x7f3f60107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 --2- 172.21.15.49:0/1014425220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f60100840 0x7f3f60100ca0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 -- 172.21.15.49:0/1014425220 >> 172.21.15.49:0/1014425220 conn(0x7f3f600fabe0 msgr2=0x7f3f600fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 -- 172.21.15.49:0/1014425220 shutdown_connections 2023-10-08T09:29:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.332+0000 7f3f66387700 1 -- 172.21.15.49:0/1014425220 wait complete. 2023-10-08T09:29:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 Processor -- start 2023-10-08T09:29:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 -- start start 2023-10-08T09:29:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f600ff650 0x7f3f60103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f60100840 0x7f3f60101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f601011e0 0x7f3f60102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f60102900 con 0x7f3f600ff650 2023-10-08T09:29:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f60102a70 con 0x7f3f601011e0 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f66387700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601a0710 con 0x7f3f60100840 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f5ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f600ff650 0x7f3f60103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f5f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f60100840 0x7f3f60101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f5ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f600ff650 0x7f3f60103590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54058/0 (socket says 172.21.15.49:54058) 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f5ffff700 1 -- 172.21.15.49:0/3580361046 learned_addr learned my addr 172.21.15.49:0/3580361046 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.333+0000 7f3f64924700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f601011e0 0x7f3f60102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3580361046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f601011e0 msgr2=0x7f3f60102160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f601011e0 0x7f3f60102160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f600ff650 msgr2=0x7f3f60103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f600ff650 0x7f3f60103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5f7fe700 1 -- 172.21.15.49:0/3580361046 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540095f0 con 0x7f3f60100840 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f600ff650 0x7f3f60103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f60100840 0x7f3f60101c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009fd0 tx=0x7f3f5400f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5d7fa700 1 -- 172.21.15.49:0/3580361046 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5401d070 con 0x7f3f60100840 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f601a0940 con 0x7f3f60100840 2023-10-08T09:29:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5d7fa700 1 -- 172.21.15.49:0/3580361046 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f540059a0 con 0x7f3f60100840 2023-10-08T09:29:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f5d7fa700 1 -- 172.21.15.49:0/3580361046 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54004970 con 0x7f3f60100840 2023-10-08T09:29:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.334+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f601a0d50 con 0x7f3f60100840 2023-10-08T09:29:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.337+0000 7f3f5d7fa700 1 -- 172.21.15.49:0/3580361046 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3f54005190 con 0x7f3f60100840 2023-10-08T09:29:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f5d7fa700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3f480a1480 0x7f3f480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 Processor -- start 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f5d7fa700 1 -- 172.21.15.49:0/3580361046 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3f540cd950 con 0x7f3f60100840 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 -- start start 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d27a0 0x55d0d52d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d32b0 0x55d0d52d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d523a400 con 0x55d0d52d27a0 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d522bbc0 con 0x55d0d52d32b0 2023-10-08T09:29:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.338+0000 7f3f72570b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d52294e0 con 0x55d0d52d1e40 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f5ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f5ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41100/0 (socket says 172.21.15.49:41100) 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f5ffff700 1 -- 172.21.15.49:0/2510208657 learned_addr learned my addr 172.21.15.49:0/2510208657 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d27a0 0x55d0d52d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f64924700 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d32b0 0x55d0d52d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f64924700 1 -- 172.21.15.49:0/2510208657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 msgr2=0x55d0d52d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f64924700 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f64924700 1 -- 172.21.15.49:0/2510208657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d27a0 msgr2=0x55d0d52d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f64924700 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d27a0 0x55d0d52d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f64924700 1 -- 172.21.15.49:0/2510208657 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d504fd80 con 0x55d0d52d32b0 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d27a0 0x55d0d52d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:29:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f64924700 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d32b0 0x55d0d52d3730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f58011580 tx=0x7f3f58009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9843bce8ad764cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.339+0000 7f3f467fc700 1 -- 172.21.15.49:0/2510208657 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800e040 con 0x55d0d52d32b0 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f467fc700 1 -- 172.21.15.49:0/2510208657 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f5800f040 con 0x55d0d52d32b0 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f467fc700 1 -- 172.21.15.49:0/2510208657 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58004450 con 0x55d0d52d32b0 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 -- 172.21.15.49:0/2510208657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d32b0 msgr2=0x55d0d52d3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d32b0 0x55d0d52d3730 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f58011580 tx=0x7f3f58009920 comp rx=0 tx=0).stop 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 -- 172.21.15.49:0/2510208657 shutdown_connections 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d32b0 0x55d0d52d3730 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 --2- 172.21.15.49:0/2510208657 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d27a0 0x55d0d52d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 -- 172.21.15.49:0/2510208657 >> 172.21.15.49:0/2510208657 conn(0x55d0d5246200 msgr2=0x55d0d52cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 -- 172.21.15.49:0/2510208657 shutdown_connections 2023-10-08T09:29:22.353 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.340+0000 7f3f72570b80 1 -- 172.21.15.49:0/2510208657 wait complete. 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3f480a1480 0x7f3f480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 -- start start 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 Processor -- start 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3f480a1480 0x7f3f480a3940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009880 tx=0x7f3f500047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d5224dd0 con 0x55d0d52d1e40 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d51c16b0 con 0x55d0d52d27a0 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d51c1350 con 0x55d0d52d32b0 2023-10-08T09:29:22.354 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f72570b80 10 client.0 _ll_register_callbacks cb 0x55d0d501e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f5ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52db890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f5f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f5ffff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52db890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54074/0 (socket says 172.21.15.49:54074) 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f5ffff700 1 -- 172.21.15.49:0/3278118530 learned_addr learned my addr 172.21.15.49:0/3278118530 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.341+0000 7f3f5f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43686/0 (socket says 172.21.15.49:43686) 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.342+0000 7f3f64924700 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.342+0000 7f3f5ffff700 1 -- 172.21.15.49:0/3278118530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 msgr2=0x55d0d52e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.342+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52e2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.342+0000 7f3f5ffff700 1 -- 172.21.15.49:0/3278118530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 msgr2=0x55d0d52dbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.342+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52dbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.342+0000 7f3f5ffff700 1 -- 172.21.15.49:0/3278118530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f601a0d50 con 0x55d0d52d1e40 2023-10-08T09:29:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.343+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52db890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3f5001eda0 tx=0x7f3f50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f468e97ad5d13b65 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.343+0000 7f3f3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.343+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/3278118530 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50023070 con 0x55d0d52d1e40 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.344+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/3278118530 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3f5001ee30 con 0x55d0d52d1e40 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.344+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0d51ece50 con 0x55d0d52d1e40 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.344+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/3278118530 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f50016040 con 0x55d0d52d1e40 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 10 client.17148 fetch_fsmap learned FSMap version 41 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f3f58011230 con 0x55d0d52d1e40 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/3278118530 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d0d51ece50 con 0x55d0d52d1e40 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/3278118530 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3f50027c10 con 0x55d0d52d1e40 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 10 client.17148 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 20 client.17148 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d52e7620 0x55d0d52e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 4 client.17148 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0d521ddb0 con 0x55d0d52e7620 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.345+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d52e7620 0x55d0d52e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.346+0000 7f3f3f7fe700 10 client.17148 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.346+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d52e7620 0x55d0d52e9a40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89d698b6a86b5fa server_cookie=e6760ecb07b5e636 in_seq=0 out_seq=0 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.346+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/3278118530 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d0d521ddb0 con 0x55d0d52e7620 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.346+0000 7f3f3f7fe700 10 client.17148 handle_command_reply: tid=0 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.346+0000 7f3f72570b80 1 client.17148 shutdown 2023-10-08T09:29:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 msgr2=0x55d0d52db890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52db890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3f5001eda0 tx=0x7f3f50004750 comp rx=0 tx=0).stop 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 shutdown_connections 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d52e7620 0x55d0d52e9a40 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52db890 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 --2- 172.21.15.49:0/3278118530 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52dbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 >> 172.21.15.49:0/3278118530 conn(0x55d0d5246200 msgr2=0x55d0d52cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f3f7fe700 0 client.17148 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 shutdown_connections 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 1 -- 172.21.15.49:0/3278118530 wait complete. 2023-10-08T09:29:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.347+0000 7f3f72570b80 20 client.17148 trim_cache size 0 max 16384 2023-10-08T09:29:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 Processor -- start 2023-10-08T09:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 -- start start 2023-10-08T09:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d50a58c0 con 0x55d0d52d1e40 2023-10-08T09:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d51ece50 con 0x55d0d52d27a0 2023-10-08T09:29:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.365+0000 7f3f72570b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d52d3130 con 0x55d0d52d32b0 2023-10-08T09:29:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f5f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52d9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41116/0 (socket says 172.21.15.49:41116) 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f5f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43694/0 (socket says 172.21.15.49:43694) 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 -- 172.21.15.49:0/68155572 learned_addr learned my addr 172.21.15.49:0/68155572 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 -- 172.21.15.49:0/68155572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 msgr2=0x55d0d52e2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 -- 172.21.15.49:0/68155572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 msgr2=0x55d0d52d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 -- 172.21.15.49:0/68155572 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d5038c00 con 0x55d0d52d32b0 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f5f7fe700 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f64924700 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52d9890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3f58011520 tx=0x7f3f58011ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d66b4c0ae43320 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f46ffd700 1 -- 172.21.15.49:0/68155572 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800e040 con 0x55d0d52d32b0 2023-10-08T09:29:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.366+0000 7f3f46ffd700 1 -- 172.21.15.49:0/68155572 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f58003ec0 con 0x55d0d52d32b0 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f46ffd700 1 -- 172.21.15.49:0/68155572 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58025430 con 0x55d0d52d32b0 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 -- 172.21.15.49:0/68155572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 msgr2=0x55d0d52d9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52d9890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3f58011520 tx=0x7f3f58011ea0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 -- 172.21.15.49:0/68155572 shutdown_connections 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d32b0 0x55d0d52d9890 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d1e40 0x55d0d52d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 --2- 172.21.15.49:0/68155572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 -- 172.21.15.49:0/68155572 >> 172.21.15.49:0/68155572 conn(0x55d0d5246200 msgr2=0x55d0d52dadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.367+0000 7f3f72570b80 1 -- 172.21.15.49:0/68155572 shutdown_connections 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 -- 172.21.15.49:0/68155572 wait complete. 2023-10-08T09:29:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 -- start start 2023-10-08T09:29:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 Processor -- start 2023-10-08T09:29:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52e0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d32b0 0x55d0d52e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d52e1f60 con 0x55d0d52d32b0 2023-10-08T09:29:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d52e20d0 con 0x55d0d52d27a0 2023-10-08T09:29:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d52e2290 con 0x55d0d52d1e40 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.368+0000 7f3f72570b80 10 client.0 _ll_register_callbacks cb 0x7f3f5001e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d32b0 0x55d0d52e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d32b0 0x55d0d52e1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54086/0 (socket says 172.21.15.49:54086) 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 -- 172.21.15.49:0/300645318 learned_addr learned my addr 172.21.15.49:0/300645318 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 -- 172.21.15.49:0/300645318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 msgr2=0x55d0d52e0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 -- 172.21.15.49:0/300645318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 msgr2=0x55d0d52e0a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e0a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 -- 172.21.15.49:0/300645318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0d504fd80 con 0x55d0d52d32b0 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.369+0000 7f3f64924700 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d32b0 0x55d0d52e1a20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3f58028040 tx=0x7f3f580098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95abcae62ca90d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.370+0000 7f3f3f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.370+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/300645318 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800e050 con 0x55d0d52d32b0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.370+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/300645318 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3f58007910 con 0x55d0d52d32b0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.370+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f5800f040 con 0x55d0d52d32b0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.370+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/300645318 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f58007e70 con 0x55d0d52d32b0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.370+0000 7f3f72570b80 10 client.17154 fetch_fsmap learned FSMap version 41 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.370+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d0d50761f0 con 0x55d0d52d32b0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.371+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/300645318 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f5800f040 con 0x55d0d52d32b0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.371+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/300645318 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3f5801b020 con 0x55d0d52d32b0 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.372+0000 7f3f72570b80 10 client.17154 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.372+0000 7f3f72570b80 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.372+0000 7f3f72570b80 20 client.17154 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.372+0000 7f3f72570b80 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d5344240 0x55d0d5346650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.372+0000 7f3f72570b80 4 client.17154 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:29:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.372+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d0d521dad0 con 0x55d0d5344240 2023-10-08T09:29:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.385+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d5344240 0x55d0d5346650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:21 smithi049 ceph-mon[99485]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.0 MiB/s wr, 505 op/s 2023-10-08T09:29:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.427+0000 7f3f5ffff700 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d5344240 0x55d0d5346650 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212b85a7dbf9fe3d server_cookie=e7be31635043af76 in_seq=0 out_seq=0 2023-10-08T09:29:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.427+0000 7f3f3f7fe700 10 client.17154 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:22.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:29:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.530+0000 7f3f3f7fe700 1 -- 172.21.15.49:0/300645318 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d0d521dad0 con 0x55d0d5344240 2023-10-08T09:29:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.530+0000 7f3f3f7fe700 10 client.17154 handle_command_reply: tid=0 2023-10-08T09:29:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.530+0000 7f3f72570b80 1 client.17154 shutdown 2023-10-08T09:29:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.531+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d32b0 msgr2=0x55d0d52e1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.531+0000 7f3f72570b80 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d32b0 0x55d0d52e1a20 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3f58028040 tx=0x7f3f580098e0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.535 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.532+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 shutdown_connections 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.532+0000 7f3f72570b80 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55d0d5344240 0x55d0d5346650 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.533+0000 7f3f72570b80 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55d0d52d1e40 0x55d0d52e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.533+0000 7f3f72570b80 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55d0d52d32b0 0x55d0d52e1a20 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.533+0000 7f3f72570b80 1 --2- 172.21.15.49:0/300645318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55d0d52d27a0 0x55d0d52e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.533+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 >> 172.21.15.49:0/300645318 conn(0x55d0d5246200 msgr2=0x55d0d52db640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.533+0000 7f3f3f7fe700 0 client.17154 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.533+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 shutdown_connections 2023-10-08T09:29:22.536 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.533+0000 7f3f72570b80 1 -- 172.21.15.49:0/300645318 wait complete. 2023-10-08T09:29:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.534+0000 7f3f72570b80 20 client.17154 trim_cache size 0 max 16384 2023-10-08T09:29:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:29:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:29:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:29:22.537 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:29:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3f480a1480 msgr2=0x7f3f480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3f480a1480 0x7f3f480a3940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009880 tx=0x7f3f500047c0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f60100840 msgr2=0x7f3f60101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.538 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f60100840 0x7f3f60101c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009fd0 tx=0x7f3f5400f670 comp rx=0 tx=0).stop 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 shutdown_connections 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3f480a1480 0x7f3f480a3940 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f601011e0 0x7f3f60102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3f600ff650 0x7f3f60103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 --2- 172.21.15.49:0/3580361046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3f60100840 0x7f3f60101c20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.535+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 >> 172.21.15.49:0/3580361046 conn(0x7f3f600fabe0 msgr2=0x7f3f600fb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.536+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 shutdown_connections 2023-10-08T09:29:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:22.536+0000 7f3f66387700 1 -- 172.21.15.49:0/3580361046 wait complete. 2023-10-08T09:29:22.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.873+0000 7fe7a9709700 1 -- 172.21.15.49:0/961899494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a41077f0 msgr2=0x7fe7a4109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.881 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.873+0000 7fe7a9709700 1 --2- 172.21.15.49:0/961899494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a41077f0 0x7fe7a4109c20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe78c01a9a0 tx=0x7fe78c01acb0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.883+0000 7fe7a9709700 1 -- 172.21.15.49:0/961899494 shutdown_connections 2023-10-08T09:29:22.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.883+0000 7fe7a9709700 1 --2- 172.21.15.49:0/961899494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410cb30 0x7fe7a410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.883+0000 7fe7a9709700 1 --2- 172.21.15.49:0/961899494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a41077f0 0x7fe7a4109c20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.883+0000 7fe7a9709700 1 --2- 172.21.15.49:0/961899494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a410a160 0x7fe7a410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.886 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.883+0000 7fe7a9709700 1 -- 172.21.15.49:0/961899494 >> 172.21.15.49:0/961899494 conn(0x7fe7a406d790 msgr2=0x7fe7a406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.887 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.885+0000 7fe7a9709700 1 -- 172.21.15.49:0/961899494 shutdown_connections 2023-10-08T09:29:22.896 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.895+0000 7fe7a9709700 1 -- 172.21.15.49:0/961899494 wait complete. 2023-10-08T09:29:22.898 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.896+0000 7fe7a9709700 1 Processor -- start 2023-10-08T09:29:22.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a9709700 1 -- start start 2023-10-08T09:29:22.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a9709700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a41077f0 0x7fe7a419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a9709700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410a160 0x7fe7a419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a9709700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a410cb30 0x7fe7a41a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a9709700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41a49b0 con 0x7fe7a410cb30 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a9709700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41a4af0 con 0x7fe7a41077f0 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a9709700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41a4c80 con 0x7fe7a410a160 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a27fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410a160 0x7fe7a419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a27fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410a160 0x7fe7a419d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41134/0 (socket says 172.21.15.49:41134) 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.907+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3231178887 learned_addr learned my addr 172.21.15.49:0/3231178887 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.913+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3231178887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a41077f0 msgr2=0x7fe7a419ce80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.913+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a41077f0 0x7fe7a419ce80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.913+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a410cb30 msgr2=0x7fe7a41a43e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:29:22.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.913+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a410cb30 0x7fe7a41a43e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.913+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3231178887 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c01a600 con 0x7fe7a410a160 2023-10-08T09:29:22.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.925+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410a160 0x7fe7a419d3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe79400b550 tx=0x7fe79400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.935 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.932+0000 7fe79bfff700 1 -- 172.21.15.49:0/3231178887 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794011830 con 0x7fe7a410a160 2023-10-08T09:29:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.932+0000 7fe79bfff700 1 -- 172.21.15.49:0/3231178887 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe794011e50 con 0x7fe7a410a160 2023-10-08T09:29:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.932+0000 7fe79bfff700 1 -- 172.21.15.49:0/3231178887 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79400f2e0 con 0x7fe7a410a160 2023-10-08T09:29:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.933+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a41a4f60 con 0x7fe7a410a160 2023-10-08T09:29:22.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.933+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a41a5330 con 0x7fe7a410a160 2023-10-08T09:29:22.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.941+0000 7fe7b58f2b80 1 Processor -- start 2023-10-08T09:29:22.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.947+0000 7fe79bfff700 1 -- 172.21.15.49:0/3231178887 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe794010580 con 0x7fe7a410a160 2023-10-08T09:29:22.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.948+0000 7fe79bfff700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7900a15d0 0x7fe7900a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.948+0000 7fe79bfff700 1 -- 172.21.15.49:0/3231178887 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe7940ccb00 con 0x7fe7a410a160 2023-10-08T09:29:22.950 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.949+0000 7fe7a37fe700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7900a15d0 0x7fe7900a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.949+0000 7fe7b58f2b80 1 -- start start 2023-10-08T09:29:22.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e4122060 0x5590e4122480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e41229c0 0x5590e4123270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4123c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e408a460 con 0x5590e4122060 2023-10-08T09:29:22.951 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e407bc20 con 0x5590e4123800 2023-10-08T09:29:22.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e4079540 con 0x5590e41229c0 2023-10-08T09:29:22.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7a2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e4122060 0x5590e4122480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7a2ffd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e4122060 0x5590e4122480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54104/0 (socket says 172.21.15.49:54104) 2023-10-08T09:29:22.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7a2ffd700 1 -- 172.21.15.49:0/3902682906 learned_addr learned my addr 172.21.15.49:0/3902682906 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.952 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7a37fe700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7900a15d0 0x7fe7900a3a90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00ab30 tx=0x7fe79c009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7a37fe700 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4123c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.950+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e41229c0 0x5590e4123270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.951+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3902682906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 msgr2=0x5590e4123c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.951+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4123c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.951+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3902682906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e4122060 msgr2=0x5590e4122480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.951+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e4122060 0x5590e4122480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.953 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.951+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3902682906 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590e3e9fde0 con 0x5590e41229c0 2023-10-08T09:29:22.954 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.952+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e41229c0 0x5590e4123270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe7940ca270 tx=0x7fe794095e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17dfee497daad7e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.953+0000 7fe7997fa700 1 -- 172.21.15.49:0/3902682906 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794099a90 con 0x5590e41229c0 2023-10-08T09:29:22.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.953+0000 7fe7997fa700 1 -- 172.21.15.49:0/3902682906 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79409dbe0 con 0x5590e41229c0 2023-10-08T09:29:22.955 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.953+0000 7fe7997fa700 1 -- 172.21.15.49:0/3902682906 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940991e0 con 0x5590e41229c0 2023-10-08T09:29:22.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.954+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3902682906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e41229c0 msgr2=0x5590e4123270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.956 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.954+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e41229c0 0x5590e4123270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe7940ca270 tx=0x7fe794095e80 comp rx=0 tx=0).stop 2023-10-08T09:29:22.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.954+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3902682906 shutdown_connections 2023-10-08T09:29:22.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.954+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4123c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.954+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e4122060 0x5590e4122480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.954+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3902682906 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e41229c0 0x5590e4123270 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.957 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.954+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3902682906 >> 172.21.15.49:0/3902682906 conn(0x5590e4096260 msgr2=0x5590e411ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.955+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3902682906 shutdown_connections 2023-10-08T09:29:22.958 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.955+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3902682906 wait complete. 2023-10-08T09:29:22.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.955+0000 7fe7b58f2b80 1 -- start start 2023-10-08T09:29:22.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.955+0000 7fe7b58f2b80 1 Processor -- start 2023-10-08T09:29:22.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e412bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e412c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.959 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e4133540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e4074e30 con 0x5590e41229c0 2023-10-08T09:29:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e4011710 con 0x5590e4122060 2023-10-08T09:29:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e40113b0 con 0x5590e4123800 2023-10-08T09:29:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7b58f2b80 10 client.0 _ll_register_callbacks cb 0x5590e3e6e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:22.960 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7a37fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e4133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7a37fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e4133540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41144/0 (socket says 172.21.15.49:41144) 2023-10-08T09:29:22.961 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.956+0000 7fe7a37fe700 1 -- 172.21.15.49:0/1561252369 learned_addr learned my addr 172.21.15.49:0/1561252369 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:22.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.958+0000 7fe7a2ffd700 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e412bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.959+0000 7fe7a2ffd700 1 -- 172.21.15.49:0/1561252369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 msgr2=0x5590e4133540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.962 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.959+0000 7fe7a2ffd700 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e4133540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.959+0000 7fe7a2ffd700 1 -- 172.21.15.49:0/1561252369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 msgr2=0x5590e412c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.959+0000 7fe7a2ffd700 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e412c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.959+0000 7fe7a2ffd700 1 -- 172.21.15.49:0/1561252369 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a41a5330 con 0x5590e4122060 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.960+0000 7fe7a2ffd700 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e412bfc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe78c003820 tx=0x7fe78c01c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d377a79f5b78d8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.961+0000 7fe78a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.961+0000 7fe78a7fc700 1 -- 172.21.15.49:0/1561252369 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c01cee0 con 0x5590e4122060 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.961+0000 7fe78a7fc700 1 -- 172.21.15.49:0/1561252369 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe78c005240 con 0x5590e4122060 2023-10-08T09:29:22.963 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.961+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590e403ceb0 con 0x5590e4122060 2023-10-08T09:29:22.965 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.962+0000 7fe78a7fc700 1 -- 172.21.15.49:0/1561252369 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe78c005400 con 0x5590e4122060 2023-10-08T09:29:22.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 10 client.26729 fetch_fsmap learned FSMap version 41 2023-10-08T09:29:22.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5590e3e88c60 con 0x5590e4122060 2023-10-08T09:29:22.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe78a7fc700 1 -- 172.21.15.49:0/1561252369 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590e403ceb0 con 0x5590e4122060 2023-10-08T09:29:22.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe78a7fc700 1 -- 172.21.15.49:0/1561252369 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe78c0039f0 con 0x5590e4122060 2023-10-08T09:29:22.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 10 client.26729 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:29:22.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:22.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 20 client.26729 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:22.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e4159ea0 0x5590e415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:22.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 4 client.26729 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:29:22.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.967+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590e406de10 con 0x5590e4159ea0 2023-10-08T09:29:22.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.968+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e4159ea0 0x5590e415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:22.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.968+0000 7fe78a7fc700 10 client.26729 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:22.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.968+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e4159ea0 0x5590e415c2c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19dae9d4feeff2be server_cookie=3cd321871f30c95c in_seq=0 out_seq=0 2023-10-08T09:29:22.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.969+0000 7fe78a7fc700 1 -- 172.21.15.49:0/1561252369 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5590e406de10 con 0x5590e4159ea0 2023-10-08T09:29:22.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.969+0000 7fe78a7fc700 10 client.26729 handle_command_reply: tid=0 2023-10-08T09:29:22.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.969+0000 7fe7b58f2b80 1 client.26729 shutdown 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.969+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 msgr2=0x5590e412bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e412bfc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe78c003820 tx=0x7fe78c01c9f0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 shutdown_connections 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e4159ea0 0x5590e415c2c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e4133540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e412bfc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/1561252369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e412c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 >> 172.21.15.49:0/1561252369 conn(0x5590e4096260 msgr2=0x5590e411c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:22.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe78a7fc700 0 client.26729 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 shutdown_connections 2023-10-08T09:29:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/1561252369 wait complete. 2023-10-08T09:29:22.973 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:22.970+0000 7fe7b58f2b80 20 client.26729 trim_cache size 0 max 16384 2023-10-08T09:29:23.005 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.003+0000 7fe7b58f2b80 1 Processor -- start 2023-10-08T09:29:23.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.007+0000 7fe7b58f2b80 1 -- start start 2023-10-08T09:29:23.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.007+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4122060 0x5590e412b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:23.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.007+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:23.015 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.007+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4126840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.007+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e3ef5920 con 0x5590e41229c0 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.007+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e403ceb0 con 0x5590e4123800 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.007+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e4117570 con 0x5590e4122060 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.012+0000 7fe7a37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4126840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.012+0000 7fe7a37fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4126840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43734/0 (socket says 172.21.15.49:43734) 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.012+0000 7fe7a37fe700 1 -- 172.21.15.49:0/3315341241 learned_addr learned my addr 172.21.15.49:0/3315341241 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.012+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3315341241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4122060 msgr2=0x5590e412b4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4122060 0x5590e412b4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3315341241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 msgr2=0x5590e4126840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4126840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.016 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe7a27fc700 1 -- 172.21.15.49:0/3315341241 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590e3ec6250 con 0x5590e41229c0 2023-10-08T09:29:23.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133500 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x5590e4122e00 tx=0x7fe7940b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d0c36140562973 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:23.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe799ffb700 1 -- 172.21.15.49:0/3315341241 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940b6070 con 0x5590e41229c0 2023-10-08T09:29:23.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe799ffb700 1 -- 172.21.15.49:0/3315341241 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe794099a90 con 0x5590e41229c0 2023-10-08T09:29:23.017 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.013+0000 7fe799ffb700 1 -- 172.21.15.49:0/3315341241 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940991e0 con 0x5590e41229c0 2023-10-08T09:29:23.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.021+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3315341241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 msgr2=0x5590e4133500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:23.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.021+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133500 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x5590e4122e00 tx=0x7fe7940b9040 comp rx=0 tx=0).stop 2023-10-08T09:29:23.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3315341241 shutdown_connections 2023-10-08T09:29:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4122060 0x5590e412b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4123800 0x5590e4126840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/3315341241 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133500 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3315341241 >> 172.21.15.49:0/3315341241 conn(0x5590e4096260 msgr2=0x5590e4160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3315341241 shutdown_connections 2023-10-08T09:29:23.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/3315341241 wait complete. 2023-10-08T09:29:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.036+0000 7fe7b58f2b80 1 -- start start 2023-10-08T09:29:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 1 Processor -- start 2023-10-08T09:29:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e4132ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e41314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e4131be0 con 0x5590e41229c0 2023-10-08T09:29:23.039 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e4131d50 con 0x5590e4122060 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590e4131f10 con 0x5590e4123800 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7b58f2b80 10 client.0 _ll_register_callbacks cb 0x7fe78c005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7a27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7a27fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54124/0 (socket says 172.21.15.49:54124) 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.037+0000 7fe7a27fc700 1 -- 172.21.15.49:0/554494179 learned_addr learned my addr 172.21.15.49:0/554494179 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe7a27fc700 1 -- 172.21.15.49:0/554494179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 msgr2=0x5590e41314f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e41314f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe7a27fc700 1 -- 172.21.15.49:0/554494179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 msgr2=0x5590e4132ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e4132ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.040 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe7a27fc700 1 -- 172.21.15.49:0/554494179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590e3e9fde0 con 0x5590e41229c0 2023-10-08T09:29:23.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe7a27fc700 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe7940b9010 tx=0x7fe79409a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4055a6b5ed12598d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:23.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe78a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:29:23.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe78a7fc700 1 -- 172.21.15.49:0/554494179 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79407d070 con 0x5590e41229c0 2023-10-08T09:29:23.041 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe78a7fc700 1 -- 172.21.15.49:0/554494179 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe794099cf0 con 0x5590e41229c0 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.038+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe79409dbe0 con 0x5590e41229c0 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe78a7fc700 1 -- 172.21.15.49:0/554494179 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7940b6050 con 0x5590e41229c0 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 10 client.17166 fetch_fsmap learned FSMap version 41 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5590e3e9e2b0 con 0x5590e41229c0 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe78a7fc700 1 -- 172.21.15.49:0/554494179 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79409dbe0 con 0x5590e41229c0 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe78a7fc700 1 -- 172.21.15.49:0/554494179 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 41) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fe794080070 con 0x5590e41229c0 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 10 client.17166 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 20 client.17166 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e41aeba0 0x5590e41b1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 4 client.17166 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:29:23.042 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.039+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590e406db30 con 0x5590e41aeba0 2023-10-08T09:29:23.053 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.052+0000 7fe7a2ffd700 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e41aeba0 0x5590e41b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:23.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.052+0000 7fe78a7fc700 10 client.17166 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:23.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.052+0000 7fe7a2ffd700 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e41aeba0 0x5590e41b1060 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd84e2ddc8ae0c2 server_cookie=c00843d1bd1b2486 in_seq=0 out_seq=0 2023-10-08T09:29:23.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.056+0000 7fe78a7fc700 1 -- 172.21.15.49:0/554494179 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5590e406db30 con 0x5590e41aeba0 2023-10-08T09:29:23.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.056+0000 7fe78a7fc700 10 client.17166 handle_command_reply: tid=0 2023-10-08T09:29:23.064 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.056+0000 7fe7b58f2b80 1 client.17166 shutdown 2023-10-08T09:29:23.070 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.067+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 msgr2=0x5590e4133420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:23.071 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.067+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe7940b9010 tx=0x7fe79409a250 comp rx=0 tx=0).stop 2023-10-08T09:29:23.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.071+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 shutdown_connections 2023-10-08T09:29:23.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.071+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5590e41aeba0 0x5590e41b1060 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.071+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5590e4123800 0x5590e41314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.075 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.071+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x5590e4122060 0x5590e4132ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.071+0000 7fe7b58f2b80 1 --2- 172.21.15.49:0/554494179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5590e41229c0 0x5590e4133420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.071+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 >> 172.21.15.49:0/554494179 conn(0x5590e4096260 msgr2=0x5590e412b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:23.076 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.071+0000 7fe78a7fc700 0 client.17166 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:23.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.074+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 shutdown_connections 2023-10-08T09:29:23.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.076+0000 7fe7b58f2b80 1 -- 172.21.15.49:0/554494179 wait complete. 2023-10-08T09:29:23.085 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:29:23.085 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:29:23.086 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.082+0000 7fe7b58f2b80 20 client.17166 trim_cache size 0 max 16384 2023-10-08T09:29:23.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.088+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7900a15d0 msgr2=0x7fe7900a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:23.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.088+0000 7fe7a9709700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7900a15d0 0x7fe7900a3a90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00ab30 tx=0x7fe79c009250 comp rx=0 tx=0).stop 2023-10-08T09:29:23.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.088+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410a160 msgr2=0x7fe7a419d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:23.089 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.088+0000 7fe7a9709700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410a160 0x7fe7a419d3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe79400b550 tx=0x7fe79400b910 comp rx=0 tx=0).stop 2023-10-08T09:29:23.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.089+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 shutdown_connections 2023-10-08T09:29:23.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.089+0000 7fe7a9709700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fe7900a15d0 0x7fe7900a3a90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.089+0000 7fe7a9709700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7a410cb30 0x7fe7a41a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.099 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.089+0000 7fe7a9709700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe7a41077f0 0x7fe7a419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.089+0000 7fe7a9709700 1 --2- 172.21.15.49:0/3231178887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe7a410a160 0x7fe7a419d3c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:23.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.089+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 >> 172.21.15.49:0/3231178887 conn(0x7fe7a406d790 msgr2=0x7fe7a4107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:23.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.098+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 shutdown_connections 2023-10-08T09:29:23.100 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:23.098+0000 7fe7a9709700 1 -- 172.21.15.49:0/3231178887 wait complete. 2023-10-08T09:29:23.119 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:29:23.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:23 smithi049 ceph-mon[99485]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 504 op/s 2023-10-08T09:29:23.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:23 smithi184 ceph-mon[103675]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 504 op/s 2023-10-08T09:29:23.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:23 smithi132 ceph-mon[103389]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.0 MiB/s wr, 504 op/s 2023-10-08T09:29:24.313 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 840.0 s, 508.6 tps, lat 193.165 ms stddev 204.184 2023-10-08T09:29:26.257 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:25 smithi184 ceph-mon[103675]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.9 MiB/s wr, 484 op/s 2023-10-08T09:29:26.304 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:25 smithi132 ceph-mon[103389]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.9 MiB/s wr, 484 op/s 2023-10-08T09:29:26.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:25 smithi049 ceph-mon[99485]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.9 MiB/s wr, 484 op/s 2023-10-08T09:29:28.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:27 smithi184 ceph-mon[103675]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 503 op/s 2023-10-08T09:29:28.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:27 smithi132 ceph-mon[103389]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 503 op/s 2023-10-08T09:29:28.386 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:27 smithi049 ceph-mon[99485]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.7 MiB/s wr, 503 op/s 2023-10-08T09:29:29.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 845.0 s, 534.8 tps, lat 183.319 ms stddev 196.795 2023-10-08T09:29:29.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:29 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:29:29.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:29 smithi184 ceph-mon[103675]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 497 op/s 2023-10-08T09:29:29.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:29 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:29:29.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:29 smithi132 ceph-mon[103389]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 497 op/s 2023-10-08T09:29:29.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:29 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:29:29.646 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:29 smithi049 ceph-mon[99485]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.6 MiB/s wr, 497 op/s 2023-10-08T09:29:32.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:31 smithi184 ceph-mon[103675]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 7.8 MiB/s wr, 502 op/s 2023-10-08T09:29:32.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:31 smithi132 ceph-mon[103389]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 7.8 MiB/s wr, 502 op/s 2023-10-08T09:29:32.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:31 smithi049 ceph-mon[99485]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 7.8 MiB/s wr, 502 op/s 2023-10-08T09:29:33.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:33 smithi184 ceph-mon[103675]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 7.4 MiB/s wr, 519 op/s 2023-10-08T09:29:33.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:33 smithi132 ceph-mon[103389]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 7.4 MiB/s wr, 519 op/s 2023-10-08T09:29:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:33 smithi049 ceph-mon[99485]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 7.4 MiB/s wr, 519 op/s 2023-10-08T09:29:34.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 850.0 s, 514.6 tps, lat 200.885 ms stddev 232.212 2023-10-08T09:29:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.758+0000 7f51a06ea700 1 -- 172.21.15.49:0/777724219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980fee00 msgr2=0x7f51980ff220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.758+0000 7f51a06ea700 1 --2- 172.21.15.49:0/777724219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980fee00 0x7f51980ff220 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f519001a9a0 tx=0x7f519001acb0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.759+0000 7f51a06ea700 1 -- 172.21.15.49:0/777724219 shutdown_connections 2023-10-08T09:29:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.759+0000 7f51a06ea700 1 --2- 172.21.15.49:0/777724219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980fee00 0x7f51980ff220 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.760 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.759+0000 7f51a06ea700 1 --2- 172.21.15.49:0/777724219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5198100990 0x7f5198107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.759+0000 7f51a06ea700 1 --2- 172.21.15.49:0/777724219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51980ffff0 0x7f5198100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.759+0000 7f51a06ea700 1 -- 172.21.15.49:0/777724219 >> 172.21.15.49:0/777724219 conn(0x7f51980fa370 msgr2=0x7f51980fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.762+0000 7f51a06ea700 1 -- 172.21.15.49:0/777724219 shutdown_connections 2023-10-08T09:29:35.763 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.762+0000 7f51a06ea700 1 -- 172.21.15.49:0/777724219 wait complete. 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.762+0000 7f51a06ea700 1 Processor -- start 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f51a06ea700 1 -- start start 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f51a06ea700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51980fee00 0x7f5198107170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f51a06ea700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980ffff0 0x7f519810be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f51a06ea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5198100990 0x7f519810e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f51a06ea700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519810ec40 con 0x7f51980fee00 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f51a06ea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519810edb0 con 0x7f5198100990 2023-10-08T09:29:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f51a06ea700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519810ef70 con 0x7f51980ffff0 2023-10-08T09:29:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f519dc85700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980ffff0 0x7f519810be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f519dc85700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980ffff0 0x7f519810be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41170/0 (socket says 172.21.15.49:41170) 2023-10-08T09:29:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f519dc85700 1 -- 172.21.15.49:0/2788183198 learned_addr learned my addr 172.21.15.49:0/2788183198 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f519ec87700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5198100990 0x7f519810e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.763+0000 7f519e486700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51980fee00 0x7f5198107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.775 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f519dc85700 1 -- 172.21.15.49:0/2788183198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5198100990 msgr2=0x7f519810e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.776 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f519dc85700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5198100990 0x7f519810e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f519dc85700 1 -- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51980fee00 msgr2=0x7f5198107170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f519dc85700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51980fee00 0x7f5198107170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f519dc85700 1 -- 172.21.15.49:0/2788183198 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519001a600 con 0x7f51980ffff0 2023-10-08T09:29:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f519dc85700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980ffff0 0x7f519810be00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f518800ba50 tx=0x7f518800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f51977fe700 1 -- 172.21.15.49:0/2788183198 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518800c930 con 0x7f51980ffff0 2023-10-08T09:29:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519810f1f0 con 0x7f51980ffff0 2023-10-08T09:29:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.764+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519810f6d0 con 0x7f51980ffff0 2023-10-08T09:29:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.765+0000 7f51977fe700 1 -- 172.21.15.49:0/2788183198 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5188014440 con 0x7f51980ffff0 2023-10-08T09:29:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.765+0000 7f51977fe700 1 -- 172.21.15.49:0/2788183198 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880075b0 con 0x7f51980ffff0 2023-10-08T09:29:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.768+0000 7f51977fe700 1 -- 172.21.15.49:0/2788183198 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f518800ca90 con 0x7f51980ffff0 2023-10-08T09:29:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.786+0000 7f51ac8d3b80 1 Processor -- start 2023-10-08T09:29:35.789 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.787+0000 7f51ac8d3b80 1 -- start start 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.787+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796be7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.787+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796be7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.787+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796be8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.787+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796b4f400 con 0x557796be6e40 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.787+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796b40bc0 con 0x557796be82b0 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.787+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796b3e4e0 con 0x557796be77a0 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.788+0000 7f51977fe700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f51800a1580 0x7f51800a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.788+0000 7f51977fe700 1 -- 172.21.15.49:0/2788183198 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f51880cc5c0 con 0x7f51980ffff0 2023-10-08T09:29:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.793+0000 7f519ec87700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796be7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.793+0000 7f519ec87700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796be7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54142/0 (socket says 172.21.15.49:54142) 2023-10-08T09:29:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.793+0000 7f519ec87700 1 -- 172.21.15.49:0/3010796738 learned_addr learned my addr 172.21.15.49:0/3010796738 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.793+0000 7f519ec87700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f51800a1580 0x7f51800a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.797 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.795+0000 7f519dc85700 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796be8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.801+0000 7f519ec87700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f51800a1580 0x7f51800a3a40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f518c006ae0 tx=0x7f518c009690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.802+0000 7f519ec87700 1 -- 172.21.15.49:0/3010796738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 msgr2=0x557796be7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.802+0000 7f519ec87700 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796be7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.802+0000 7f519ec87700 1 -- 172.21.15.49:0/3010796738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 msgr2=0x557796be8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.802+0000 7f519ec87700 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796be8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.802+0000 7f519ec87700 1 -- 172.21.15.49:0/3010796738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557796964d80 con 0x557796be6e40 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.803+0000 7f519ec87700 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796be7260 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f518c015fd0 tx=0x7f518c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecdf92bee0f51cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.803+0000 7f5194ff9700 1 -- 172.21.15.49:0/3010796738 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c024070 con 0x557796be6e40 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.803+0000 7f5194ff9700 1 -- 172.21.15.49:0/3010796738 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518c018d00 con 0x557796be6e40 2023-10-08T09:29:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.803+0000 7f5194ff9700 1 -- 172.21.15.49:0/3010796738 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c006710 con 0x557796be6e40 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.804+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/3010796738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 msgr2=0x557796be7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.804+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796be7260 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f518c015fd0 tx=0x7f518c019000 comp rx=0 tx=0).stop 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.809+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/3010796738 shutdown_connections 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.809+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796be8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.809+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796be7260 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.809+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/3010796738 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796be7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.809+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/3010796738 >> 172.21.15.49:0/3010796738 conn(0x557796b5b200 msgr2=0x557796be18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.811+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/3010796738 shutdown_connections 2023-10-08T09:29:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.811+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/3010796738 wait complete. 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.811+0000 7f51ac8d3b80 1 -- start start 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.811+0000 7f51ac8d3b80 1 Processor -- start 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796bf0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 0x557796bf0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796bf7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796b39dd0 con 0x557796be82b0 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796ad66b0 con 0x557796be77a0 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796ad6350 con 0x557796be6e40 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f51ac8d3b80 10 client.0 _ll_register_callbacks cb 0x5577969337a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519ec87700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796bf7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796bf0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796bf0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41184/0 (socket says 172.21.15.49:41184) 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 -- 172.21.15.49:0/2335756378 learned_addr learned my addr 172.21.15.49:0/2335756378 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519ec87700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796bf7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54154/0 (socket says 172.21.15.49:54154) 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 -- 172.21.15.49:0/2335756378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 msgr2=0x557796bf0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 0x557796bf0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 -- 172.21.15.49:0/2335756378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 msgr2=0x557796bf7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796bf7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519e486700 1 -- 172.21.15.49:0/2335756378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c015d90 con 0x557796be6e40 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.812+0000 7f519ec87700 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796bf7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f519e486700 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796bf0870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5190003820 tx=0x7f5190005e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5a4faba8e0144a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f517dffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:29:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f517dffb700 1 -- 172.21.15.49:0/2335756378 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519002d050 con 0x557796be6e40 2023-10-08T09:29:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f517dffb700 1 -- 172.21.15.49:0/2335756378 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f519001cd30 con 0x557796be6e40 2023-10-08T09:29:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557796b01e50 con 0x557796be6e40 2023-10-08T09:29:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f517dffb700 1 -- 172.21.15.49:0/2335756378 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51900039f0 con 0x557796be6e40 2023-10-08T09:29:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f51ac8d3b80 10 client.26797 fetch_fsmap learned FSMap version 42 2023-10-08T09:29:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.813+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55779694dc00 con 0x557796be6e40 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.818+0000 7f517dffb700 1 -- 172.21.15.49:0/2335756378 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557796b01e50 con 0x557796be6e40 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.818+0000 7f517dffb700 1 -- 172.21.15.49:0/2335756378 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f519003c070 con 0x557796be6e40 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.818+0000 7f51ac8d3b80 10 client.26797 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.818+0000 7f51ac8d3b80 10 client.26797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.818+0000 7f51ac8d3b80 20 client.26797 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.819+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796bfc630 0x557796bfea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.819+0000 7f51ac8d3b80 4 client.26797 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.819+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557796b32db0 con 0x557796bfc630 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.819+0000 7f519dc85700 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796bfc630 0x557796bfea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.819+0000 7f517dffb700 10 client.26797 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.820+0000 7f519dc85700 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796bfc630 0x557796bfea50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7df34383e407992 server_cookie=9404e0d14876946f in_seq=0 out_seq=0 2023-10-08T09:29:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.820+0000 7f517dffb700 1 -- 172.21.15.49:0/2335756378 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557796b32db0 con 0x557796bfc630 2023-10-08T09:29:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.820+0000 7f517dffb700 10 client.26797 handle_command_reply: tid=0 2023-10-08T09:29:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.820+0000 7f51ac8d3b80 1 client.26797 shutdown 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 msgr2=0x557796bf0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796bf0870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5190003820 tx=0x7f5190005e90 comp rx=0 tx=0).stop 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 shutdown_connections 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796bfc630 0x557796bfea50 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796bf0870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796bf7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2335756378 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 0x557796bf0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 >> 172.21.15.49:0/2335756378 conn(0x557796b5b200 msgr2=0x557796be1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f517dffb700 0 client.26797 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 shutdown_connections 2023-10-08T09:29:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2335756378 wait complete. 2023-10-08T09:29:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.828+0000 7f51ac8d3b80 20 client.26797 trim_cache size 0 max 16384 2023-10-08T09:29:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 Processor -- start 2023-10-08T09:29:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 -- start start 2023-10-08T09:29:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796be7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 0x557796bf7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796befa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577969ba8c0 con 0x557796be82b0 2023-10-08T09:29:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796bdc2b0 con 0x557796be77a0 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796bef350 con 0x557796be6e40 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f519ec87700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796befa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f519e486700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796be7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f519ec87700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796befa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54162/0 (socket says 172.21.15.49:54162) 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f519ec87700 1 -- 172.21.15.49:0/2697529967 learned_addr learned my addr 172.21.15.49:0/2697529967 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.850+0000 7f519dc85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 0x557796bf7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.851+0000 7f519e486700 1 -- 172.21.15.49:0/2697529967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 msgr2=0x557796bf7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.851+0000 7f519e486700 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 0x557796bf7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.851+0000 7f519e486700 1 -- 172.21.15.49:0/2697529967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 msgr2=0x557796befa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.851+0000 7f519e486700 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796befa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.851+0000 7f519e486700 1 -- 172.21.15.49:0/2697529967 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55779698b1f0 con 0x557796be6e40 2023-10-08T09:29:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.851+0000 7f519ec87700 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796befa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:29:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.851+0000 7f519e486700 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796be7220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f519001a8f0 tx=0x7f519001acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642ea500d3eca2a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.854+0000 7f51957fa700 1 -- 172.21.15.49:0/2697529967 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519002d040 con 0x557796be6e40 2023-10-08T09:29:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.854+0000 7f51957fa700 1 -- 172.21.15.49:0/2697529967 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f519002e040 con 0x557796be6e40 2023-10-08T09:29:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.854+0000 7f51957fa700 1 -- 172.21.15.49:0/2697529967 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190004110 con 0x557796be6e40 2023-10-08T09:29:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2697529967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 msgr2=0x557796be7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796be7220 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f519001a8f0 tx=0x7f519001acb0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2697529967 shutdown_connections 2023-10-08T09:29:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be6e40 0x557796be7220 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be82b0 0x557796befa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/2697529967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be77a0 0x557796bf7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2697529967 >> 172.21.15.49:0/2697529967 conn(0x557796b5b200 msgr2=0x557796beb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2697529967 shutdown_connections 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.855+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/2697529967 wait complete. 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 -- start start 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 Processor -- start 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796bed160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796bed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796c4e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796c4e980 con 0x557796be6e40 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796c4eac0 con 0x557796be82b0 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557796c4ec30 con 0x557796be77a0 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f51ac8d3b80 10 client.0 _ll_register_callbacks cb 0x7f519001af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f519e486700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796bed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f519dc85700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796bed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f519dc85700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796bed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41200/0 (socket says 172.21.15.49:41200) 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f519e486700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796bed160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54164/0 (socket says 172.21.15.49:54164) 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f519dc85700 1 -- 172.21.15.49:0/1243342324 learned_addr learned my addr 172.21.15.49:0/1243342324 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.856+0000 7f519ec87700 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796c4e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f519dc85700 1 -- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 msgr2=0x557796c4e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f519dc85700 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796c4e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f519dc85700 1 -- 172.21.15.49:0/1243342324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 msgr2=0x557796bed160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f519dc85700 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796bed160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f519dc85700 1 -- 172.21.15.49:0/1243342324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557796964d80 con 0x557796be77a0 2023-10-08T09:29:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f519e486700 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796bed160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f519dc85700 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796bed6a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f518800ba20 tx=0x7f51880c9c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed13b8159ee9fc08 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f517dffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f517dffb700 1 -- 172.21.15.49:0/1243342324 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518809d6a0 con 0x557796be77a0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f517dffb700 1 -- 172.21.15.49:0/1243342324 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f51880a18e0 con 0x557796be77a0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f518c018d00 con 0x557796be77a0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f517dffb700 1 -- 172.21.15.49:0/1243342324 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51880cc2d0 con 0x557796be77a0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f51ac8d3b80 10 client.26803 fetch_fsmap learned FSMap version 42 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.857+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55779694dc00 con 0x557796be77a0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f517dffb700 1 -- 172.21.15.49:0/1243342324 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f518807c080 con 0x557796be77a0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f517dffb700 1 -- 172.21.15.49:0/1243342324 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f51880a1450 con 0x557796be77a0 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f51ac8d3b80 10 client.26803 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:29:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f51ac8d3b80 10 client.26803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f51ac8d3b80 20 client.26803 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796c50740 0x557796c52b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f51ac8d3b80 4 client.26803 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:29:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557796b32ad0 con 0x557796c50740 2023-10-08T09:29:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.858+0000 7f519e486700 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796c50740 0x557796c52b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.859+0000 7f519e486700 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796c50740 0x557796c52b60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f709da5974b45a server_cookie=74cc0086273e373 in_seq=0 out_seq=0 2023-10-08T09:29:35.865 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.859+0000 7f517dffb700 10 client.26803 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.870+0000 7f517dffb700 1 -- 172.21.15.49:0/1243342324 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557796b32ad0 con 0x557796c50740 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.870+0000 7f517dffb700 10 client.26803 handle_command_reply: tid=0 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.870+0000 7f51ac8d3b80 1 client.26803 shutdown 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 msgr2=0x557796bed6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796bed6a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f518800ba20 tx=0x7f51880c9c00 comp rx=0 tx=0).stop 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 shutdown_connections 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x557796c50740 0x557796c52b60 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.873 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x557796be82b0 0x557796c4e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f517dffb700 0 client.26803 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557796be6e40 0x557796bed160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 --2- 172.21.15.49:0/1243342324 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557796be77a0 0x557796bed6a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 >> 172.21.15.49:0/1243342324 conn(0x557796b5b200 msgr2=0x557796befa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 shutdown_connections 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 1 -- 172.21.15.49:0/1243342324 wait complete. 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.871+0000 7f51ac8d3b80 20 client.26803 trim_cache size 0 max 16384 2023-10-08T09:29:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:29:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:29:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:29:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f51800a1580 msgr2=0x7f51800a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f51800a1580 0x7f51800a3a40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f518c006ae0 tx=0x7f518c009690 comp rx=0 tx=0).stop 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980ffff0 msgr2=0x7f519810be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980ffff0 0x7f519810be00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f518800ba50 tx=0x7f518800bd60 comp rx=0 tx=0).stop 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 shutdown_connections 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f51800a1580 0x7f51800a3a40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5198100990 0x7f519810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51980fee00 0x7f5198107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 --2- 172.21.15.49:0/2788183198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51980ffff0 0x7f519810be00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 >> 172.21.15.49:0/2788183198 conn(0x7f51980fa370 msgr2=0x7f519806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:35.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 shutdown_connections 2023-10-08T09:29:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:35.872+0000 7f51a06ea700 1 -- 172.21.15.49:0/2788183198 wait complete. 2023-10-08T09:29:36.301 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:36 smithi132 ceph-mon[103389]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 7.2 MiB/s wr, 489 op/s 2023-10-08T09:29:36.400 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:36 smithi049 ceph-mon[99485]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 7.2 MiB/s wr, 489 op/s 2023-10-08T09:29:36.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:36 smithi184 ceph-mon[103675]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 7.2 MiB/s wr, 489 op/s 2023-10-08T09:29:37.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:37 smithi049 ceph-mon[99485]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 6.3 MiB/s wr, 486 op/s 2023-10-08T09:29:37.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:37 smithi184 ceph-mon[103675]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 6.3 MiB/s wr, 486 op/s 2023-10-08T09:29:37.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:37 smithi132 ceph-mon[103389]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 6.3 MiB/s wr, 486 op/s 2023-10-08T09:29:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 855.0 s, 505.8 tps, lat 200.211 ms stddev 218.765 2023-10-08T09:29:40.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:40 smithi132 ceph-mon[103389]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.8 MiB/s wr, 486 op/s 2023-10-08T09:29:40.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:39 smithi049 ceph-mon[99485]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.8 MiB/s wr, 486 op/s 2023-10-08T09:29:40.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:40 smithi184 ceph-mon[103675]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.8 MiB/s wr, 486 op/s 2023-10-08T09:29:41.389 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:41 smithi049 ceph-mon[99485]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2023-10-08T09:29:41.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:41 smithi184 ceph-mon[103675]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2023-10-08T09:29:41.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:41 smithi132 ceph-mon[103389]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2023-10-08T09:29:43.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:43 smithi132 ceph-mon[103389]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 6.9 MiB/s wr, 513 op/s 2023-10-08T09:29:43.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:43 smithi049 ceph-mon[99485]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 6.9 MiB/s wr, 513 op/s 2023-10-08T09:29:43.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:43 smithi184 ceph-mon[103675]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 6.9 MiB/s wr, 513 op/s 2023-10-08T09:29:44.312 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 860.0 s, 519.0 tps, lat 190.283 ms stddev 200.538 2023-10-08T09:29:46.249 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:45 smithi184 ceph-mon[103675]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 482 op/s 2023-10-08T09:29:46.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:45 smithi132 ceph-mon[103389]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 482 op/s 2023-10-08T09:29:46.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:45 smithi049 ceph-mon[99485]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 482 op/s 2023-10-08T09:29:48.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:47 smithi184 ceph-mon[103675]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 492 op/s 2023-10-08T09:29:48.259 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:47 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:29:48.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:47 smithi132 ceph-mon[103389]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 492 op/s 2023-10-08T09:29:48.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:47 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:29:48.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:47 smithi049 ceph-mon[99485]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.7 MiB/s wr, 492 op/s 2023-10-08T09:29:48.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:47 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:29:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.097+0000 7f933c4d9700 1 -- 172.21.15.49:0/1828466115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93341077f0 msgr2=0x7f9334109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.098 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.097+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1828466115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93341077f0 0x7f9334109c20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f933001a9a0 tx=0x7f933001acb0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.098+0000 7f933c4d9700 1 -- 172.21.15.49:0/1828466115 shutdown_connections 2023-10-08T09:29:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.098+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1828466115 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933410cb30 0x7f933410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.100 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.098+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1828466115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93341077f0 0x7f9334109c20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.098+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1828466115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f933410a160 0x7f933410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.101 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.098+0000 7f933c4d9700 1 -- 172.21.15.49:0/1828466115 >> 172.21.15.49:0/1828466115 conn(0x7f933406d770 msgr2=0x7f933406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.107+0000 7f933c4d9700 1 -- 172.21.15.49:0/1828466115 shutdown_connections 2023-10-08T09:29:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.113+0000 7f933c4d9700 1 -- 172.21.15.49:0/1828466115 wait complete. 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.114+0000 7f933c4d9700 1 Processor -- start 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933c4d9700 1 -- start start 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933c4d9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93341077f0 0x7f9334118070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933c4d9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f933410a160 0x7f93341185b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933c4d9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933410cb30 0x7f93341c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933c4d9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341c69d0 con 0x7f93341077f0 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933c4d9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341c6b10 con 0x7f933410a160 2023-10-08T09:29:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933c4d9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341c6c50 con 0x7f933410cb30 2023-10-08T09:29:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933a275700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93341077f0 0x7f9334118070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933a275700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93341077f0 0x7f9334118070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54176/0 (socket says 172.21.15.49:54176) 2023-10-08T09:29:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933a275700 1 -- 172.21.15.49:0/1413126760 learned_addr learned my addr 172.21.15.49:0/1413126760 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.125+0000 7f933aa76700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933410cb30 0x7f93341c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.126+0000 7f933aa76700 1 -- 172.21.15.49:0/1413126760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f933410a160 msgr2=0x7f93341185b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T09:29:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.126+0000 7f933aa76700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f933410a160 0x7f93341185b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.126+0000 7f933aa76700 1 -- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93341077f0 msgr2=0x7f9334118070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.126+0000 7f933aa76700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93341077f0 0x7f9334118070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.126+0000 7f933aa76700 1 -- 172.21.15.49:0/1413126760 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933001a600 con 0x7f933410cb30 2023-10-08T09:29:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.126+0000 7f933aa76700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933410cb30 0x7f93341c6490 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f932c00efc0 tx=0x7f932c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.131+0000 7f932b7fe700 1 -- 172.21.15.49:0/1413126760 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c018070 con 0x7f933410cb30 2023-10-08T09:29:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.132+0000 7f932b7fe700 1 -- 172.21.15.49:0/1413126760 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932c00f040 con 0x7f933410cb30 2023-10-08T09:29:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.132+0000 7f932b7fe700 1 -- 172.21.15.49:0/1413126760 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c003dd0 con 0x7f933410cb30 2023-10-08T09:29:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.132+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93341c6e50 con 0x7f933410cb30 2023-10-08T09:29:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.132+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93341c7280 con 0x7f933410cb30 2023-10-08T09:29:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 Processor -- start 2023-10-08T09:29:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 -- start start 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784740260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847407a0 0x55a784740cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847412b0 0x55a784741730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7846a8400 con 0x55a7847407a0 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a784699bc0 con 0x55a78473fe40 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f93486c2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7846974e0 con 0x55a7847412b0 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f932b7fe700 1 -- 172.21.15.49:0/1413126760 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f932c003f90 con 0x7f933410cb30 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f9339a74700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847412b0 0x55a784741730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f9339a74700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847412b0 0x55a784741730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41214/0 (socket says 172.21.15.49:41214) 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f9339a74700 1 -- 172.21.15.49:0/2470268624 learned_addr learned my addr 172.21.15.49:0/2470268624 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.140+0000 7f9339a74700 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784740260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f9339a74700 1 -- 172.21.15.49:0/2470268624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 msgr2=0x55a784740260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f9339a74700 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784740260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f9339a74700 1 -- 172.21.15.49:0/2470268624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847407a0 msgr2=0x55a784740cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f9339a74700 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847407a0 0x55a784740cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f9339a74700 1 -- 172.21.15.49:0/2470268624 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7844bdd80 con 0x55a7847412b0 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f9339a74700 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847412b0 0x55a784741730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9324012590 tx=0x7f932400b530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a299767d8b5f12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f932b7fe700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93200a1650 0x7f93200a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f932b7fe700 1 -- 172.21.15.49:0/1413126760 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f932c0ccb20 con 0x7f933410cb30 2023-10-08T09:29:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.141+0000 7f9328d82700 1 -- 172.21.15.49:0/2470268624 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324012bb0 con 0x55a7847412b0 2023-10-08T09:29:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.142+0000 7f9328d82700 1 -- 172.21.15.49:0/2470268624 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9324004d90 con 0x55a7847412b0 2023-10-08T09:29:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.142+0000 7f9328d82700 1 -- 172.21.15.49:0/2470268624 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324006670 con 0x55a7847412b0 2023-10-08T09:29:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.143+0000 7f93486c2b80 1 -- 172.21.15.49:0/2470268624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847412b0 msgr2=0x55a784741730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.143+0000 7f93486c2b80 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847412b0 0x55a784741730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9324012590 tx=0x7f932400b530 comp rx=0 tx=0).stop 2023-10-08T09:29:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.145+0000 7f933a275700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93200a1650 0x7f93200a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.146+0000 7f93486c2b80 1 -- 172.21.15.49:0/2470268624 shutdown_connections 2023-10-08T09:29:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.146+0000 7f93486c2b80 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847412b0 0x55a784741730 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.146+0000 7f93486c2b80 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784740260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.146+0000 7f93486c2b80 1 --2- 172.21.15.49:0/2470268624 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847407a0 0x55a784740cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.146+0000 7f93486c2b80 1 -- 172.21.15.49:0/2470268624 >> 172.21.15.49:0/2470268624 conn(0x55a7846b4200 msgr2=0x55a78473a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.146+0000 7f933a275700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93200a1650 0x7f93200a3b10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9330005850 tx=0x7f933001c270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.149+0000 7f93486c2b80 1 -- 172.21.15.49:0/2470268624 shutdown_connections 2023-10-08T09:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.155+0000 7f93486c2b80 1 -- 172.21.15.49:0/2470268624 wait complete. 2023-10-08T09:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.155+0000 7f93486c2b80 1 -- start start 2023-10-08T09:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.155+0000 7f93486c2b80 1 Processor -- start 2023-10-08T09:29:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.156+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78473fe40 0x55a7847498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.156+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847407a0 0x55a784749df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.156+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 0x55a784750e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.156+0000 7f93486c2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a784692dd0 con 0x55a7847412b0 2023-10-08T09:29:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.156+0000 7f93486c2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78462f6b0 con 0x55a7847407a0 2023-10-08T09:29:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.156+0000 7f93486c2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a78462f350 con 0x55a78473fe40 2023-10-08T09:29:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.156+0000 7f93486c2b80 10 client.0 _ll_register_callbacks cb 0x55a78448c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78473fe40 0x55a7847498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78473fe40 0x55a7847498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41222/0 (socket says 172.21.15.49:41222) 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 -- 172.21.15.49:0/3025602840 learned_addr learned my addr 172.21.15.49:0/3025602840 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f9339a74700 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847407a0 0x55a784749df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 -- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847407a0 msgr2=0x55a784749df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847407a0 0x55a784749df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 -- 172.21.15.49:0/3025602840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 msgr2=0x55a784750e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 0x55a784750e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 -- 172.21.15.49:0/3025602840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9324013040 con 0x55a78473fe40 2023-10-08T09:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.157+0000 7f933a275700 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78473fe40 0x55a7847498b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f933001cde0 tx=0x7f933001a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c9c4e960167416c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f931dffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f931dffb700 1 -- 172.21.15.49:0/3025602840 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330010e80 con 0x55a78473fe40 2023-10-08T09:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f931dffb700 1 -- 172.21.15.49:0/3025602840 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f93300283f0 con 0x55a78473fe40 2023-10-08T09:29:49.159 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a78465ae50 con 0x55a78473fe40 2023-10-08T09:29:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f931dffb700 1 -- 172.21.15.49:0/3025602840 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93300285b0 con 0x55a78473fe40 2023-10-08T09:29:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f93486c2b80 10 client.26815 fetch_fsmap learned FSMap version 42 2023-10-08T09:29:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a7844a6c00 con 0x55a78473fe40 2023-10-08T09:29:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.159+0000 7f931dffb700 1 -- 172.21.15.49:0/3025602840 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a78465ae50 con 0x55a78473fe40 2023-10-08T09:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f931dffb700 1 -- 172.21.15.49:0/3025602840 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f933003b8d0 con 0x55a78473fe40 2023-10-08T09:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f93486c2b80 10 client.26815 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f93486c2b80 10 client.26815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f93486c2b80 20 client.26815 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a784755630 0x55a784757a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f93486c2b80 4 client.26815 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:29:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a78468bdb0 con 0x55a784755630 2023-10-08T09:29:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.160+0000 7f9339a74700 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a784755630 0x55a784757a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.161+0000 7f9339a74700 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a784755630 0x55a784757a50 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996dd8c78fb3b9a server_cookie=7cac06cd1136e808 in_seq=0 out_seq=0 2023-10-08T09:29:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.161+0000 7f931dffb700 10 client.26815 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.166+0000 7f931dffb700 1 -- 172.21.15.49:0/3025602840 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a78468bdb0 con 0x55a784755630 2023-10-08T09:29:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.166+0000 7f931dffb700 10 client.26815 handle_command_reply: tid=0 2023-10-08T09:29:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.166+0000 7f93486c2b80 1 client.26815 shutdown 2023-10-08T09:29:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.172+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78473fe40 msgr2=0x55a7847498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.172+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78473fe40 0x55a7847498b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f933001cde0 tx=0x7f933001a320 comp rx=0 tx=0).stop 2023-10-08T09:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 shutdown_connections 2023-10-08T09:29:49.178 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a784755630 0x55a784757a50 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78473fe40 0x55a7847498b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 0x55a784750e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3025602840 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847407a0 0x55a784749df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 >> 172.21.15.49:0/3025602840 conn(0x55a7846b4200 msgr2=0x55a78473a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f931dffb700 0 client.26815 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 shutdown_connections 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 1 -- 172.21.15.49:0/3025602840 wait complete. 2023-10-08T09:29:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.178+0000 7f93486c2b80 20 client.26815 trim_cache size 0 max 16384 2023-10-08T09:29:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.215+0000 7f93486c2b80 1 Processor -- start 2023-10-08T09:29:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.218+0000 7f93486c2b80 1 -- start start 2023-10-08T09:29:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.218+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784749660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.218+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847407a0 0x55a784750df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.218+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 0x55a784749ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.218+0000 7f93486c2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7845138c0 con 0x55a7847412b0 2023-10-08T09:29:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.218+0000 7f93486c2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a784747eb0 con 0x55a78473fe40 2023-10-08T09:29:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.218+0000 7f93486c2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a784751fb0 con 0x55a7847407a0 2023-10-08T09:29:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.224+0000 7f9339a74700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847407a0 0x55a784750df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.224+0000 7f933a275700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784749660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.224+0000 7f933a275700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784749660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43806/0 (socket says 172.21.15.49:43806) 2023-10-08T09:29:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.224+0000 7f933a275700 1 -- 172.21.15.49:0/3570669190 learned_addr learned my addr 172.21.15.49:0/3570669190 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.226+0000 7f933a275700 1 -- 172.21.15.49:0/3570669190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847407a0 msgr2=0x55a784750df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.226+0000 7f933a275700 1 --2- 172.21.15.49:0/3570669190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847407a0 0x55a784750df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.226+0000 7f933a275700 1 -- 172.21.15.49:0/3570669190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 msgr2=0x55a784749ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.226+0000 7f933a275700 1 --2- 172.21.15.49:0/3570669190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 0x55a784749ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.227+0000 7f933a275700 1 -- 172.21.15.49:0/3570669190 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7844e41f0 con 0x55a78473fe40 2023-10-08T09:29:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.231+0000 7f933a275700 1 --2- 172.21.15.49:0/3570669190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784749660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f933001cdb0 tx=0x7f933001cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55e721a39b07c279 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.231+0000 7f9329583700 1 -- 172.21.15.49:0/3570669190 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330010d80 con 0x55a78473fe40 2023-10-08T09:29:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.231+0000 7f9329583700 1 -- 172.21.15.49:0/3570669190 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f933003b930 con 0x55a78473fe40 2023-10-08T09:29:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.231+0000 7f9329583700 1 -- 172.21.15.49:0/3570669190 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300283f0 con 0x55a78473fe40 2023-10-08T09:29:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.232+0000 7f93486c2b80 1 -- 172.21.15.49:0/3570669190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 msgr2=0x55a784749660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.232+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3570669190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784749660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f933001cdb0 tx=0x7f933001cac0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.235 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.233+0000 7f93486c2b80 1 -- 172.21.15.49:0/3570669190 shutdown_connections 2023-10-08T09:29:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.233+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3570669190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a7847412b0 0x55a784749ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.233+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3570669190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a78473fe40 0x55a784749660 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.233+0000 7f93486c2b80 1 --2- 172.21.15.49:0/3570669190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a7847407a0 0x55a784750df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.233+0000 7f93486c2b80 1 -- 172.21.15.49:0/3570669190 >> 172.21.15.49:0/3570669190 conn(0x55a7846b4200 msgr2=0x55a784744210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:49.251 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.246+0000 7f93486c2b80 1 -- 172.21.15.49:0/3570669190 shutdown_connections 2023-10-08T09:29:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.251+0000 7f93486c2b80 1 -- 172.21.15.49:0/3570669190 wait complete. 2023-10-08T09:29:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 -- start start 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 Processor -- start 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78473fe40 0x55a78474ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847412b0 0x55a78474f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78474fff0 0x55a784750470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7847509b0 con 0x55a78473fe40 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a784750b20 con 0x55a7847412b0 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a784750ce0 con 0x55a78474fff0 2023-10-08T09:29:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.252+0000 7f93486c2b80 10 client.0 _ll_register_callbacks cb 0x7f933001cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.260+0000 7f933a275700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78473fe40 0x55a78474ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.260+0000 7f933a275700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78473fe40 0x55a78474ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54198/0 (socket says 172.21.15.49:54198) 2023-10-08T09:29:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.260+0000 7f933a275700 1 -- 172.21.15.49:0/1010919894 learned_addr learned my addr 172.21.15.49:0/1010919894 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.260+0000 7f933aa76700 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78474fff0 0x55a784750470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f933aa76700 1 -- 172.21.15.49:0/1010919894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847412b0 msgr2=0x55a78474f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f933aa76700 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847412b0 0x55a78474f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f933aa76700 1 -- 172.21.15.49:0/1010919894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78473fe40 msgr2=0x55a78474ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f933aa76700 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78473fe40 0x55a78474ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f933aa76700 1 -- 172.21.15.49:0/1010919894 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7844a6c00 con 0x55a78474fff0 2023-10-08T09:29:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f933aa76700 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78474fff0 0x55a784750470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55a78473c4a0 tx=0x7f932c09afa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2232cfa3e2f15623 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f931dffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:29:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f931dffb700 1 -- 172.21.15.49:0/1010919894 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c09aa60 con 0x55a78474fff0 2023-10-08T09:29:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.262+0000 7f931dffb700 1 -- 172.21.15.49:0/1010919894 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f932c09ebb0 con 0x55a78474fff0 2023-10-08T09:29:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.263+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9324004d90 con 0x55a78474fff0 2023-10-08T09:29:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.263+0000 7f931dffb700 1 -- 172.21.15.49:0/1010919894 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f932c0a2d60 con 0x55a78474fff0 2023-10-08T09:29:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 10 client.26821 fetch_fsmap learned FSMap version 42 2023-10-08T09:29:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a7844bc250 con 0x55a78474fff0 2023-10-08T09:29:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f931dffb700 1 -- 172.21.15.49:0/1010919894 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9324004d90 con 0x55a78474fff0 2023-10-08T09:29:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f931dffb700 1 -- 172.21.15.49:0/1010919894 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f932c004ad0 con 0x55a78474fff0 2023-10-08T09:29:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 10 client.26821 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:29:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 20 client.26821 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a7847aa700 0x55a7847acb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 4 client.26821 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:29:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.265+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a78468bad0 con 0x55a7847aa700 2023-10-08T09:29:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.272+0000 7f933a275700 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a7847aa700 0x55a7847acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.273+0000 7f933a275700 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a7847aa700 0x55a7847acb20 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acde0c08985a9277 server_cookie=bf4971b3606c89ca in_seq=0 out_seq=0 2023-10-08T09:29:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.273+0000 7f931dffb700 10 client.26821 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:49.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 865.0 s, 482.0 tps, lat 196.867 ms stddev 209.936 2023-10-08T09:29:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.333+0000 7f931dffb700 1 -- 172.21.15.49:0/1010919894 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a78468bad0 con 0x55a7847aa700 2023-10-08T09:29:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.333+0000 7f931dffb700 10 client.26821 handle_command_reply: tid=0 2023-10-08T09:29:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.333+0000 7f93486c2b80 1 client.26821 shutdown 2023-10-08T09:29:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.333+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78474fff0 msgr2=0x55a784750470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.333+0000 7f93486c2b80 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78474fff0 0x55a784750470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55a78473c4a0 tx=0x7f932c09afa0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 shutdown_connections 2023-10-08T09:29:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55a7847aa700 0x55a7847acb20 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a78474fff0 0x55a784750470 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55a78473fe40 0x55a78474ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 --2- 172.21.15.49:0/1010919894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55a7847412b0 0x55a78474f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 >> 172.21.15.49:0/1010919894 conn(0x55a7846b4200 msgr2=0x55a784749660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f931dffb700 0 client.26821 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 shutdown_connections 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 1 -- 172.21.15.49:0/1010919894 wait complete. 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.334+0000 7f93486c2b80 20 client.26821 trim_cache size 0 max 16384 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:29:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:29:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:29:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:29:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93200a1650 msgr2=0x7f93200a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93200a1650 0x7f93200a3b10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9330005850 tx=0x7f933001c270 comp rx=0 tx=0).stop 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933410cb30 msgr2=0x7f93341c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933410cb30 0x7f93341c6490 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f932c00efc0 tx=0x7f932c00c3a0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 shutdown_connections 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f93200a1650 0x7f93200a3b10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f933410cb30 0x7f93341c6490 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93341077f0 0x7f9334118070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 --2- 172.21.15.49:0/1413126760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f933410a160 0x7f93341185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 >> 172.21.15.49:0/1413126760 conn(0x7f933406d770 msgr2=0x7f9334107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 shutdown_connections 2023-10-08T09:29:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:29:49.335+0000 7f933c4d9700 1 -- 172.21.15.49:0/1413126760 wait complete. 2023-10-08T09:29:50.250 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:49 smithi184 ceph-mon[103675]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 481 op/s 2023-10-08T09:29:50.302 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:49 smithi132 ceph-mon[103389]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 481 op/s 2023-10-08T09:29:50.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:49 smithi049 ceph-mon[99485]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 481 op/s 2023-10-08T09:29:51.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:29:51.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:29:51.248 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:50 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:29:51.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:29:51.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:29:51.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:50 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:29:51.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:29:51.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:29:51.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:50 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:29:52.250 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:51 smithi184 ceph-mon[103675]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 514 op/s 2023-10-08T09:29:52.300 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:51 smithi132 ceph-mon[103389]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 514 op/s 2023-10-08T09:29:52.384 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:51 smithi049 ceph-mon[99485]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.4 MiB/s wr, 514 op/s 2023-10-08T09:29:53.120 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:29:53.393 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 -- 172.21.15.49:0/1318115432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806c3b0 msgr2=0x7f97c8070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/1318115432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806c3b0 0x7f97c8070c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b0a0 tx=0x7f97c000b3b0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 -- 172.21.15.49:0/1318115432 shutdown_connections 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/1318115432 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c806a7b0 0x7f97c806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/1318115432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806c3b0 0x7f97c8070c50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/1318115432 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806b9f0 0x7f97c806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 -- 172.21.15.49:0/1318115432 >> 172.21.15.49:0/1318115432 conn(0x7f97c80fac10 msgr2=0x7f97c80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 -- 172.21.15.49:0/1318115432 shutdown_connections 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.392+0000 7f97cf5e8700 1 -- 172.21.15.49:0/1318115432 wait complete. 2023-10-08T09:29:53.394 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.393+0000 7f97cf5e8700 1 Processor -- start 2023-10-08T09:29:53.395 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.393+0000 7f97cf5e8700 1 -- start start 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.393+0000 7f97cf5e8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806a7b0 0x7f97c80683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.393+0000 7f97cf5e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806b9f0 0x7f97c8190a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.393+0000 7f97cf5e8700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c806c3b0 0x7f97c8191060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.393+0000 7f97cf5e8700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8191800 con 0x7f97c806a7b0 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cf5e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8191970 con 0x7f97c806b9f0 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cf5e8700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8191b30 con 0x7f97c806c3b0 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806a7b0 0x7f97c80683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.396 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806a7b0 0x7f97c80683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54212/0 (socket says 172.21.15.49:54212) 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 -- 172.21.15.49:0/901490967 learned_addr learned my addr 172.21.15.49:0/901490967 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cdb85700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c806c3b0 0x7f97c8191060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97ccb83700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806b9f0 0x7f97c8190a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 -- 172.21.15.49:0/901490967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c806c3b0 msgr2=0x7f97c8191060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c806c3b0 0x7f97c8191060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 -- 172.21.15.49:0/901490967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806b9f0 msgr2=0x7f97c8190a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806b9f0 0x7f97c8190a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97cd384700 1 -- 172.21.15.49:0/901490967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c4009520 con 0x7f97c806a7b0 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.394+0000 7f97ccb83700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806b9f0 0x7f97c8190a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:29:53.397 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.395+0000 7f97cd384700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806a7b0 0x7f97c80683b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f97c400eb30 tx=0x7f97c400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:53.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.395+0000 7f97be7fc700 1 -- 172.21.15.49:0/901490967 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4009880 con 0x7f97c806a7b0 2023-10-08T09:29:53.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.395+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c0009d00 con 0x7f97c806a7b0 2023-10-08T09:29:53.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.395+0000 7f97be7fc700 1 -- 172.21.15.49:0/901490967 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97c40044e0 con 0x7f97c806a7b0 2023-10-08T09:29:53.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.395+0000 7f97be7fc700 1 -- 172.21.15.49:0/901490967 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4003e90 con 0x7f97c806a7b0 2023-10-08T09:29:53.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.395+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c8192190 con 0x7f97c806a7b0 2023-10-08T09:29:53.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.399+0000 7f97be7fc700 1 -- 172.21.15.49:0/901490967 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f97c4004000 con 0x7f97c806a7b0 2023-10-08T09:29:53.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.400+0000 7f97be7fc700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f97b40a1480 0x7f97b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.400+0000 7f97be7fc700 1 -- 172.21.15.49:0/901490967 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f97c4014070 con 0x7f97c806a7b0 2023-10-08T09:29:53.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.401+0000 7f97ccb83700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f97b40a1480 0x7f97b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.402 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.401+0000 7f97ccb83700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f97b40a1480 0x7f97b40a3940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f97c8068910 tx=0x7f97b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:53.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.402+0000 7f97db7d1b80 1 Processor -- start 2023-10-08T09:29:53.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.402+0000 7f97db7d1b80 1 -- start start 2023-10-08T09:29:53.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9db480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.405 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9dc800 0x55ce0e9dcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e943460 con 0x55ce0e9db060 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e934c20 con 0x55ce0e9dc800 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e932540 con 0x55ce0e9db9c0 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9db480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cdb85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9dc800 0x55ce0e9dcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9db480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54220/0 (socket says 172.21.15.49:54220) 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97ccb83700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 -- 172.21.15.49:0/1054069554 learned_addr learned my addr 172.21.15.49:0/1054069554 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:53.406 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 -- 172.21.15.49:0/1054069554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 msgr2=0x55ce0e9dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 -- 172.21.15.49:0/1054069554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9dc800 msgr2=0x55ce0e9dcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9dc800 0x55ce0e9dcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 -- 172.21.15.49:0/1054069554 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce0e758de0 con 0x55ce0e9db060 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cdb85700 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9dc800 0x55ce0e9dcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.403+0000 7f97cd384700 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9db480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f97c4096940 tx=0x7f97c40968b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b480a99b03fe1e25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97b37fe700 1 -- 172.21.15.49:0/1054069554 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400cc90 con 0x55ce0e9db060 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97b37fe700 1 -- 172.21.15.49:0/1054069554 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97c409a440 con 0x55ce0e9db060 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97b37fe700 1 -- 172.21.15.49:0/1054069554 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40cd1c0 con 0x55ce0e9db060 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1054069554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 msgr2=0x55ce0e9db480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.407 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9db480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f97c4096940 tx=0x7f97c40968b0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1054069554 shutdown_connections 2023-10-08T09:29:53.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9dc800 0x55ce0e9dcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9db480 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1054069554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.404+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1054069554 >> 172.21.15.49:0/1054069554 conn(0x55ce0e94f260 msgr2=0x55ce0e9d5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:53.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1054069554 shutdown_connections 2023-10-08T09:29:53.408 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1054069554 wait complete. 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 -- start start 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 Processor -- start 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e92de30 con 0x55ce0e9dc800 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e8ca710 con 0x55ce0e9db060 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e8ca3b0 con 0x55ce0e9db9c0 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.405+0000 7f97db7d1b80 10 client.0 _ll_register_callbacks cb 0x55ce0e727800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:53.412 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54226/0 (socket says 172.21.15.49:54226) 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 -- 172.21.15.49:0/3028148988 learned_addr learned my addr 172.21.15.49:0/3028148988 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 -- 172.21.15.49:0/3028148988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 msgr2=0x55ce0e9e53d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e53d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 -- 172.21.15.49:0/3028148988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 msgr2=0x55ce0e9e4e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9e4e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 -- 172.21.15.49:0/3028148988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c8071680 con 0x55ce0e9dc800 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97cdb85700 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec410 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f97c0037000 tx=0x7f97c0009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddf807ec15ef1aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:29:53.413 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3028148988 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0044070 con 0x55ce0e9dc800 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce0e8f5eb0 con 0x55ce0e9dc800 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.406+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3028148988 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f97c0003a00 con 0x55ce0e9dc800 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3028148988 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97c0003bc0 con 0x55ce0e9dc800 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 10 client.17196 fetch_fsmap learned FSMap version 42 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f97c8192190 con 0x55ce0e9dc800 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3028148988 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce0e8f5eb0 con 0x55ce0e9dc800 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3028148988 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f97c000b0d0 con 0x55ce0e9dc800 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 10 client.17196 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 20 client.17196 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:53.414 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0e9f0ce0 0x55ce0e9f3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 4 client.17196 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce0e926e10 con 0x55ce0e9f0ce0 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.407+0000 7f97cd384700 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0e9f0ce0 0x55ce0e9f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.408+0000 7f97cd384700 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0e9f0ce0 0x55ce0e9f3100 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c131eabe63b4d5b server_cookie=685ec122a83a2ec6 in_seq=0 out_seq=0 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.408+0000 7f97b0ff9700 10 client.17196 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.408+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3028148988 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ce0e926e10 con 0x55ce0e9f0ce0 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.408+0000 7f97b0ff9700 10 client.17196 handle_command_reply: tid=0 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.408+0000 7f97db7d1b80 1 client.17196 shutdown 2023-10-08T09:29:53.415 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.408+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9dc800 msgr2=0x55ce0e9ec410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.408+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec410 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f97c0037000 tx=0x7f97c0009180 comp rx=0 tx=0).stop 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 shutdown_connections 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0e9f0ce0 0x55ce0e9f3100 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec410 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3028148988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 >> 172.21.15.49:0/3028148988 conn(0x55ce0e94f260 msgr2=0x55ce0e9d54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97b0ff9700 0 client.17196 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 shutdown_connections 2023-10-08T09:29:53.416 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3028148988 wait complete. 2023-10-08T09:29:53.417 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.409+0000 7f97db7d1b80 20 client.17196 trim_cache size 0 max 16384 2023-10-08T09:29:53.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.433+0000 7f97db7d1b80 1 Processor -- start 2023-10-08T09:29:53.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.433+0000 7f97db7d1b80 1 -- start start 2023-10-08T09:29:53.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.435 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e7ae920 con 0x55ce0e9db9c0 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e9e4a20 con 0x55ce0e9db060 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e9ed590 con 0x55ce0e9dc800 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97ccb83700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43846/0 (socket says 172.21.15.49:43846) 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 -- 172.21.15.49:0/1476311756 learned_addr learned my addr 172.21.15.49:0/1476311756 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 -- 172.21.15.49:0/1476311756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 msgr2=0x55ce0e9ec330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 -- 172.21.15.49:0/1476311756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db9c0 msgr2=0x55ce0e9dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:53.436 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 -- 172.21.15.49:0/1476311756 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce0e741c60 con 0x55ce0e9db060 2023-10-08T09:29:53.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97ccb83700 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:29:53.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.434+0000 7f97cd384700 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9df760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97c40caf90 tx=0x7f97c40091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0abd0b43d9a265 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:53.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.435+0000 7f97b3fff700 1 -- 172.21.15.49:0/1476311756 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40b7040 con 0x55ce0e9db060 2023-10-08T09:29:53.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.435+0000 7f97b3fff700 1 -- 172.21.15.49:0/1476311756 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97c40042b0 con 0x55ce0e9db060 2023-10-08T09:29:53.437 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.435+0000 7f97b3fff700 1 -- 172.21.15.49:0/1476311756 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40cd1c0 con 0x55ce0e9db060 2023-10-08T09:29:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.436+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1476311756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 msgr2=0x55ce0e9df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.436+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9df760 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97c40caf90 tx=0x7f97c40091e0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.447+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1476311756 shutdown_connections 2023-10-08T09:29:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.447+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9ec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.447+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db060 0x55ce0e9df760 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.447+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/1476311756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.447+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1476311756 >> 172.21.15.49:0/1476311756 conn(0x55ce0e94f260 msgr2=0x55ce0e9e2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:53.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1476311756 shutdown_connections 2023-10-08T09:29:53.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 -- 172.21.15.49:0/1476311756 wait complete. 2023-10-08T09:29:53.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 -- start start 2023-10-08T09:29:53.466 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 Processor -- start 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9e8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9eb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e9ebba0 con 0x55ce0e9db060 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e9ebd10 con 0x55ce0e9db9c0 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.458+0000 7f97db7d1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce0e9ebe80 con 0x55ce0e9dc800 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.459+0000 7f97db7d1b80 10 client.0 _ll_register_callbacks cb 0x7f97c00074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97ccb83700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97ccb83700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43854/0 (socket says 172.21.15.49:43854) 2023-10-08T09:29:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97ccb83700 1 -- 172.21.15.49:0/3738847922 learned_addr learned my addr 172.21.15.49:0/3738847922 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:29:53.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97cdb85700 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9eb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97cd384700 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97cdb85700 1 -- 172.21.15.49:0/3738847922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db9c0 msgr2=0x55ce0e9e8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97cdb85700 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97cdb85700 1 -- 172.21.15.49:0/3738847922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 msgr2=0x55ce0e9e8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97cdb85700 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9e8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.465+0000 7f97cdb85700 1 -- 172.21.15.49:0/3738847922 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b800e040 con 0x55ce0e9dc800 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97cdb85700 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9eb660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55ce0e9f06f0 tx=0x7f97c000b290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375b080015690c14 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3738847922 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0044040 con 0x55ce0e9dc800 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3738847922 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f97c0003a00 con 0x55ce0e9dc800 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97c409a440 con 0x55ce0e9dc800 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3738847922 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97c0009b80 con 0x55ce0e9dc800 2023-10-08T09:29:53.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97db7d1b80 10 client.26833 fetch_fsmap learned FSMap version 42 2023-10-08T09:29:53.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.466+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ce0e758de0 con 0x55ce0e9dc800 2023-10-08T09:29:53.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3738847922 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97c409a440 con 0x55ce0e9dc800 2023-10-08T09:29:53.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3738847922 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f97c004d790 con 0x55ce0e9dc800 2023-10-08T09:29:53.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97db7d1b80 10 client.26833 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:29:53.478 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97db7d1b80 10 client.26833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:29:53.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97db7d1b80 20 client.26833 populate_metadata read hostname 'smithi049' 2023-10-08T09:29:53.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0ea457e0 0x55ce0ea47c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:29:53.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97db7d1b80 4 client.26833 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:29:53.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce0e926b30 con 0x55ce0ea457e0 2023-10-08T09:29:53.479 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.467+0000 7f97cd384700 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0ea457e0 0x55ce0ea47c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:29:53.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.468+0000 7f97cd384700 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0ea457e0 0x55ce0ea47c00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e052e20f41aeb0b server_cookie=226669a0d9a8507a in_seq=0 out_seq=0 2023-10-08T09:29:53.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.469+0000 7f97b0ff9700 10 client.26833 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:53.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.469+0000 7f97b0ff9700 1 -- 172.21.15.49:0/3738847922 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ce0e926b30 con 0x55ce0ea457e0 2023-10-08T09:29:53.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.469+0000 7f97b0ff9700 10 client.26833 handle_command_reply: tid=0 2023-10-08T09:29:53.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.470+0000 7f97db7d1b80 1 client.26833 shutdown 2023-10-08T09:29:53.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.480+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 msgr2=0x55ce0e9eb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.487 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.480+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9eb660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55ce0e9f06f0 tx=0x7f97c000b290 comp rx=0 tx=0).stop 2023-10-08T09:29:53.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:53 smithi184 ceph-mon[103675]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 519 op/s 2023-10-08T09:29:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.497+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 shutdown_connections 2023-10-08T09:29:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.497+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55ce0ea457e0 0x55ce0ea47c00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.497+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55ce0e9dc800 0x55ce0e9eb660 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.497+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55ce0e9db060 0x55ce0e9e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.497+0000 7f97db7d1b80 1 --2- 172.21.15.49:0/3738847922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55ce0e9db9c0 0x55ce0e9e8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.497+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 >> 172.21.15.49:0/3738847922 conn(0x55ce0e94f260 msgr2=0x55ce0e9e2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:53.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.497+0000 7f97b0ff9700 0 client.26833 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:29:53.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.508+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 shutdown_connections 2023-10-08T09:29:53.512 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.510+0000 7f97db7d1b80 1 -- 172.21.15.49:0/3738847922 wait complete. 2023-10-08T09:29:53.521 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:29:53.521 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:29:53.522 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:29:53.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.518+0000 7f97db7d1b80 20 client.26833 trim_cache size 0 max 16384 2023-10-08T09:29:53.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.521+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f97b40a1480 msgr2=0x7f97b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.521+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f97b40a1480 0x7f97b40a3940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f97c8068910 tx=0x7f97b8008040 comp rx=0 tx=0).stop 2023-10-08T09:29:53.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.521+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806a7b0 msgr2=0x7f97c80683b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:29:53.523 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.521+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806a7b0 0x7f97c80683b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f97c400eb30 tx=0x7f97c400ee40 comp rx=0 tx=0).stop 2023-10-08T09:29:53.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:53 smithi132 ceph-mon[103389]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 519 op/s 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.536+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 shutdown_connections 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.536+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f97b40a1480 0x7f97b40a3940 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.536+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f97c806c3b0 0x7f97c8191060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.536+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f97c806a7b0 0x7f97c80683b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.536+0000 7f97cf5e8700 1 --2- 172.21.15.49:0/901490967 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c806b9f0 0x7f97c8190a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.536+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 >> 172.21.15.49:0/901490967 conn(0x7f97c80fac10 msgr2=0x7f97c806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.540+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 shutdown_connections 2023-10-08T09:29:53.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:29:53.542+0000 7f97cf5e8700 1 -- 172.21.15.49:0/901490967 wait complete. 2023-10-08T09:29:53.577 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:29:53.581 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:53 smithi049 ceph-mon[99485]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 519 op/s 2023-10-08T09:29:54.316 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 870.0 s, 487.2 tps, lat 210.103 ms stddev 247.576 2023-10-08T09:29:56.301 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:56 smithi132 ceph-mon[103389]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 490 op/s 2023-10-08T09:29:56.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:56 smithi049 ceph-mon[99485]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 490 op/s 2023-10-08T09:29:56.509 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:56 smithi184 ceph-mon[103675]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 490 op/s 2023-10-08T09:29:58.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:58 smithi049 ceph-mon[99485]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2023-10-08T09:29:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:58 smithi184 ceph-mon[103675]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2023-10-08T09:29:58.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:58 smithi132 ceph-mon[103389]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2023-10-08T09:29:59.311 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 875.0 s, 476.0 tps, lat 216.801 ms stddev 258.801 2023-10-08T09:29:59.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:29:59 smithi049 ceph-mon[99485]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 474 op/s 2023-10-08T09:29:59.503 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:29:59 smithi184 ceph-mon[103675]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 474 op/s 2023-10-08T09:29:59.551 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:29:59 smithi132 ceph-mon[103389]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 474 op/s 2023-10-08T09:30:00.502 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:00 smithi184 ceph-mon[103675]: overall HEALTH_OK 2023-10-08T09:30:00.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:00 smithi132 ceph-mon[103389]: overall HEALTH_OK 2023-10-08T09:30:00.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:00 smithi049 ceph-mon[99485]: overall HEALTH_OK 2023-10-08T09:30:01.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:01 smithi184 ceph-mon[103675]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 524 op/s 2023-10-08T09:30:01.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:01 smithi132 ceph-mon[103389]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 524 op/s 2023-10-08T09:30:01.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:01 smithi049 ceph-mon[99485]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 MiB/s wr, 524 op/s 2023-10-08T09:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.533+0000 7f8ab8f48700 1 -- 172.21.15.49:0/3659103582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4108920 msgr2=0x7f8ab410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.533+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/3659103582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4108920 0x7f8ab410f1d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00b0d0 tx=0x7f8aac00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.533+0000 7f8ab8f48700 1 -- 172.21.15.49:0/3659103582 shutdown_connections 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.533+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/3659103582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab4106d90 0x7f8ab41071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.533+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/3659103582 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4108920 0x7f8ab410f1d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.533+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/3659103582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4107f80 0x7f8ab41083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.533+0000 7f8ab8f48700 1 -- 172.21.15.49:0/3659103582 >> 172.21.15.49:0/3659103582 conn(0x7f8ab4075d20 msgr2=0x7f8ab4078160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.534+0000 7f8ab8f48700 1 -- 172.21.15.49:0/3659103582 shutdown_connections 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.534+0000 7f8ab8f48700 1 -- 172.21.15.49:0/3659103582 wait complete. 2023-10-08T09:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.534+0000 7f8ab8f48700 1 Processor -- start 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.534+0000 7f8ab8f48700 1 -- start start 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab8f48700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106d90 0x7f8ab407de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab8f48700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4107f80 0x7f8ab407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab8f48700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab4108920 0x7f8ab4117200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab8f48700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4081410 con 0x7f8ab4106d90 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab8f48700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4081580 con 0x7f8ab4107f80 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab8f48700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab4117740 con 0x7f8ab4108920 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106d90 0x7f8ab407de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106d90 0x7f8ab407de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54248/0 (socket says 172.21.15.49:54248) 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab259c700 1 -- 172.21.15.49:0/2098666219 learned_addr learned my addr 172.21.15.49:0/2098666219 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.535+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4107f80 0x7f8ab407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.546 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab4108920 0x7f8ab4117200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab259c700 1 -- 172.21.15.49:0/2098666219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab4108920 msgr2=0x7f8ab4117200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab259c700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab4108920 0x7f8ab4117200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab259c700 1 -- 172.21.15.49:0/2098666219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4107f80 msgr2=0x7f8ab407e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab259c700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4107f80 0x7f8ab407e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab259c700 1 -- 172.21.15.49:0/2098666219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac009d00 con 0x7f8ab4106d90 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab259c700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106d90 0x7f8ab407de90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800b550 tx=0x7f8aa800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8aa37fe700 1 -- 172.21.15.49:0/2098666219 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8011830 con 0x7f8ab4106d90 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8aa37fe700 1 -- 172.21.15.49:0/2098666219 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa8011e50 con 0x7f8ab4106d90 2023-10-08T09:30:02.547 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab4117960 con 0x7f8ab4106d90 2023-10-08T09:30:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.536+0000 7f8aa37fe700 1 -- 172.21.15.49:0/2098666219 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800f2e0 con 0x7f8ab4106d90 2023-10-08T09:30:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.537+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab4117e70 con 0x7f8ab4106d90 2023-10-08T09:30:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.541+0000 7f8ac5131b80 1 Processor -- start 2023-10-08T09:30:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.541+0000 7f8aa37fe700 1 -- 172.21.15.49:0/2098666219 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8aa800f440 con 0x7f8ab4106d90 2023-10-08T09:30:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.542+0000 7f8aa37fe700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a9c0a15d0 0x7f8a9c0a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.542+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a9c0a15d0 0x7f8a9c0a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.542+0000 7f8aa37fe700 1 -- 172.21.15.49:0/2098666219 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8aa80ccae0 con 0x7f8ab4106d90 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.544+0000 7f8ac5131b80 1 -- start start 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c813e40 0x55923c814260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c814cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c8152b0 0x55923c815730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c77c400 con 0x55923c813e40 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c76dbc0 con 0x55923c8147a0 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c76b4e0 con 0x55923c8152b0 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c813e40 0x55923c814260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c813e40 0x55923c814260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54256/0 (socket says 172.21.15.49:54256) 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 -- 172.21.15.49:0/2178180300 learned_addr learned my addr 172.21.15.49:0/2178180300 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a9c0a15d0 0x7f8a9c0a3a90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007c90 tx=0x7f8aac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c814cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c8152b0 0x55923c815730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 -- 172.21.15.49:0/2178180300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c8152b0 msgr2=0x55923c815730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c8152b0 0x55923c815730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 -- 172.21.15.49:0/2178180300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 msgr2=0x55923c814cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c814cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 -- 172.21.15.49:0/2178180300 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923c591d80 con 0x55923c813e40 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.545+0000 7f8ab259c700 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c813e40 0x55923c814260 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8099770 tx=0x7f8aa8099680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c256cf9a9957abb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8aa0ff9700 1 -- 172.21.15.49:0/2178180300 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa809a170 con 0x55923c813e40 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8aa0ff9700 1 -- 172.21.15.49:0/2178180300 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa8099a70 con 0x55923c813e40 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8aa0ff9700 1 -- 172.21.15.49:0/2178180300 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8099cd0 con 0x55923c813e40 2023-10-08T09:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8ac5131b80 1 -- 172.21.15.49:0/2178180300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c813e40 msgr2=0x55923c814260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c813e40 0x55923c814260 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8099770 tx=0x7f8aa8099680 comp rx=0 tx=0).stop 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8ac5131b80 1 -- 172.21.15.49:0/2178180300 shutdown_connections 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c8152b0 0x55923c815730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c813e40 0x55923c814260 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/2178180300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c814cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.546+0000 7f8ac5131b80 1 -- 172.21.15.49:0/2178180300 >> 172.21.15.49:0/2178180300 conn(0x55923c788200 msgr2=0x55923c80e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 -- 172.21.15.49:0/2178180300 shutdown_connections 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 -- 172.21.15.49:0/2178180300 wait complete. 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 -- start start 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 Processor -- start 2023-10-08T09:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c81dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8152b0 0x55923c824d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c766dd0 con 0x55923c8152b0 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c7036b0 con 0x55923c8147a0 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c703350 con 0x55923c813e40 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ac5131b80 10 client.0 _ll_register_callbacks cb 0x55923c5607a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ab1d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c81dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41296/0 (socket says 172.21.15.49:41296) 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ab1d9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c81dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43878/0 (socket says 172.21.15.49:43878) 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ab259c700 1 -- 172.21.15.49:0/1866514394 learned_addr learned my addr 172.21.15.49:0/1866514394 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.547+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8152b0 0x55923c824d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8ab1d9b700 1 -- 172.21.15.49:0/1866514394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 msgr2=0x55923c81d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8ab1d9b700 1 -- 172.21.15.49:0/1866514394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8152b0 msgr2=0x55923c824d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8152b0 0x55923c824d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8ab1d9b700 1 -- 172.21.15.49:0/1866514394 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab4117e70 con 0x55923c8147a0 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c81dcc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4005950 tx=0x7f8aa404fbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a64fec9cfae0c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8a99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8a99ffb700 1 -- 172.21.15.49:0/1866514394 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa405a070 con 0x55923c8147a0 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55923c72ee50 con 0x55923c8147a0 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.548+0000 7f8a99ffb700 1 -- 172.21.15.49:0/1866514394 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8aa4003a90 con 0x55923c8147a0 2023-10-08T09:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8a99ffb700 1 -- 172.21.15.49:0/1866514394 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aa4003c50 con 0x55923c8147a0 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 10 client.26783 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55923c57ac00 con 0x55923c8147a0 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8a99ffb700 1 -- 172.21.15.49:0/1866514394 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55923c72ee50 con 0x55923c8147a0 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8a99ffb700 1 -- 172.21.15.49:0/1866514394 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8aa4005320 con 0x55923c8147a0 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 10 client.26783 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 10 client.26783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 20 client.26783 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c829610 0x55923c82ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 4 client.26783 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55923c75fdb0 con 0x55923c829610 2023-10-08T09:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.549+0000 7f8ab259c700 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c829610 0x55923c82ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.550+0000 7f8ab259c700 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c829610 0x55923c82ba30 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e780ade1c430eaa server_cookie=1dc15b258b892b29 in_seq=0 out_seq=0 2023-10-08T09:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.550+0000 7f8a99ffb700 10 client.26783 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.550+0000 7f8a99ffb700 1 -- 172.21.15.49:0/1866514394 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55923c75fdb0 con 0x55923c829610 2023-10-08T09:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.550+0000 7f8a99ffb700 10 client.26783 handle_command_reply: tid=0 2023-10-08T09:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.551+0000 7f8ac5131b80 1 client.26783 shutdown 2023-10-08T09:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.551+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 msgr2=0x55923c81dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.551+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c81dcc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4005950 tx=0x7f8aa404fbb0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 shutdown_connections 2023-10-08T09:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c829610 0x55923c82ba30 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8a99ffb700 0 client.26783 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8152b0 0x55923c824d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1866514394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8147a0 0x55923c81dcc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 >> 172.21.15.49:0/1866514394 conn(0x55923c788200 msgr2=0x55923c80e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 shutdown_connections 2023-10-08T09:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1866514394 wait complete. 2023-10-08T09:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.552+0000 7f8ac5131b80 20 client.26783 trim_cache size 0 max 16384 2023-10-08T09:30:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 Processor -- start 2023-10-08T09:30:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 -- start start 2023-10-08T09:30:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c824c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c818200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c5e78c0 con 0x55923c8147a0 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c80b830 con 0x55923c8152b0 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c825e80 con 0x55923c813e40 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c824c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c824c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54270/0 (socket says 172.21.15.49:54270) 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 -- 172.21.15.49:0/1558981567 learned_addr learned my addr 172.21.15.49:0/1558981567 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c818200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab259c700 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 -- 172.21.15.49:0/1558981567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 msgr2=0x55923c81cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 -- 172.21.15.49:0/1558981567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 msgr2=0x55923c818200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c818200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.570+0000 7f8ab1d9b700 1 -- 172.21.15.49:0/1558981567 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923c5b81f0 con 0x55923c8147a0 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.571+0000 7f8ab1d9b700 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c824c20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8aa405f470 tx=0x7f8aa405f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75ca7104a493a14 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.571+0000 7f8aa17fa700 1 -- 172.21.15.49:0/1558981567 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa405a070 con 0x55923c8147a0 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.571+0000 7f8aa17fa700 1 -- 172.21.15.49:0/1558981567 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa405f5c0 con 0x55923c8147a0 2023-10-08T09:30:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.571+0000 7f8aa17fa700 1 -- 172.21.15.49:0/1558981567 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4055450 con 0x55923c8147a0 2023-10-08T09:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.571+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1558981567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 msgr2=0x55923c824c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.571+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c824c20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8aa405f470 tx=0x7f8aa405f4a0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1558981567 shutdown_connections 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c81cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c818200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/1558981567 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c824c20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1558981567 >> 172.21.15.49:0/1558981567 conn(0x55923c788200 msgr2=0x55923c830390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1558981567 shutdown_connections 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- 172.21.15.49:0/1558981567 wait complete. 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- start start 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 Processor -- start 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c822a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c822fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c823f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c824470 con 0x55923c8147a0 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c8245e0 con 0x55923c8152b0 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923c8247a0 con 0x55923c813e40 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ac5131b80 10 client.0 _ll_register_callbacks cb 0x7f8aa40041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c822a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ab1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c822fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ab259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c822a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41308/0 (socket says 172.21.15.49:41308) 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.572+0000 7f8ab2d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c823f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab259c700 1 -- 172.21.15.49:0/3641103308 learned_addr learned my addr 172.21.15.49:0/3641103308 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab1d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c822fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54276/0 (socket says 172.21.15.49:54276) 2023-10-08T09:30:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab2d9d700 1 -- 172.21.15.49:0/3641103308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 msgr2=0x55923c822a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c822a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab2d9d700 1 -- 172.21.15.49:0/3641103308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 msgr2=0x55923c822fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c822fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab2d9d700 1 -- 172.21.15.49:0/3641103308 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa4064040 con 0x55923c8152b0 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8ab2d9d700 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c823f30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007b60 tx=0x7f8aac0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b11c2c889e45c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.573+0000 7f8a99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.574+0000 7f8a99ffb700 1 -- 172.21.15.49:0/3641103308 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007990 con 0x55923c8152b0 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.574+0000 7f8a99ffb700 1 -- 172.21.15.49:0/3641103308 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f8aac022040 con 0x55923c8152b0 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.574+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8aa8099a70 con 0x55923c8152b0 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.574+0000 7f8a99ffb700 1 -- 172.21.15.49:0/3641103308 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aac034410 con 0x55923c8152b0 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.574+0000 7f8ac5131b80 10 client.26789 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.574+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55923c590250 con 0x55923c8152b0 2023-10-08T09:30:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.574+0000 7f8a99ffb700 1 -- 172.21.15.49:0/3641103308 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa8099a70 con 0x55923c8152b0 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8a99ffb700 1 -- 172.21.15.49:0/3641103308 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8aac02a660 con 0x55923c8152b0 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8ac5131b80 10 client.26789 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8ac5131b80 10 client.26789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8ac5131b80 20 client.26789 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c87e480 0x55923c8808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8ac5131b80 4 client.26789 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55923c75fad0 con 0x55923c87e480 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.575+0000 7f8ab259c700 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c87e480 0x55923c8808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.576+0000 7f8a99ffb700 10 client.26789 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.576+0000 7f8ab259c700 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c87e480 0x55923c8808a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d71564221e86d95 server_cookie=d125589791248950 in_seq=0 out_seq=0 2023-10-08T09:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8a99ffb700 1 -- 172.21.15.49:0/3641103308 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55923c75fad0 con 0x55923c87e480 2023-10-08T09:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8a99ffb700 10 client.26789 handle_command_reply: tid=0 2023-10-08T09:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 client.26789 shutdown 2023-10-08T09:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 msgr2=0x55923c823f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c823f30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007b60 tx=0x7f8aac0099e0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 shutdown_connections 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55923c87e480 0x55923c8808a0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55923c813e40 0x55923c822a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55923c8152b0 0x55923c823f30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 --2- 172.21.15.49:0/3641103308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55923c8147a0 0x55923c822fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8a99ffb700 0 client.26789 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 >> 172.21.15.49:0/3641103308 conn(0x55923c788200 msgr2=0x55923c818200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 shutdown_connections 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 1 -- 172.21.15.49:0/3641103308 wait complete. 2023-10-08T09:30:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.585+0000 7f8ac5131b80 20 client.26789 trim_cache size 0 max 16384 2023-10-08T09:30:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:30:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:30:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:30:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:30:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a9c0a15d0 msgr2=0x7f8a9c0a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a9c0a15d0 0x7f8a9c0a3a90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007c90 tx=0x7f8aac00b040 comp rx=0 tx=0).stop 2023-10-08T09:30:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106d90 msgr2=0x7f8ab407de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106d90 0x7f8ab407de90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800b550 tx=0x7f8aa800b860 comp rx=0 tx=0).stop 2023-10-08T09:30:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 shutdown_connections 2023-10-08T09:30:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f8a9c0a15d0 0x7f8a9c0a3a90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ab4108920 0x7f8ab4117200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ab4106d90 0x7f8ab407de90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 --2- 172.21.15.49:0/2098666219 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ab4107f80 0x7f8ab407e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 >> 172.21.15.49:0/2098666219 conn(0x7f8ab4075d20 msgr2=0x7f8ab410b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 shutdown_connections 2023-10-08T09:30:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:02.586+0000 7f8ab8f48700 1 -- 172.21.15.49:0/2098666219 wait complete. 2023-10-08T09:30:03.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:03 smithi132 ceph-mon[103389]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 507 op/s 2023-10-08T09:30:03.635 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:03 smithi049 ceph-mon[99485]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 507 op/s 2023-10-08T09:30:03.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:03 smithi184 ceph-mon[103675]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 507 op/s 2023-10-08T09:30:04.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 880.0 s, 497.8 tps, lat 203.983 ms stddev 224.348 2023-10-08T09:30:06.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:06 smithi132 ceph-mon[103389]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 484 op/s 2023-10-08T09:30:06.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:06 smithi049 ceph-mon[99485]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 484 op/s 2023-10-08T09:30:06.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:06 smithi184 ceph-mon[103675]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 484 op/s 2023-10-08T09:30:08.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:08 smithi049 ceph-mon[99485]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 499 op/s 2023-10-08T09:30:08.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:08 smithi184 ceph-mon[103675]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 499 op/s 2023-10-08T09:30:08.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:08 smithi132 ceph-mon[103389]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.1 MiB/s wr, 499 op/s 2023-10-08T09:30:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 885.0 s, 492.8 tps, lat 200.557 ms stddev 198.563 2023-10-08T09:30:09.388 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:09 smithi049 ceph-mon[99485]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 496 op/s 2023-10-08T09:30:09.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:09 smithi184 ceph-mon[103675]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 496 op/s 2023-10-08T09:30:09.548 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:09 smithi132 ceph-mon[103389]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.4 MiB/s wr, 496 op/s 2023-10-08T09:30:12.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:11 smithi184 ceph-mon[103675]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.1 MiB/s wr, 546 op/s 2023-10-08T09:30:12.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:11 smithi132 ceph-mon[103389]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.1 MiB/s wr, 546 op/s 2023-10-08T09:30:12.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:11 smithi049 ceph-mon[99485]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.1 MiB/s wr, 546 op/s 2023-10-08T09:30:13.636 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:13 smithi049 ceph-mon[99485]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2023-10-08T09:30:13.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:13 smithi184 ceph-mon[103675]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2023-10-08T09:30:13.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:13 smithi132 ceph-mon[103389]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.3 MiB/s wr, 532 op/s 2023-10-08T09:30:14.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 890.0 s, 572.8 tps, lat 173.916 ms stddev 186.126 2023-10-08T09:30:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.873+0000 7f36acc01700 1 -- 172.21.15.49:0/2912420855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 msgr2=0x7f36a810c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.873+0000 7f36acc01700 1 --2- 172.21.15.49:0/2912420855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a810c5f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009f90 tx=0x7f36a000b330 comp rx=0 tx=0).stop 2023-10-08T09:30:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.887+0000 7f36acc01700 1 -- 172.21.15.49:0/2912420855 shutdown_connections 2023-10-08T09:30:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.887+0000 7f36acc01700 1 --2- 172.21.15.49:0/2912420855 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a810cb30 0x7f36a810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.887+0000 7f36acc01700 1 --2- 172.21.15.49:0/2912420855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a81077f0 0x7f36a8109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.887+0000 7f36acc01700 1 --2- 172.21.15.49:0/2912420855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a810c5f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.887+0000 7f36acc01700 1 -- 172.21.15.49:0/2912420855 >> 172.21.15.49:0/2912420855 conn(0x7f36a806d770 msgr2=0x7f36a806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.901+0000 7f36acc01700 1 -- 172.21.15.49:0/2912420855 shutdown_connections 2023-10-08T09:30:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.901+0000 7f36acc01700 1 -- 172.21.15.49:0/2912420855 wait complete. 2023-10-08T09:30:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.901+0000 7f36acc01700 1 Processor -- start 2023-10-08T09:30:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36acc01700 1 -- start start 2023-10-08T09:30:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36acc01700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a81077f0 0x7f36a8115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36acc01700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a8116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36acc01700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a810cb30 0x7f36a81b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36acc01700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a8119390 con 0x7f36a810a160 2023-10-08T09:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36acc01700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a8119500 con 0x7f36a810cb30 2023-10-08T09:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36acc01700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a81b5bd0 con 0x7f36a81077f0 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36a5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a8116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36a5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a8116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54288/0 (socket says 172.21.15.49:54288) 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.902+0000 7f36a5d9b700 1 -- 172.21.15.49:0/1077255452 learned_addr learned my addr 172.21.15.49:0/1077255452 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.903+0000 7f36a659c700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a81077f0 0x7f36a8115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36a5d9b700 1 -- 172.21.15.49:0/1077255452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a81077f0 msgr2=0x7f36a8115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a81077f0 0x7f36a8115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36a5d9b700 1 -- 172.21.15.49:0/1077255452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a810cb30 msgr2=0x7f36a81b5690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a810cb30 0x7f36a81b5690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36a5d9b700 1 -- 172.21.15.49:0/1077255452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a0009c40 con 0x7f36a810a160 2023-10-08T09:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a8116300 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f36a0004c60 tx=0x7f36a0004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36977fe700 1 -- 172.21.15.49:0/1077255452 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a000e040 con 0x7f36a810a160 2023-10-08T09:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a81b5dd0 con 0x7f36a810a160 2023-10-08T09:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a81b61f0 con 0x7f36a810a160 2023-10-08T09:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36977fe700 1 -- 172.21.15.49:0/1077255452 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a000be00 con 0x7f36a810a160 2023-10-08T09:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.904+0000 7f36977fe700 1 -- 172.21.15.49:0/1077255452 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0005260 con 0x7f36a810a160 2023-10-08T09:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.908+0000 7f36977fe700 1 -- 172.21.15.49:0/1077255452 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f36a00053c0 con 0x7f36a810a160 2023-10-08T09:30:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.910+0000 7f36b8deab80 1 Processor -- start 2023-10-08T09:30:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.910+0000 7f36977fe700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f36900a1550 0x7f36900a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.911+0000 7f36977fe700 1 -- 172.21.15.49:0/1077255452 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f36a00cf7c0 con 0x7f36a810a160 2023-10-08T09:30:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.915+0000 7f36b8deab80 1 -- start start 2023-10-08T09:30:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.915+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d3c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.915+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3c7a0 0x55b231d3ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.915+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.915+0000 7f36b8deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231ca4400 con 0x55b231d3d2b0 2023-10-08T09:30:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.915+0000 7f36b8deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231c95bc0 con 0x55b231d3be40 2023-10-08T09:30:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.915+0000 7f36b8deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231c934e0 con 0x55b231d3c7a0 2023-10-08T09:30:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.916+0000 7f36a6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.916+0000 7f36a6d9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d3d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54298/0 (socket says 172.21.15.49:54298) 2023-10-08T09:30:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.916+0000 7f36a6d9d700 1 -- 172.21.15.49:0/661559304 learned_addr learned my addr 172.21.15.49:0/661559304 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f36a6d9d700 1 -- 172.21.15.49:0/661559304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3c7a0 msgr2=0x55b231d3ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:30:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/661559304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3c7a0 0x55b231d3ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f36a6d9d700 1 -- 172.21.15.49:0/661559304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 msgr2=0x55b231d3c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/661559304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d3c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f36a6d9d700 1 -- 172.21.15.49:0/661559304 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b231ab9d80 con 0x55b231d3d2b0 2023-10-08T09:30:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/661559304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d3d730 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f369c00bd00 tx=0x7f369c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0d7638ab672a1b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f3694ff9700 1 -- 172.21.15.49:0/661559304 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c010070 con 0x55b231d3d2b0 2023-10-08T09:30:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f3694ff9700 1 -- 172.21.15.49:0/661559304 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f369c00be50 con 0x55b231d3d2b0 2023-10-08T09:30:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.917+0000 7f3694ff9700 1 -- 172.21.15.49:0/661559304 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c0064c0 con 0x55b231d3d2b0 2023-10-08T09:30:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.918+0000 7f36b8deab80 1 -- 172.21.15.49:0/661559304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 msgr2=0x55b231d3d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.919 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.918+0000 7f36b8deab80 1 --2- 172.21.15.49:0/661559304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d3d730 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f369c00bd00 tx=0x7f369c00d330 comp rx=0 tx=0).stop 2023-10-08T09:30:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.920+0000 7f36b8deab80 1 -- 172.21.15.49:0/661559304 shutdown_connections 2023-10-08T09:30:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.920+0000 7f36b8deab80 1 --2- 172.21.15.49:0/661559304 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d3d730 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.920+0000 7f36b8deab80 1 --2- 172.21.15.49:0/661559304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d3c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.920+0000 7f36b8deab80 1 --2- 172.21.15.49:0/661559304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3c7a0 0x55b231d3ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.920+0000 7f36b8deab80 1 -- 172.21.15.49:0/661559304 >> 172.21.15.49:0/661559304 conn(0x55b231cb0200 msgr2=0x55b231d368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.925+0000 7f36b8deab80 1 -- 172.21.15.49:0/661559304 shutdown_connections 2023-10-08T09:30:15.926 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.925+0000 7f36b8deab80 1 -- 172.21.15.49:0/661559304 wait complete. 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 -- start start 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f36900a1550 0x7f36900a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 Processor -- start 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d45a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d45f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d4afa0 0x55b231d4d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231c8edd0 con 0x55b231d4afa0 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231c2b6b0 con 0x55b231d3be40 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231c2b350 con 0x55b231d3d2b0 2023-10-08T09:30:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36b8deab80 10 client.0 _ll_register_callbacks cb 0x55b231a887a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.926+0000 7f36a5d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d45f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d45f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41338/0 (socket says 172.21.15.49:41338) 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 -- 172.21.15.49:0/487849767 learned_addr learned my addr 172.21.15.49:0/487849767 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a659c700 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d45a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d4afa0 0x55b231d4d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 -- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 msgr2=0x55b231d45a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d45a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 -- 172.21.15.49:0/487849767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d4afa0 msgr2=0x55b231d4d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d4afa0 0x55b231d4d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 -- 172.21.15.49:0/487849767 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c00b9b0 con 0x55b231d3d2b0 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f36900a1550 0x7f36900a3a10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f369c00bcd0 tx=0x7f369c00b690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.927+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d45f70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f36a0004e60 tx=0x7f36a009c140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575387e77cb21f97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.936+0000 7f368dffb700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:30:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.936+0000 7f368dffb700 1 -- 172.21.15.49:0/487849767 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00a0930 con 0x55b231d3d2b0 2023-10-08T09:30:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.936+0000 7f368dffb700 1 -- 172.21.15.49:0/487849767 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f36a00a4ae0 con 0x55b231d3d2b0 2023-10-08T09:30:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.936+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b231c56e50 con 0x55b231d3d2b0 2023-10-08T09:30:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.941+0000 7f368dffb700 1 -- 172.21.15.49:0/487849767 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a00cd040 con 0x55b231d3d2b0 2023-10-08T09:30:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.944+0000 7f36b8deab80 10 client.26851 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.944+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b231aa2c00 con 0x55b231d3d2b0 2023-10-08T09:30:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.944+0000 7f368dffb700 1 -- 172.21.15.49:0/487849767 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b231c56e50 con 0x55b231d3d2b0 2023-10-08T09:30:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.944+0000 7f368dffb700 1 -- 172.21.15.49:0/487849767 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f36a00a4650 con 0x55b231d3d2b0 2023-10-08T09:30:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.945+0000 7f36b8deab80 10 client.26851 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.945+0000 7f36b8deab80 10 client.26851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.945+0000 7f36b8deab80 20 client.26851 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.945+0000 7f36b8deab80 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231d53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.945+0000 7f36b8deab80 4 client.26851 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:30:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.945+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b231c87db0 con 0x55b231d51990 2023-10-08T09:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.947+0000 7f36a659c700 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231d53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.950+0000 7f36a659c700 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231d53db0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c50e52a14a969ac server_cookie=4618e2edff52b14c in_seq=0 out_seq=0 2023-10-08T09:30:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.950+0000 7f368dffb700 10 client.26851 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f368dffb700 1 -- 172.21.15.49:0/487849767 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b231c87db0 con 0x55b231d51990 2023-10-08T09:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f368dffb700 10 client.26851 handle_command_reply: tid=0 2023-10-08T09:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 client.26851 shutdown 2023-10-08T09:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 msgr2=0x55b231d45f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d45f70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f36a0004e60 tx=0x7f36a009c140 comp rx=0 tx=0).stop 2023-10-08T09:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 shutdown_connections 2023-10-08T09:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231d53db0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d4afa0 0x55b231d4d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f368dffb700 0 client.26851 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d45a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 --2- 172.21.15.49:0/487849767 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d45f70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 >> 172.21.15.49:0/487849767 conn(0x55b231cb0200 msgr2=0x55b231d36870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 shutdown_connections 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.951+0000 7f36b8deab80 1 -- 172.21.15.49:0/487849767 wait complete. 2023-10-08T09:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.952+0000 7f36b8deab80 20 client.26851 trim_cache size 0 max 16384 2023-10-08T09:30:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.973+0000 7f36b8deab80 1 Processor -- start 2023-10-08T09:30:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.973+0000 7f36b8deab80 1 -- start start 2023-10-08T09:30:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3be40 0x55b231d53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d3cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d4afa0 0x55b231d61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36b8deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231b0f8c0 con 0x55b231d3be40 2023-10-08T09:30:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36b8deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231d455a0 con 0x55b231d4afa0 2023-10-08T09:30:15.975 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36b8deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231d3d0c0 con 0x55b231d3d2b0 2023-10-08T09:30:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3be40 0x55b231d53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a659c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3be40 0x55b231d53d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54310/0 (socket says 172.21.15.49:54310) 2023-10-08T09:30:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a659c700 1 -- 172.21.15.49:0/139905010 learned_addr learned my addr 172.21.15.49:0/139905010 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d3cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d4afa0 0x55b231d61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a5d9b700 1 -- 172.21.15.49:0/139905010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d4afa0 msgr2=0x55b231d61550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:15.979 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d4afa0 0x55b231d61550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a5d9b700 1 -- 172.21.15.49:0/139905010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3be40 msgr2=0x55b231d53d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3be40 0x55b231d53d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a5d9b700 1 -- 172.21.15.49:0/139905010 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b231ae01f0 con 0x55b231d3d2b0 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a659c700 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3be40 0x55b231d53d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.974+0000 7f36a6d9d700 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d4afa0 0x55b231d61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.975+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d3cb80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36a0099250 tx=0x7f36a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2796f85332b6ece6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.975+0000 7f36957fa700 1 -- 172.21.15.49:0/139905010 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0009470 con 0x55b231d3d2b0 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.975+0000 7f36957fa700 1 -- 172.21.15.49:0/139905010 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a00a4ae0 con 0x55b231d3d2b0 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.975+0000 7f36957fa700 1 -- 172.21.15.49:0/139905010 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00a4650 con 0x55b231d3d2b0 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- 172.21.15.49:0/139905010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 msgr2=0x55b231d3cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.980 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d3cb80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36a0099250 tx=0x7f36a000b330 comp rx=0 tx=0).stop 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- 172.21.15.49:0/139905010 shutdown_connections 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d4afa0 0x55b231d61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3be40 0x55b231d53d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 --2- 172.21.15.49:0/139905010 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d3d2b0 0x55b231d3cb80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- 172.21.15.49:0/139905010 >> 172.21.15.49:0/139905010 conn(0x55b231cb0200 msgr2=0x55b231d586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- 172.21.15.49:0/139905010 shutdown_connections 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- 172.21.15.49:0/139905010 wait complete. 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- start start 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 Processor -- start 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d5e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d5ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.981 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d4afa0 0x55b231d5fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231d602e0 con 0x55b231d3d2b0 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231d60450 con 0x55b231d3be40 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b231d60610 con 0x55b231d4afa0 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36b8deab80 10 client.0 _ll_register_callbacks cb 0x55b231a887a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.976+0000 7f36a5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d5ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d5e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a6d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d4afa0 0x55b231d5fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d5ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54318/0 (socket says 172.21.15.49:54318) 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d5e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43930/0 (socket says 172.21.15.49:43930) 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 -- 172.21.15.49:0/3487765588 learned_addr learned my addr 172.21.15.49:0/3487765588 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 -- 172.21.15.49:0/3487765588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d4afa0 msgr2=0x55b231d5fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:15.982 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d4afa0 0x55b231d5fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 -- 172.21.15.49:0/3487765588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 msgr2=0x55b231d5e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d5e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 -- 172.21.15.49:0/3487765588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3698009c50 con 0x55b231d3d2b0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36a5d9b700 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d5ee30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f36a0007a10 tx=0x7f36a009c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1b16e0928bd8a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f368dffb700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f368dffb700 1 -- 172.21.15.49:0/3487765588 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0009470 con 0x55b231d3d2b0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f368dffb700 1 -- 172.21.15.49:0/3487765588 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f36a00a0930 con 0x55b231d3d2b0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f369c00be50 con 0x55b231d3d2b0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f368dffb700 1 -- 172.21.15.49:0/3487765588 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a009db10 con 0x55b231d3d2b0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36b8deab80 10 client.17238 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.977+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b231aa2c00 con 0x55b231d3d2b0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f368dffb700 1 -- 172.21.15.49:0/3487765588 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f369c00be50 con 0x55b231d3d2b0 2023-10-08T09:30:15.983 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f368dffb700 1 -- 172.21.15.49:0/3487765588 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f36a00b7020 con 0x55b231d3d2b0 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36b8deab80 10 client.17238 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36b8deab80 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36b8deab80 20 client.17238 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36b8deab80 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231da7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36b8deab80 4 client.17238 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b231c87ad0 con 0x55b231d51990 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36a659c700 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231da7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f368dffb700 10 client.17238 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:15.984 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.978+0000 7f36a659c700 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231da7ec0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c18c24cea974862 server_cookie=9218dddc74ba58a4 in_seq=0 out_seq=0 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.990+0000 7f368dffb700 1 -- 172.21.15.49:0/3487765588 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b231c87ad0 con 0x55b231d51990 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f368dffb700 10 client.17238 handle_command_reply: tid=0 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 client.17238 shutdown 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 msgr2=0x55b231d5ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d5ee30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f36a0007a10 tx=0x7f36a009c140 comp rx=0 tx=0).stop 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 shutdown_connections 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x55b231d51990 0x55b231da7ec0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55b231d4afa0 0x55b231d5fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x55b231d3be40 0x55b231d5e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 --2- 172.21.15.49:0/3487765588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55b231d3d2b0 0x55b231d5ee30 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 >> 172.21.15.49:0/3487765588 conn(0x55b231cb0200 msgr2=0x55b231d32c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f368dffb700 0 client.17238 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 shutdown_connections 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 1 -- 172.21.15.49:0/3487765588 wait complete. 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.991+0000 7f36b8deab80 20 client.17238 trim_cache size 0 max 16384 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:30:15.993 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:30:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f36900a1550 msgr2=0x7f36900a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f36900a1550 0x7f36900a3a10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f369c00bcd0 tx=0x7f369c00b690 comp rx=0 tx=0).stop 2023-10-08T09:30:15.994 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 msgr2=0x7f36a8116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a8116300 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f36a0004c60 tx=0x7f36a0004d40 comp rx=0 tx=0).stop 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 shutdown_connections 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f36900a1550 0x7f36900a3a10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a81077f0 0x7f36a8115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a810cb30 0x7f36a81b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 --2- 172.21.15.49:0/1077255452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36a810a160 0x7f36a8116300 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 >> 172.21.15.49:0/1077255452 conn(0x7f36a806d770 msgr2=0x7f36a810a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 shutdown_connections 2023-10-08T09:30:15.995 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:15.992+0000 7f36acc01700 1 -- 172.21.15.49:0/1077255452 wait complete. 2023-10-08T09:30:16.247 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:15 smithi184 ceph-mon[103675]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 502 op/s 2023-10-08T09:30:16.299 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:15 smithi132 ceph-mon[103389]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 502 op/s 2023-10-08T09:30:16.385 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:15 smithi049 ceph-mon[99485]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 502 op/s 2023-10-08T09:30:18.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:18 smithi132 ceph-mon[103389]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 515 op/s 2023-10-08T09:30:18.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:18 smithi049 ceph-mon[99485]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 515 op/s 2023-10-08T09:30:18.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:18 smithi184 ceph-mon[103675]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 515 op/s 2023-10-08T09:30:19.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 895.0 s, 444.6 tps, lat 220.377 ms stddev 248.211 2023-10-08T09:30:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:20 smithi132 ceph-mon[103389]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 489 op/s 2023-10-08T09:30:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:20 smithi049 ceph-mon[99485]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 489 op/s 2023-10-08T09:30:20.508 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:20 smithi184 ceph-mon[103675]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 489 op/s 2023-10-08T09:30:22.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:22 smithi132 ceph-mon[103389]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 544 op/s 2023-10-08T09:30:22.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:22 smithi049 ceph-mon[99485]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 544 op/s 2023-10-08T09:30:22.506 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:22 smithi184 ceph-mon[103675]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 544 op/s 2023-10-08T09:30:23.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:23 smithi132 ceph-mon[103389]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 528 op/s 2023-10-08T09:30:23.582 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-08T09:30:23.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:23 smithi049 ceph-mon[99485]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 528 op/s 2023-10-08T09:30:23.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:23 smithi184 ceph-mon[103675]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 528 op/s 2023-10-08T09:30:23.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.914+0000 7f125a4a9700 1 -- 172.21.15.49:0/2543687364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12540ff080 msgr2=0x7f12540ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:23.915 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.914+0000 7f125a4a9700 1 --2- 172.21.15.49:0/2543687364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12540ff080 0x7f12540ff4a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1244009990 tx=0x7f1244009ca0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.915+0000 7f125a4a9700 1 -- 172.21.15.49:0/2543687364 shutdown_connections 2023-10-08T09:30:23.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.915+0000 7f125a4a9700 1 --2- 172.21.15.49:0/2543687364 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12540ff080 0x7f12540ff4a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.915+0000 7f125a4a9700 1 --2- 172.21.15.49:0/2543687364 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12541003a0 0x7f1254100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.915+0000 7f125a4a9700 1 --2- 172.21.15.49:0/2543687364 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12540ff9e0 0x7f12540ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.915+0000 7f125a4a9700 1 -- 172.21.15.49:0/2543687364 >> 172.21.15.49:0/2543687364 conn(0x7f12540fac40 msgr2=0x7f12540fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:23.916 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.915+0000 7f125a4a9700 1 -- 172.21.15.49:0/2543687364 shutdown_connections 2023-10-08T09:30:23.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.915+0000 7f125a4a9700 1 -- 172.21.15.49:0/2543687364 wait complete. 2023-10-08T09:30:23.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 Processor -- start 2023-10-08T09:30:23.917 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 -- start start 2023-10-08T09:30:23.918 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12540ff080 0x7f12541989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12540ff9e0 0x7f1254198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12541003a0 0x7f125419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541a0520 con 0x7f12541003a0 2023-10-08T09:30:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541a0660 con 0x7f12540ff9e0 2023-10-08T09:30:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.916+0000 7f125a4a9700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541a07f0 con 0x7f12540ff080 2023-10-08T09:30:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12541003a0 0x7f125419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.919 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12541003a0 0x7f125419ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54334/0 (socket says 172.21.15.49:54334) 2023-10-08T09:30:23.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 -- 172.21.15.49:0/4234491579 learned_addr learned my addr 172.21.15.49:0/4234491579 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:23.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 -- 172.21.15.49:0/4234491579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12540ff080 msgr2=0x7f12541989f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:30:23.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12540ff080 0x7f12541989f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 -- 172.21.15.49:0/4234491579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12540ff9e0 msgr2=0x7f1254198f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T09:30:23.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12540ff9e0 0x7f1254198f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.920 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 -- 172.21.15.49:0/4234491579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12440095f0 con 0x7f12541003a0 2023-10-08T09:30:23.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f1258a46700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12541003a0 0x7f125419ff50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b780 tx=0x7f124c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:23.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f12517fa700 1 -- 172.21.15.49:0/4234491579 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00f840 con 0x7f12541003a0 2023-10-08T09:30:23.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12541012f0 con 0x7f12541003a0 2023-10-08T09:30:23.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f12517fa700 1 -- 172.21.15.49:0/4234491579 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f124c00fe60 con 0x7f12541003a0 2023-10-08T09:30:23.921 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f12517fa700 1 -- 172.21.15.49:0/4234491579 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c00d3d0 con 0x7f12541003a0 2023-10-08T09:30:23.922 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.917+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1254101800 con 0x7f12541003a0 2023-10-08T09:30:23.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.921+0000 7f12517fa700 1 -- 172.21.15.49:0/4234491579 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f124c017690 con 0x7f12541003a0 2023-10-08T09:30:23.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.922+0000 7f12517fa700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f123c0a1500 0x7f123c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.926 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.922+0000 7f12517fa700 1 -- 172.21.15.49:0/4234491579 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f124c0ccf40 con 0x7f12541003a0 2023-10-08T09:30:23.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.922+0000 7f1266692b80 1 Processor -- start 2023-10-08T09:30:23.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1266692b80 1 -- start start 2023-10-08T09:30:23.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1266692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fd4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1266692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd49d0 0x556988fd5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.927 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1266692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fd5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1266692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f2b460 con 0x556988fd4070 2023-10-08T09:30:23.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1266692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f1cc20 con 0x556988fd5810 2023-10-08T09:30:23.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1266692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f1a540 con 0x556988fd49d0 2023-10-08T09:30:23.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f12537fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd49d0 0x556988fd5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f12537fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd49d0 0x556988fd5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41372/0 (socket says 172.21.15.49:41372) 2023-10-08T09:30:23.928 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f12537fe700 1 -- 172.21.15.49:0/3282385845 learned_addr learned my addr 172.21.15.49:0/3282385845 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.923+0000 7f1253fff700 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fd4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f123c0a1500 0x7f123c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 -- 172.21.15.49:0/3282385845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd49d0 msgr2=0x556988fd5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd49d0 0x556988fd5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 -- 172.21.15.49:0/3282385845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 msgr2=0x556988fd5ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T09:30:23.929 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fd5ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 -- 172.21.15.49:0/3282385845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556988d40de0 con 0x556988fd4070 2023-10-08T09:30:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fd4490 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f124400b680 tx=0x7f124400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a47dfde4057a98e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f1253fff700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f123c0a1500 0x7f123c0a39c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f124400b2e0 tx=0x7f124401d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f123a7fc700 1 -- 172.21.15.49:0/3282385845 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400b950 con 0x556988fd4070 2023-10-08T09:30:23.930 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f123a7fc700 1 -- 172.21.15.49:0/3282385845 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1244004500 con 0x556988fd4070 2023-10-08T09:30:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.924+0000 7f123a7fc700 1 -- 172.21.15.49:0/3282385845 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400f6a0 con 0x556988fd4070 2023-10-08T09:30:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 -- 172.21.15.49:0/3282385845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 msgr2=0x556988fd4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fd4490 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f124400b680 tx=0x7f124400bbb0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 -- 172.21.15.49:0/3282385845 shutdown_connections 2023-10-08T09:30:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fd5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.931 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fd4490 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 --2- 172.21.15.49:0/3282385845 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd49d0 0x556988fd5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 -- 172.21.15.49:0/3282385845 >> 172.21.15.49:0/3282385845 conn(0x556988f37260 msgr2=0x556988fbde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:23.932 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.925+0000 7f1266692b80 1 -- 172.21.15.49:0/3282385845 shutdown_connections 2023-10-08T09:30:23.936 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.930+0000 7f1266692b80 1 -- 172.21.15.49:0/3282385845 wait complete. 2023-10-08T09:30:23.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.931+0000 7f1266692b80 1 -- start start 2023-10-08T09:30:23.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.931+0000 7f1266692b80 1 Processor -- start 2023-10-08T09:30:23.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.931+0000 7f1266692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1266692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 0x556988fc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.937 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1266692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fcb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1266692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f15e30 con 0x556988fd49d0 2023-10-08T09:30:23.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1266692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988eb2710 con 0x556988fd5810 2023-10-08T09:30:23.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1266692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988eb23b0 con 0x556988fd4070 2023-10-08T09:30:23.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1266692b80 10 client.0 _ll_register_callbacks cb 0x556988d0f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:23.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1258a46700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fcb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.938 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1258a46700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fcb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43960/0 (socket says 172.21.15.49:43960) 2023-10-08T09:30:23.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1258a46700 1 -- 172.21.15.49:0/1482631609 learned_addr learned my addr 172.21.15.49:0/1482631609 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:23.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.932+0000 7f1253fff700 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f1253fff700 1 -- 172.21.15.49:0/1482631609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 msgr2=0x556988fcb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:23.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f1253fff700 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fcb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.939 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f1253fff700 1 -- 172.21.15.49:0/1482631609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 msgr2=0x556988fc4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:23.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f1253fff700 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 0x556988fc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f1253fff700 1 -- 172.21.15.49:0/1482631609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1244020040 con 0x556988fd4070 2023-10-08T09:30:23.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f1253fff700 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc4430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1244009ce0 tx=0x7f12440055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845d0fda62057654 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:23.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f12337fe700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:30:23.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f12337fe700 1 -- 172.21.15.49:0/1482631609 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440058e0 con 0x556988fd4070 2023-10-08T09:30:23.940 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556988eddeb0 con 0x556988fd4070 2023-10-08T09:30:23.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.933+0000 7f12337fe700 1 -- 172.21.15.49:0/1482631609 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1244004670 con 0x556988fd4070 2023-10-08T09:30:23.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.934+0000 7f12337fe700 1 -- 172.21.15.49:0/1482631609 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1244004830 con 0x556988fd4070 2023-10-08T09:30:23.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.934+0000 7f1266692b80 10 client.26866 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:23.941 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.934+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x556988d29c60 con 0x556988fd4070 2023-10-08T09:30:23.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.934+0000 7f12337fe700 1 -- 172.21.15.49:0/1482631609 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556988eddeb0 con 0x556988fd4070 2023-10-08T09:30:23.942 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.934+0000 7f12337fe700 1 -- 172.21.15.49:0/1482631609 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1244037070 con 0x556988fd4070 2023-10-08T09:30:23.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.935+0000 7f1266692b80 10 client.26866 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:23.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.935+0000 7f1266692b80 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:23.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.935+0000 7f1266692b80 20 client.26866 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:23.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.935+0000 7f1266692b80 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fcfe00 0x556988fd2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.935+0000 7f1266692b80 4 client.26866 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:30:23.943 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.935+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556988f0ee10 con 0x556988fcfe00 2023-10-08T09:30:23.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.935+0000 7f12537fe700 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fcfe00 0x556988fd2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.938+0000 7f12337fe700 10 client.26866 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:23.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.938+0000 7f12537fe700 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fcfe00 0x556988fd2220 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b952f4d70560b80 server_cookie=3cf1c096d36fb276 in_seq=0 out_seq=0 2023-10-08T09:30:23.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.938+0000 7f12337fe700 1 -- 172.21.15.49:0/1482631609 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556988f0ee10 con 0x556988fcfe00 2023-10-08T09:30:23.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.938+0000 7f12337fe700 10 client.26866 handle_command_reply: tid=0 2023-10-08T09:30:23.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.938+0000 7f1266692b80 1 client.26866 shutdown 2023-10-08T09:30:23.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 msgr2=0x556988fc4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:23.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc4430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1244009ce0 tx=0x7f12440055f0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 shutdown_connections 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fcfe00 0x556988fd2220 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc4430 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f12337fe700 0 client.26866 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd5810 0x556988fcb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 --2- 172.21.15.49:0/1482631609 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 0x556988fc4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 >> 172.21.15.49:0/1482631609 conn(0x556988f37260 msgr2=0x556988fbd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 shutdown_connections 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 1 -- 172.21.15.49:0/1482631609 wait complete. 2023-10-08T09:30:23.946 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.939+0000 7f1266692b80 20 client.26866 trim_cache size 0 max 16384 2023-10-08T09:30:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 Processor -- start 2023-10-08T09:30:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 -- start start 2023-10-08T09:30:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fcfe00 0x556988fc3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fcb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.968 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd49d0 0x556988fc41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988d96920 con 0x556988fd4070 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988f2b460 con 0x556988fd49d0 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.967+0000 7f1266692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988fd5110 con 0x556988fcfe00 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd49d0 0x556988fc41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd49d0 0x556988fc41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43970/0 (socket says 172.21.15.49:43970) 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 -- 172.21.15.49:0/3254254372 learned_addr learned my addr 172.21.15.49:0/3254254372 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1253fff700 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fcfe00 0x556988fc3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.969 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f12537fe700 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fcb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 -- 172.21.15.49:0/3254254372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fcfe00 msgr2=0x556988fc3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:23.970 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fcfe00 0x556988fc3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 -- 172.21.15.49:0/3254254372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 msgr2=0x556988fcb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fcb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1258a46700 1 -- 172.21.15.49:0/3254254372 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556988d67250 con 0x556988fd49d0 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.968+0000 7f1253fff700 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fcfe00 0x556988fc3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.969+0000 7f1258a46700 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd49d0 0x556988fc41e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f124c096380 tx=0x7f124c0998b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1503cea805da0bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.969+0000 7f12537fe700 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fcb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.969+0000 7f123affd700 1 -- 172.21.15.49:0/3254254372 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c09e020 con 0x556988fd49d0 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.969+0000 7f123affd700 1 -- 172.21.15.49:0/3254254372 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f124c0a2260 con 0x556988fd49d0 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.969+0000 7f123affd700 1 -- 172.21.15.49:0/3254254372 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c09a390 con 0x556988fd49d0 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 -- 172.21.15.49:0/3254254372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd49d0 msgr2=0x556988fc41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:23.971 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd49d0 0x556988fc41e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f124c096380 tx=0x7f124c0998b0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 -- 172.21.15.49:0/3254254372 shutdown_connections 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fcfe00 0x556988fc3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fd49d0 0x556988fc41e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 --2- 172.21.15.49:0/3254254372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd4070 0x556988fcb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 -- 172.21.15.49:0/3254254372 >> 172.21.15.49:0/3254254372 conn(0x556988f37260 msgr2=0x556988fdbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 -- 172.21.15.49:0/3254254372 shutdown_connections 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 -- 172.21.15.49:0/3254254372 wait complete. 2023-10-08T09:30:23.972 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 -- start start 2023-10-08T09:30:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.970+0000 7f1266692b80 1 Processor -- start 2023-10-08T09:30:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1266692b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fcfe00 0x556988fc8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1266692b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1266692b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 0x556988fca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.975 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1266692b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988fca600 con 0x556988fd49d0 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1266692b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988fca770 con 0x556988fcfe00 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1266692b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556988fca930 con 0x556988fd4070 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1266692b80 10 client.0 _ll_register_callbacks cb 0x7f1248009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fcfe00 0x556988fc8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fcfe00 0x556988fc8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43972/0 (socket says 172.21.15.49:43972) 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 -- 172.21.15.49:0/1603040801 learned_addr learned my addr 172.21.15.49:0/1603040801 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f12537fe700 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 -- 172.21.15.49:0/1603040801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 msgr2=0x556988fc9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 -- 172.21.15.49:0/1603040801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 msgr2=0x556988fca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 0x556988fca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.976 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f1253fff700 1 -- 172.21.15.49:0/1603040801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1254101800 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.971+0000 7f12537fe700 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f1253fff700 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fcfe00 0x556988fc8c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1244000c00 tx=0x7f124400bb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e637619ae31df87 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f12337fe700 10 client.0 ms_handle_connect on v2:172.21.15.132:3300/0 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f12337fe700 1 -- 172.21.15.49:0/1603040801 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244004a80 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f12337fe700 1 -- 172.21.15.49:0/1603040801 <== mon.2 v2:172.21.15.132:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1244039830 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1244004500 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f12337fe700 1 -- 172.21.15.49:0/1603040801 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12440399f0 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f1266692b80 10 client.26810 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x556988d3f2b0 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f12337fe700 1 -- 172.21.15.49:0/1603040801 <== mon.2 v2:172.21.15.132:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1244004500 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.972+0000 7f12337fe700 1 -- 172.21.15.49:0/1603040801 <== mon.2 v2:172.21.15.132:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1244041070 con 0x556988fcfe00 2023-10-08T09:30:23.977 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f1266692b80 10 client.26810 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f1266692b80 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f1266692b80 20 client.26810 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f1266692b80 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fd5810 0x5569890519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f1266692b80 4 client.26810 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556988f0eb30 con 0x556988fd5810 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f12537fe700 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fd5810 0x5569890519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f12337fe700 10 client.26810 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.973+0000 7f12537fe700 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fd5810 0x5569890519e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a457bb78665fc53 server_cookie=f1072530541c5ae4 in_seq=0 out_seq=0 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f12337fe700 1 -- 172.21.15.49:0/1603040801 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556988f0eb30 con 0x556988fd5810 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f12337fe700 10 client.26810 handle_command_reply: tid=0 2023-10-08T09:30:23.978 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 client.26810 shutdown 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fcfe00 msgr2=0x556988fc8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fcfe00 0x556988fc8c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1244000c00 tx=0x7f124400bb80 comp rx=0 tx=0).stop 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 shutdown_connections 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x556988fd5810 0x5569890519e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556988fd49d0 0x556988fca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x556988fcfe00 0x556988fc8c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 --2- 172.21.15.49:0/1603040801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556988fd4070 0x556988fc9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f12337fe700 0 client.26810 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 >> 172.21.15.49:0/1603040801 conn(0x556988f37260 msgr2=0x556988fdb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 shutdown_connections 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 1 -- 172.21.15.49:0/1603040801 wait complete. 2023-10-08T09:30:23.979 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f1266692b80 20 client.26810 trim_cache size 0 max 16384 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "scrubs": { 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "11fa72c1-d842-473a-b9e5-10890ebefc1b": { 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "path": "/", 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "tag": "11fa72c1-d842-473a-b9e5-10890ebefc1b", 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "options": "recursive,force" 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:23.980 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:30:23.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f123c0a1500 msgr2=0x7f123c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:23.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f125a4a9700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f123c0a1500 0x7f123c0a39c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f124400b2e0 tx=0x7f124401d000 comp rx=0 tx=0).stop 2023-10-08T09:30:23.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12541003a0 msgr2=0x7f125419ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:23.981 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f125a4a9700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12541003a0 0x7f125419ff50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b780 tx=0x7f124c00bb40 comp rx=0 tx=0).stop 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 shutdown_connections 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.974+0000 7f125a4a9700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f123c0a1500 0x7f123c0a39c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.975+0000 7f125a4a9700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12540ff080 0x7f12541989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.975+0000 7f125a4a9700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f12541003a0 0x7f125419ff50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.975+0000 7f125a4a9700 1 --2- 172.21.15.49:0/4234491579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12540ff9e0 0x7f1254198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.975+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 >> 172.21.15.49:0/4234491579 conn(0x7f12540fac40 msgr2=0x7f12540fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.975+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 shutdown_connections 2023-10-08T09:30:23.982 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:23.975+0000 7f125a4a9700 1 -- 172.21.15.49:0/4234491579 wait complete. 2023-10-08T09:30:23.991 INFO:tasks.cephfs.filesystem:scrub status for tag:11fa72c1-d842-473a-b9e5-10890ebefc1b - {'path': '/', 'tag': '11fa72c1-d842-473a-b9e5-10890ebefc1b', 'options': 'recursive,force'} 2023-10-08T09:30:23.992 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-10-08T09:30:24.309 INFO:teuthology.orchestra.run.smithi049.stderr:progress: 900.0 s, 495.0 tps, lat 203.463 ms stddev 246.851 2023-10-08T09:30:24.703 INFO:teuthology.orchestra.run.smithi049.stdout:transaction type: 2023-10-08T09:30:24.704 INFO:teuthology.orchestra.run.smithi049.stdout:scaling factor: 32 2023-10-08T09:30:24.704 INFO:teuthology.orchestra.run.smithi049.stdout:query mode: simple 2023-10-08T09:30:24.704 INFO:teuthology.orchestra.run.smithi049.stdout:number of clients: 100 2023-10-08T09:30:24.704 INFO:teuthology.orchestra.run.smithi049.stdout:number of threads: 4 2023-10-08T09:30:24.704 INFO:teuthology.orchestra.run.smithi049.stdout:duration: 900 s 2023-10-08T09:30:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:number of transactions actually processed: 751947 2023-10-08T09:30:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:latency average = 119.712 ms 2023-10-08T09:30:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:latency stddev = 170.941 ms 2023-10-08T09:30:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:tps = 835.131084 (including connections establishing) 2023-10-08T09:30:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:tps = 835.133368 (excluding connections establishing) 2023-10-08T09:30:24.707 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-10-08T09:30:26.393 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:26 smithi049 ceph-mon[99485]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 500 op/s 2023-10-08T09:30:26.498 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:26 smithi184 ceph-mon[103675]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 500 op/s 2023-10-08T09:30:26.553 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:26 smithi132 ceph-mon[103389]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 500 op/s 2023-10-08T09:30:28.505 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:28 smithi184 ceph-mon[103675]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 538 op/s 2023-10-08T09:30:28.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:28 smithi132 ceph-mon[103389]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 538 op/s 2023-10-08T09:30:28.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:28 smithi049 ceph-mon[99485]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 538 op/s 2023-10-08T09:30:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.235+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/4210802188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d381077f0 msgr2=0x7f4d38109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.235+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/4210802188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d381077f0 0x7f4d38109c20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0098f0 tx=0x7f4d2c009c00 comp rx=0 tx=0).stop 2023-10-08T09:30:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.236+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/4210802188 shutdown_connections 2023-10-08T09:30:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.236+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/4210802188 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d3810cb30 0x7f4d3810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.236+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/4210802188 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d381077f0 0x7f4d38109c20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.236+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/4210802188 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d3810a160 0x7f4d3810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.236+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/4210802188 >> 172.21.15.49:0/4210802188 conn(0x7f4d3806d770 msgr2=0x7f4d3806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.248+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/4210802188 shutdown_connections 2023-10-08T09:30:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.253+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/4210802188 wait complete. 2023-10-08T09:30:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.253+0000 7f4d3ffbd700 1 Processor -- start 2023-10-08T09:30:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.255+0000 7f4d3ffbd700 1 -- start start 2023-10-08T09:30:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.255+0000 7f4d3ffbd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d381077f0 0x7f4d38117da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.255+0000 7f4d3ffbd700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d3810cb30 0x7f4d381182e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.255+0000 7f4d3ffbd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d38118820 0x7f4d3811f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.255+0000 7f4d3ffbd700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d381200a0 con 0x7f4d3810cb30 2023-10-08T09:30:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.255+0000 7f4d3ffbd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38120210 con 0x7f4d38118820 2023-10-08T09:30:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.255+0000 7f4d3ffbd700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d381203d0 con 0x7f4d381077f0 2023-10-08T09:30:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.266+0000 7f4d3e55a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d38118820 0x7f4d3811f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.266+0000 7f4d3dd59700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d381077f0 0x7f4d38117da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.266+0000 7f4d3e55a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d38118820 0x7f4d3811f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:43986/0 (socket says 172.21.15.49:43986) 2023-10-08T09:30:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.266+0000 7f4d3e55a700 1 -- 172.21.15.49:0/277965995 learned_addr learned my addr 172.21.15.49:0/277965995 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.266+0000 7f4d3d558700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d3810cb30 0x7f4d381182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.267+0000 7f4d3dd59700 1 -- 172.21.15.49:0/277965995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d38118820 msgr2=0x7f4d3811f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.267+0000 7f4d3dd59700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d38118820 0x7f4d3811f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.267+0000 7f4d3dd59700 1 -- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d3810cb30 msgr2=0x7f4d381182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.267+0000 7f4d3dd59700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d3810cb30 0x7f4d381182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.267+0000 7f4d3dd59700 1 -- 172.21.15.49:0/277965995 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c009520 con 0x7f4d381077f0 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.268+0000 7f4d3dd59700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d381077f0 0x7f4d38117da0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00b2e0 tx=0x7f4d2c00b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.274+0000 7f4d2affd700 1 -- 172.21.15.49:0/277965995 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c01c070 con 0x7f4d381077f0 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.274+0000 7f4d2affd700 1 -- 172.21.15.49:0/277965995 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d2c00bc50 con 0x7f4d381077f0 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.274+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d381205f0 con 0x7f4d381077f0 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.274+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d38120a20 con 0x7f4d381077f0 2023-10-08T09:30:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.274+0000 7f4d2affd700 1 -- 172.21.15.49:0/277965995 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c003df0 con 0x7f4d381077f0 2023-10-08T09:30:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.279+0000 7f4d4c1a6b80 1 Processor -- start 2023-10-08T09:30:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.279+0000 7f4d4c1a6b80 1 -- start start 2023-10-08T09:30:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.280+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501969e40 0x56450196a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.280+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56450196a7a0 0x56450196acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.281 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.280+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450196b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.280+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645018d2400 con 0x56450196a7a0 2023-10-08T09:30:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.280+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645018c3bc0 con 0x56450196b2b0 2023-10-08T09:30:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.280+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645018c14e0 con 0x564501969e40 2023-10-08T09:30:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.281+0000 7f4d3e55a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501969e40 0x56450196a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.281+0000 7f4d3e55a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501969e40 0x56450196a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41414/0 (socket says 172.21.15.49:41414) 2023-10-08T09:30:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.281+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2482680123 learned_addr learned my addr 172.21.15.49:0/2482680123 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.290+0000 7f4d3d558700 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450196b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.290 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.290+0000 7f4d3dd59700 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56450196a7a0 0x56450196acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.292 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.292+0000 7f4d2affd700 1 -- 172.21.15.49:0/277965995 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4d2c005050 con 0x7f4d381077f0 2023-10-08T09:30:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.293+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2482680123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 msgr2=0x56450196b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.293+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450196b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.293 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.293+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2482680123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56450196a7a0 msgr2=0x56450196acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.293+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56450196a7a0 0x56450196acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.293+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2482680123 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645016e7d80 con 0x564501969e40 2023-10-08T09:30:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.293+0000 7f4d2affd700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4d240a1600 0x7f4d240a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.293+0000 7f4d2affd700 1 -- 172.21.15.49:0/277965995 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4d2c0d3320 con 0x7f4d381077f0 2023-10-08T09:30:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.294+0000 7f4d3dd59700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4d240a1600 0x7f4d240a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.295 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.295+0000 7f4d3dd59700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4d240a1600 0x7f4d240a3ac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0a0010 tx=0x7f4d2c0d09c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.315+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501969e40 0x56450196a260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d30008630 tx=0x7f4d30008940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f92155cc128e02a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.326+0000 7f4d23fff700 1 -- 172.21.15.49:0/2482680123 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30003c00 con 0x564501969e40 2023-10-08T09:30:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.326+0000 7f4d23fff700 1 -- 172.21.15.49:0/2482680123 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d30010040 con 0x564501969e40 2023-10-08T09:30:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.326+0000 7f4d23fff700 1 -- 172.21.15.49:0/2482680123 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d300127f0 con 0x564501969e40 2023-10-08T09:30:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.327+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2482680123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501969e40 msgr2=0x56450196a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.327+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501969e40 0x56450196a260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4d30008630 tx=0x7f4d30008940 comp rx=0 tx=0).stop 2023-10-08T09:30:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.327+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2482680123 shutdown_connections 2023-10-08T09:30:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.327+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564501969e40 0x56450196a260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.327+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450196b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.327+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2482680123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x56450196a7a0 0x56450196acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.327+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2482680123 >> 172.21.15.49:0/2482680123 conn(0x5645018de200 msgr2=0x5645019648a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:29.348 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.348+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2482680123 shutdown_connections 2023-10-08T09:30:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.359+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2482680123 wait complete. 2023-10-08T09:30:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.359+0000 7f4d4c1a6b80 1 -- start start 2023-10-08T09:30:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.359+0000 7f4d4c1a6b80 1 Processor -- start 2023-10-08T09:30:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.365+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501973890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.365+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196a7a0 0x564501973dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.365+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450197ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.365+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645018bcdd0 con 0x564501969e40 2023-10-08T09:30:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.365+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645018596b0 con 0x56450196b2b0 2023-10-08T09:30:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.365+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564501859350 con 0x56450196a7a0 2023-10-08T09:30:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.365+0000 7f4d4c1a6b80 10 client.0 _ll_register_callbacks cb 0x5645016b67a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.369+0000 7f4d3d558700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196a7a0 0x564501973dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.372+0000 7f4d3d558700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196a7a0 0x564501973dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41420/0 (socket says 172.21.15.49:41420) 2023-10-08T09:30:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.372+0000 7f4d3d558700 1 -- 172.21.15.49:0/3678613276 learned_addr learned my addr 172.21.15.49:0/3678613276 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.378+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450197ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.378+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501973890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.380+0000 7f4d3e55a700 1 -- 172.21.15.49:0/3678613276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196a7a0 msgr2=0x564501973dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.380+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196a7a0 0x564501973dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.380+0000 7f4d3e55a700 1 -- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 msgr2=0x56450197ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.380+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450197ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.380+0000 7f4d3e55a700 1 -- 172.21.15.49:0/3678613276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d30007e50 con 0x564501969e40 2023-10-08T09:30:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.381+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501973890 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4d30008c90 tx=0x7f4d30000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89514cd04c0aaf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.382+0000 7f4d217fa700 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2023-10-08T09:30:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.393+0000 7f4d217fa700 1 -- 172.21.15.49:0/3678613276 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30012d50 con 0x564501969e40 2023-10-08T09:30:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.393+0000 7f4d217fa700 1 -- 172.21.15.49:0/3678613276 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4d30012eb0 con 0x564501969e40 2023-10-08T09:30:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.394+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564501884e50 con 0x564501969e40 2023-10-08T09:30:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.396+0000 7f4d217fa700 1 -- 172.21.15.49:0/3678613276 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d3002a450 con 0x564501969e40 2023-10-08T09:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.404+0000 7f4d4c1a6b80 10 client.17262 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.404+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5645016d0c00 con 0x564501969e40 2023-10-08T09:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.415+0000 7f4d217fa700 1 -- 172.21.15.49:0/3678613276 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564501884e50 con 0x564501969e40 2023-10-08T09:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.415+0000 7f4d217fa700 1 -- 172.21.15.49:0/3678613276 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4d300143f0 con 0x564501969e40 2023-10-08T09:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.416+0000 7f4d4c1a6b80 10 client.17262 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.416+0000 7f4d4c1a6b80 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.416+0000 7f4d4c1a6b80 20 client.17262 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.416+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56450197f690 0x564501981ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.416+0000 7f4d4c1a6b80 4 client.17262 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T09:30:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.416+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645018b5db0 con 0x56450197f690 2023-10-08T09:30:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.438+0000 7f4d3d558700 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56450197f690 0x564501981ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.450+0000 7f4d3d558700 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56450197f690 0x564501981ab0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7654a7b10649cdc9 server_cookie=abc80c5c062963fd in_seq=0 out_seq=0 2023-10-08T09:30:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.450+0000 7f4d217fa700 10 client.17262 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:29.464 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.461+0000 7f4d217fa700 1 -- 172.21.15.49:0/3678613276 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5645018b5db0 con 0x56450197f690 2023-10-08T09:30:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.461+0000 7f4d217fa700 10 client.17262 handle_command_reply: tid=0 2023-10-08T09:30:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.467+0000 7f4d4c1a6b80 1 client.17262 shutdown 2023-10-08T09:30:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.485+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 msgr2=0x564501973890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.485+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501973890 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4d30008c90 tx=0x7f4d30000f40 comp rx=0 tx=0).stop 2023-10-08T09:30:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.496+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 shutdown_connections 2023-10-08T09:30:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.496+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x56450197f690 0x564501981ab0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.496+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196b2b0 0x56450197ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.496+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501973890 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.496+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/3678613276 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196a7a0 0x564501973dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.496 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.496+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 >> 172.21.15.49:0/3678613276 conn(0x5645018de200 msgr2=0x564501964010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.498+0000 7f4d217fa700 0 client.17262 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.505+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 shutdown_connections 2023-10-08T09:30:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.508+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/3678613276 wait complete. 2023-10-08T09:30:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.517+0000 7f4d4c1a6b80 20 client.17262 trim_cache size 0 max 16384 2023-10-08T09:30:29.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:29 smithi132 ceph-mon[103389]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 584 op/s 2023-10-08T09:30:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 Processor -- start 2023-10-08T09:30:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 -- start start 2023-10-08T09:30:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x56450196b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x56450197ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x564501964740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450173d8c0 con 0x564501969e40 2023-10-08T09:30:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450197bfc0 con 0x56450196a7a0 2023-10-08T09:30:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.551+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645019748a0 con 0x56450196b2b0 2023-10-08T09:30:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.552+0000 7f4d3d558700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x56450197ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.552+0000 7f4d3d558700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x56450197ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:44008/0 (socket says 172.21.15.49:44008) 2023-10-08T09:30:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.552+0000 7f4d3d558700 1 -- 172.21.15.49:0/2858833635 learned_addr learned my addr 172.21.15.49:0/2858833635 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.553+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x56450196b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.561 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.561+0000 7f4d3d558700 1 -- 172.21.15.49:0/2858833635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 msgr2=0x564501964740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.561+0000 7f4d3d558700 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x564501964740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.561+0000 7f4d3d558700 1 -- 172.21.15.49:0/2858833635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 msgr2=0x56450196b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.561+0000 7f4d3d558700 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x56450196b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.561+0000 7f4d3d558700 1 -- 172.21.15.49:0/2858833635 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56450170e1f0 con 0x56450196a7a0 2023-10-08T09:30:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.573+0000 7f4d3d558700 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x56450197ada0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d340046e0 tx=0x7f4d3400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30fba1ececc6ed3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.590+0000 7f4d28ff9700 1 -- 172.21.15.49:0/2858833635 <== mon.2 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400bc10 con 0x56450196a7a0 2023-10-08T09:30:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.590+0000 7f4d28ff9700 1 -- 172.21.15.49:0/2858833635 <== mon.2 v2:172.21.15.132:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d34017040 con 0x56450196a7a0 2023-10-08T09:30:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.590+0000 7f4d28ff9700 1 -- 172.21.15.49:0/2858833635 <== mon.2 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34003ce0 con 0x56450196a7a0 2023-10-08T09:30:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.591+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2858833635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 msgr2=0x56450197ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.591+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x56450197ada0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d340046e0 tx=0x7f4d3400b390 comp rx=0 tx=0).stop 2023-10-08T09:30:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.602+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2858833635 shutdown_connections 2023-10-08T09:30:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.602+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x564501964740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.602+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x56450196b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.602+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2858833635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x56450197ada0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.602+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2858833635 >> 172.21.15.49:0/2858833635 conn(0x5645018de200 msgr2=0x564501972da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:29.613 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.613+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2858833635 shutdown_connections 2023-10-08T09:30:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2858833635 wait complete. 2023-10-08T09:30:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 -- start start 2023-10-08T09:30:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 Processor -- start 2023-10-08T09:30:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501976fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x564501977510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x56450197a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450197a670 con 0x564501969e40 2023-10-08T09:30:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.624+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450197a7e0 con 0x56450196a7a0 2023-10-08T09:30:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.625+0000 7f4d4c1a6b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56450197a9a0 con 0x56450196b2b0 2023-10-08T09:30:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.625+0000 7f4d4c1a6b80 10 client.0 _ll_register_callbacks cb 0x7f4d34004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T09:30:29.634 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.634+0000 7f4d3e55a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x56450197a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.634+0000 7f4d3e55a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501976fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.634+0000 7f4d3e55a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501976fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54400/0 (socket says 172.21.15.49:54400) 2023-10-08T09:30:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.634+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2834935692 learned_addr learned my addr 172.21.15.49:0/2834935692 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.634+0000 7f4d3d558700 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x564501977510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.635+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 msgr2=0x564501977510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.635 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.635+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x564501977510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.635+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2834935692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 msgr2=0x564501976fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T09:30:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.635+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501976fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:29.636 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.635+0000 7f4d3e55a700 1 -- 172.21.15.49:0/2834935692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d34004550 con 0x56450196b2b0 2023-10-08T09:30:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.638+0000 7f4d3e55a700 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x56450197a130 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d30008c90 tx=0x7f4d30007d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aaa79d7d1a1d1fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.638+0000 7f4d217fa700 10 client.0 ms_handle_connect on v2:172.21.15.184:3300/0 2023-10-08T09:30:29.641 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:29 smithi049 ceph-mon[99485]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 584 op/s 2023-10-08T09:30:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.646+0000 7f4d217fa700 1 -- 172.21.15.49:0/2834935692 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30003a00 con 0x56450196b2b0 2023-10-08T09:30:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.647+0000 7f4d217fa700 1 -- 172.21.15.49:0/2834935692 <== mon.1 v2:172.21.15.184:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4d30003b60 con 0x56450196b2b0 2023-10-08T09:30:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.647+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d30010040 con 0x56450196b2b0 2023-10-08T09:30:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.649+0000 7f4d217fa700 1 -- 172.21.15.49:0/2834935692 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d30003d20 con 0x56450196b2b0 2023-10-08T09:30:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.654+0000 7f4d4c1a6b80 10 client.26887 fetch_fsmap learned FSMap version 42 2023-10-08T09:30:29.654 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.654+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5645016e6250 con 0x56450196b2b0 2023-10-08T09:30:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.654+0000 7f4d217fa700 1 -- 172.21.15.49:0/2834935692 <== mon.1 v2:172.21.15.184:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d3003c030 con 0x56450196b2b0 2023-10-08T09:30:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.654+0000 7f4d217fa700 1 -- 172.21.15.49:0/2834935692 <== mon.1 v2:172.21.15.184:3300/0 5 ==== fsmap(e 42) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4d30014d20 con 0x56450196b2b0 2023-10-08T09:30:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.654+0000 7f4d4c1a6b80 10 client.26887 fetch_fsmap finished waiting for FSMap version 42 2023-10-08T09:30:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.655+0000 7f4d4c1a6b80 10 client.26887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-10-08T09:30:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.655+0000 7f4d4c1a6b80 20 client.26887 populate_metadata read hostname 'smithi049' 2023-10-08T09:30:29.655 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.655+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5645019d41f0 0x5645019d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.655+0000 7f4d4c1a6b80 4 client.26887 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-10-08T09:30:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.655+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 --> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645018b5ad0 con 0x5645019d41f0 2023-10-08T09:30:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.664+0000 7f4d3d558700 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5645019d41f0 0x5645019d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.676+0000 7f4d3d558700 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5645019d41f0 0x5645019d65e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6098eb1830075 server_cookie=b653e829a41120b5 in_seq=0 out_seq=0 2023-10-08T09:30:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:29.676+0000 7f4d217fa700 10 client.26887 ms_handle_connect on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:29.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:29 smithi184 ceph-mon[103675]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 584 op/s 2023-10-08T09:30:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d217fa700 1 -- 172.21.15.49:0/2834935692 <== mds.0 v2:172.21.15.132:6838/1629770277 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645018b5ad0 con 0x5645019d41f0 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d217fa700 10 client.26887 handle_command_reply: tid=0 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 client.26887 shutdown 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 msgr2=0x56450197a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x56450197a130 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d30008c90 tx=0x7f4d30007d80 comp rx=0 tx=0).stop 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 shutdown_connections 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:6838/1629770277,v1:172.21.15.132:6839/1629770277] conn(0x5645019d41f0 0x5645019d65e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56450196b2b0 0x56450197a130 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x564501969e40 0x564501976fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 --2- 172.21.15.49:0/2834935692 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x56450196a7a0 0x564501977510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.211 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.209+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 >> 172.21.15.49:0/2834935692 conn(0x5645018de200 msgr2=0x564501971760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.210+0000 7f4d217fa700 0 client.26887 ms_handle_reset on v2:172.21.15.132:6838/1629770277 2023-10-08T09:30:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.210+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 shutdown_connections 2023-10-08T09:30:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.210+0000 7f4d4c1a6b80 1 -- 172.21.15.49:0/2834935692 wait complete. 2023-10-08T09:30:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.210+0000 7f4d4c1a6b80 20 client.26887 trim_cache size 0 max 16384 2023-10-08T09:30:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:{ 2023-10-08T09:30:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "message": "", 2023-10-08T09:30:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout: "return_code": 0 2023-10-08T09:30:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stdout:} 2023-10-08T09:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.226+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4d240a1600 msgr2=0x7f4d240a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.226+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4d240a1600 0x7f4d240a3ac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0a0010 tx=0x7f4d2c0d09c0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.226+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d381077f0 msgr2=0x7f4d38117da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.226+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d381077f0 0x7f4d38117da0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00b2e0 tx=0x7f4d2c00b820 comp rx=0 tx=0).stop 2023-10-08T09:30:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.231+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 shutdown_connections 2023-10-08T09:30:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.231+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4d240a1600 0x7f4d240a3ac0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.232 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.231+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4d381077f0 0x7f4d38117da0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.231+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d38118820 0x7f4d3811f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.231+0000 7f4d3ffbd700 1 --2- 172.21.15.49:0/277965995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4d3810cb30 0x7f4d381182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.231+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 >> 172.21.15.49:0/277965995 conn(0x7f4d3806d770 msgr2=0x7f4d3810a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.232+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 shutdown_connections 2023-10-08T09:30:30.233 INFO:teuthology.task.background_exec.ubuntu@smithi049.front.sepia.ceph.com.smithi049.stderr:2023-10-08T09:30:30.232+0000 7f4d3ffbd700 1 -- 172.21.15.49:0/277965995 wait complete. 2023-10-08T09:30:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:32 smithi132 ceph-mon[103389]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-10-08T09:30:32.396 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:32 smithi049 ceph-mon[99485]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-10-08T09:30:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:32 smithi184 ceph-mon[103675]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.7 MiB/s wr, 621 op/s 2023-10-08T09:30:33.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:33 smithi049 ceph-mon[99485]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.1 MiB/s wr, 712 op/s 2023-10-08T09:30:33.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:33 smithi184 ceph-mon[103675]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.1 MiB/s wr, 712 op/s 2023-10-08T09:30:33.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:33 smithi132 ceph-mon[103389]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 7.1 MiB/s wr, 712 op/s 2023-10-08T09:30:36.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:36 smithi132 ceph-mon[103389]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 MiB/s wr, 693 op/s 2023-10-08T09:30:36.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:36 smithi049 ceph-mon[99485]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 MiB/s wr, 693 op/s 2023-10-08T09:30:36.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:36 smithi184 ceph-mon[103675]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 MiB/s wr, 693 op/s 2023-10-08T09:30:36.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-08T09:30:36.657 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/: 2023-10-08T09:30:36.657 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 .. 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 data 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data: 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:total 31 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 . 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 .. 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 8 09:15 base 2023-10-08T09:30:36.658 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Oct 8 09:15 current_logfiles 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 8 09:15 global 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 log 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_commit_ts 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_dynshmem 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 8 09:15 pg_hba.conf 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 8 09:15 pg_ident.conf 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:30 pg_logical 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 pg_multixact 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 pg_notify 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_replslot 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_serial 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_snapshots 2023-10-08T09:30:36.659 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:30 pg_stat 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:30 pg_stat_tmp 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:30 pg_subtrans 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_tblspc 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pg_twophase 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 8 09:30 pg_wal 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 pg_xact 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 8 09:15 postgresql.auto.conf 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 8 09:15 postgresql.conf 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Oct 8 09:15 postmaster.opts 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.660 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base: 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 8 09:15 . 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 1 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 12346 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 8 09:15 12347 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 pgsql_tmp 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:total 7185 2023-10-08T09:30:36.661 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 . 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 8 09:15 .. 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 112 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 113 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 8 09:15 12182 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12182_fsm 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12182_vm 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12184 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12186 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187 2023-10-08T09:30:36.662 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12187_fsm 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187_vm 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12189 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12191 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12192_fsm 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192_vm 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12194 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12196 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12197_fsm 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197_vm 2023-10-08T09:30:36.663 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12199 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12201 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12202_fsm 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202_vm 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12204 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12206 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12207_fsm 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207_vm 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12209 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12211 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12212 2023-10-08T09:30:36.664 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12214 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12216 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 8 09:15 1247 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1247_fsm 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1247_vm 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 8 09:15 1249 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1249_fsm 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1249_vm 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 8 09:15 1255 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1255_fsm 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1255_vm 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 8 09:15 1259 2023-10-08T09:30:36.665 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1259_fsm 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1259_vm 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417_vm 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418_vm 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 174 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 175 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2187 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224_vm 2023-10-08T09:30:36.666 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328_vm 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336_vm 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2337 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2600 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2600_fsm 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2600_vm 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2601_fsm 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601_vm 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2602 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2602_fsm 2023-10-08T09:30:36.667 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2602_vm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2603 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2603_fsm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2603_vm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604_vm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2605 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2605_fsm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2605_vm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2606_fsm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606_vm 2023-10-08T09:30:36.668 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607_fsm 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2607_vm 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 8 09:15 2608 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2608_fsm 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2608_vm 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 8 09:15 2609 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2609_fsm 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2609_vm 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610_fsm 2023-10-08T09:30:36.669 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2610_vm 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611_vm 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2612_fsm 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612_vm 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613_vm 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2615_fsm 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615_vm 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616 2023-10-08T09:30:36.670 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616_fsm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2616_vm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 8 09:15 2617 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2617_fsm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2617_vm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 8 09:15 2618 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2618_fsm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2618_vm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 8 09:15 2619 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2619_fsm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2619_vm 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620 2023-10-08T09:30:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620_vm 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2650 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2651 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2652 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2653 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2654 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2655 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2656 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2657 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 8 09:15 2658 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2659 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2660 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2661 2023-10-08T09:30:36.672 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2662 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2663 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2664 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2665 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2666 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2667 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2668 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2669 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2670 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 8 09:15 2673 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 8 09:15 2674 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 8 09:15 2675 2023-10-08T09:30:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2678 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2679 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2680 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2681 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2682 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2683 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2684 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2685 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2686 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2687 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2688 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2689 2023-10-08T09:30:36.674 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2690 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 8 09:15 2691 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2692 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2693 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2696 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2699 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2701 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2702 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2703 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2704 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2753 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2753_fsm 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2753_vm 2023-10-08T09:30:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2754 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2755 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2756 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2757 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830_vm 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2831 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832_vm 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2833 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834 2023-10-08T09:30:36.676 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834_vm 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2835 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836_vm 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2837 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 8 09:15 2838 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2838_fsm 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2838_vm 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2839 2023-10-08T09:30:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840_fsm 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2840_vm 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2841 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995_vm 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2996 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3079_fsm 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079_vm 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3080 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3081 2023-10-08T09:30:36.678 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3085 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118_vm 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3119 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3164 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256_vm 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3257 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3258 2023-10-08T09:30:36.679 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350_vm 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3351 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3379 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3380 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381_vm 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3394 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3394_fsm 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3394_vm 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3395 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439_vm 2023-10-08T09:30:36.680 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3440 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 3455 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3456 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3456_fsm 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3456_vm 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466_vm 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3467 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3468 2023-10-08T09:30:36.681 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501_vm 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3502 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3503 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3534 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3541_fsm 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541_vm 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3542 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3574 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3575 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576 2023-10-08T09:30:36.682 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576_vm 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596_vm 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3597 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598_vm 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3599 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3600_fsm 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600_vm 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3601_fsm 2023-10-08T09:30:36.683 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601_vm 2023-10-08T09:30:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602 2023-10-08T09:30:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3602_fsm 2023-10-08T09:30:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602_vm 2023-10-08T09:30:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3603 2023-10-08T09:30:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3603_fsm 2023-10-08T09:30:36.684 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3603_vm 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3604 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3605 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3606 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3607 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3608 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 3609 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3712 2023-10-08T09:30:36.686 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3764_fsm 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764_vm 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3766 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3767 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3997 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 5002 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 548 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 549 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102_vm 2023-10-08T09:30:36.687 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104_vm 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106_vm 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6110 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6111 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6112 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6113 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6117 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826_vm 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 827 2023-10-08T09:30:36.688 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 828 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:total 7185 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 8 09:15 . 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 8 09:15 .. 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 112 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 113 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 8 09:15 12182 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12182_fsm 2023-10-08T09:30:36.689 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12182_vm 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12184 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12186 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12187_fsm 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187_vm 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12189 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12191 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12192_fsm 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192_vm 2023-10-08T09:30:36.690 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12194 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12196 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12197_fsm 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197_vm 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12199 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12201 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12202_fsm 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202_vm 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12204 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12206 2023-10-08T09:30:36.691 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12207_fsm 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207_vm 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12209 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12211 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12212 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12214 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12216 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 8 09:15 1247 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1247_fsm 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1247_vm 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 8 09:15 1249 2023-10-08T09:30:36.692 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1249_fsm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1249_vm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 8 09:15 1255 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1255_fsm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1255_vm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 8 09:15 1259 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1259_fsm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1259_vm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417_vm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418_vm 2023-10-08T09:30:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 174 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 175 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2187 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224_vm 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328_vm 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336_vm 2023-10-08T09:30:36.694 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2337 2023-10-08T09:30:36.695 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2600 2023-10-08T09:30:36.695 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2600_fsm 2023-10-08T09:30:36.695 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2600_vm 2023-10-08T09:30:36.695 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601 2023-10-08T09:30:36.695 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2601_fsm 2023-10-08T09:30:36.696 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601_vm 2023-10-08T09:30:36.696 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2602 2023-10-08T09:30:36.696 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2602_fsm 2023-10-08T09:30:36.696 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2602_vm 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2603 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2603_fsm 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2603_vm 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604_vm 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2605 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2605_fsm 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2605_vm 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2606_fsm 2023-10-08T09:30:36.697 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606_vm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607_fsm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2607_vm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 8 09:15 2608 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2608_fsm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2608_vm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 8 09:15 2609 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2609_fsm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2609_vm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610_fsm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2610_vm 2023-10-08T09:30:36.698 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611_vm 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2612_fsm 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612_vm 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613_vm 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2615_fsm 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615_vm 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616 2023-10-08T09:30:36.699 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616_fsm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2616_vm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 8 09:15 2617 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2617_fsm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2617_vm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 8 09:15 2618 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2618_fsm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2618_vm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 8 09:15 2619 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2619_fsm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2619_vm 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620 2023-10-08T09:30:36.700 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620_vm 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2650 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2651 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2652 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2653 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2654 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2655 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2656 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2657 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 8 09:15 2658 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2659 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2660 2023-10-08T09:30:36.701 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2661 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2662 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2663 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2664 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2665 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2666 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2667 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2668 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2669 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2670 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 8 09:15 2673 2023-10-08T09:30:36.702 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 8 09:15 2674 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 8 09:15 2675 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2678 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2679 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2680 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2681 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2682 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2683 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2684 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2685 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2686 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2687 2023-10-08T09:30:36.703 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2688 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2689 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2690 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 8 09:15 2691 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2692 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2693 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2696 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2699 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2701 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2702 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2703 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2704 2023-10-08T09:30:36.704 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2753 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2753_fsm 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2753_vm 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2754 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2755 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2756 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2757 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830_vm 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2831 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832_vm 2023-10-08T09:30:36.705 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2833 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834_vm 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2835 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836_vm 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2837 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 8 09:15 2838 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2838_fsm 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2838_vm 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2839 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840 2023-10-08T09:30:36.706 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840_fsm 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2840_vm 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2841 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995_vm 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2996 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3079_fsm 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079_vm 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3080 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3081 2023-10-08T09:30:36.707 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3085 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118_vm 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3119 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3164 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256_vm 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3257 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3258 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350_vm 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3351 2023-10-08T09:30:36.708 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3379 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3380 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381_vm 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3394 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3394_fsm 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3394_vm 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3395 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439_vm 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3440 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 3455 2023-10-08T09:30:36.709 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3456 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3456_fsm 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3456_vm 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466_vm 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3467 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3468 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501_vm 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3502 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3503 2023-10-08T09:30:36.710 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3534 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3541_fsm 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541_vm 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3542 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3574 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3575 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576_vm 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596_vm 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3597 2023-10-08T09:30:36.711 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598_vm 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3599 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3600_fsm 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600_vm 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3601_fsm 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601_vm 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3602_fsm 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602_vm 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3603 2023-10-08T09:30:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3603_fsm 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3603_vm 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3604 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3605 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3606 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3607 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3608 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 3609 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3712 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3764_fsm 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764_vm 2023-10-08T09:30:36.713 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3766 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3767 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3997 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 5002 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 548 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 549 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102_vm 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104_vm 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106_vm 2023-10-08T09:30:36.714 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6110 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6111 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6112 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6113 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6117 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826_vm 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 827 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 828 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:30:36.715 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:total 557080 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 8 09:15 . 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 8 09:15 .. 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 112 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 113 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 8 09:15 12182 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12182_fsm 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12182_vm 2023-10-08T09:30:36.716 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12184 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12186 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12187_fsm 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12187_vm 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12189 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12191 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12192_fsm 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12192_vm 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12194 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12196 2023-10-08T09:30:36.717 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12197_fsm 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12197_vm 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12199 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12201 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12202_fsm 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12202_vm 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12204 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12206 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 12207_fsm 2023-10-08T09:30:36.718 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12207_vm 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12209 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12211 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12212 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 12214 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 12216 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 8 09:15 1247 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1247_fsm 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1247_vm 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 8 09:15 1249 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1249_fsm 2023-10-08T09:30:36.719 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1249_vm 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 8 09:15 1255 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1255_fsm 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1255_vm 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 8 09:30 1259 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1259_fsm 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1259_vm 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1417_vm 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 1418_vm 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3850240 Oct 8 09:30 16387 2023-10-08T09:30:36.720 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:30 16387_fsm 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:30 16387_vm 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3031040 Oct 8 09:30 16393 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:30 16393_fsm 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:30 16393_vm 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443351040 Oct 8 09:30 16396 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 8 09:23 16396_fsm 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:24 16396_vm 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:30 16397 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 229376 Oct 8 09:30 16399 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Oct 8 09:30 16401 2023-10-08T09:30:36.721 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40230912 Oct 8 09:30 16403 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:30 16403_fsm 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 174 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 175 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2187 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2224_vm 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2328_vm 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2336_vm 2023-10-08T09:30:36.722 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2337 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2600 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2600_fsm 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2600_vm 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2601_fsm 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2601_vm 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2602 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2602_fsm 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2602_vm 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2603 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2603_fsm 2023-10-08T09:30:36.723 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2603_vm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2604_vm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2605 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2605_fsm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2605_vm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2606_fsm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2606_vm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2607_fsm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2607_vm 2023-10-08T09:30:36.724 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Oct 8 09:15 2608 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2608_fsm 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2608_vm 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 8 09:15 2609 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2609_fsm 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2609_vm 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2610 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2610_fsm 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2610_vm 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2611_vm 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612 2023-10-08T09:30:36.725 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2612_fsm 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2612_vm 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2613_vm 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2615_fsm 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2615_vm 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2616_fsm 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2616_vm 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 8 09:15 2617 2023-10-08T09:30:36.726 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2617_fsm 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2617_vm 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 8 09:15 2618 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2618_fsm 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2618_vm 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Oct 8 09:30 2619 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:17 2619_fsm 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2619_vm 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2620_vm 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2650 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2651 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2652 2023-10-08T09:30:36.727 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2653 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2654 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2655 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2656 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2657 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Oct 8 09:15 2658 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2659 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2660 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2661 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2662 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2663 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2664 2023-10-08T09:30:36.728 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2665 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2666 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2667 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2668 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2669 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2670 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Oct 8 09:15 2673 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 8 09:15 2674 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 8 09:15 2675 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2678 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2679 2023-10-08T09:30:36.729 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2680 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2681 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2682 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2683 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2684 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2685 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2686 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2687 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2688 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 8 09:15 2689 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 8 09:15 2690 2023-10-08T09:30:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 8 09:15 2691 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2692 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2693 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:19 2696 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2699 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2701 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2702 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2703 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 2704 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2753 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2753_fsm 2023-10-08T09:30:36.731 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2753_vm 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2754 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2755 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2756 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 2757 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2830_vm 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2831 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2832_vm 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2833 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834 2023-10-08T09:30:36.732 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2834_vm 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2835 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2836_vm 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2837 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 8 09:15 2838 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2838_fsm 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2838_vm 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2839 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2840_fsm 2023-10-08T09:30:36.733 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2840_vm 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2841 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2995_vm 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2996 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3079_fsm 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3079_vm 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3080 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3081 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3085 2023-10-08T09:30:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3118_vm 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3119 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3164 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3256_vm 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3257 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3258 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3350_vm 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3351 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3379 2023-10-08T09:30:36.735 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3380 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3381_vm 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3394 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3394_fsm 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3394_vm 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3395 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3439_vm 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3440 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 8 09:15 3455 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3456 2023-10-08T09:30:36.736 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3456_fsm 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3456_vm 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3466_vm 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3467 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3468 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3501_vm 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3502 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3503 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3534 2023-10-08T09:30:36.737 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3541_fsm 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3541_vm 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3542 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3574 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3575 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3576_vm 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3596_vm 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3597 2023-10-08T09:30:36.738 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3598_vm 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3599 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3600_fsm 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3600_vm 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3601_fsm 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3601_vm 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3602_fsm 2023-10-08T09:30:36.739 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3602_vm 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3603 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3603_fsm 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3603_vm 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3604 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3605 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3606 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3607 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3608 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 8 09:15 3609 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3712 2023-10-08T09:30:36.740 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 3764_fsm 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3764_vm 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3766 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 3767 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3997 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 5002 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 548 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 549 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6102_vm 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6104_vm 2023-10-08T09:30:36.741 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6106_vm 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6110 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6111 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6112 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6113 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6117 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 826_vm 2023-10-08T09:30:36.742 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 827 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 828 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Oct 8 09:15 pg_internal.init 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:15 PG_VERSION 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 8 09:15 .. 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/global: 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:total 574 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 8 09:15 . 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1136 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1136_fsm 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1136_vm 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 1137 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1213 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1213_fsm 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1213_vm 2023-10-08T09:30:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1214 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1214_fsm 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1214_vm 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 1232 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 1233 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1260 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1260_fsm 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1260_vm 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1261 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1261_fsm 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1261_vm 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1262 2023-10-08T09:30:36.746 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 1262_fsm 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 1262_vm 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2396 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 8 09:15 2396_fsm 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2396_vm 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2397 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2671 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2672 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2676 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2677 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2694 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2695 2023-10-08T09:30:36.747 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2697 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 8 09:15 2698 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2846 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2846_vm 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2847 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2964 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2964_vm 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2965 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2966 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 2966_vm 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 2967 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3592 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 3592_vm 2023-10-08T09:30:36.748 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 3593 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 4060 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 4060_vm 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 4061 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6000 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6000_vm 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6001 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6002 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6100 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 6100_vm 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6114 2023-10-08T09:30:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 6115 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:30 pg_control 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 8 09:15 pg_filenode.map 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Oct 8 09:15 pg_internal.init 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/log: 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:total 2 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1120 Oct 8 09:30 postgresql-Sun.log 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.750 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:total 1 2023-10-08T09:30:36.751 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:30 . 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 mappings 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 8 09:30 replorigin_checkpoint 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 snapshots 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:30 .. 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.752 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:30 .. 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 . 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 members 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 offsets 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:30:36.753 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 .. 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 8 09:15 .. 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-08T09:30:36.754 INFO:teuthology.orchestra.run.smithi049.stdout:total 8 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 8 09:15 0000 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-08T09:30:36.755 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:total 13 2023-10-08T09:30:36.756 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 8 09:30 . 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Oct 8 09:30 db_0.stat 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Oct 8 09:30 db_12347.stat 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Oct 8 09:30 global.stat 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:30 . 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-08T09:30:36.757 INFO:teuthology.orchestra.run.smithi049.stdout:total 128 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:30 . 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 8 09:30 000B 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-08T09:30:36.758 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:total 1032192 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 8 09:30 . 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E2 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E3 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E4 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E5 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E6 2023-10-08T09:30:36.759 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E7 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E8 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000E9 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000EA 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000EB 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000EC 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:29 0000000100000000000000ED 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:30 0000000100000000000000EE 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:30 0000000100000000000000EF 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:30 0000000100000000000000F0 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:30 0000000100000000000000F1 2023-10-08T09:30:36.760 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:22 0000000100000000000000F2 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:22 0000000100000000000000F3 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 0000000100000000000000F4 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 0000000100000000000000F5 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:25 0000000100000000000000F6 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:22 0000000100000000000000F7 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:22 0000000100000000000000F8 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:25 0000000100000000000000F9 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 0000000100000000000000FA 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 0000000100000000000000FB 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 0000000100000000000000FC 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 0000000100000000000000FD 2023-10-08T09:30:36.761 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:25 0000000100000000000000FE 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 0000000100000000000000FF 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:24 000000010000000100000000 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 000000010000000100000001 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 000000010000000100000002 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:24 000000010000000100000003 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:23 000000010000000100000004 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:24 000000010000000100000005 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:24 000000010000000100000006 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:22 000000010000000100000007 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:27 000000010000000100000008 2023-10-08T09:30:36.762 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:25 000000010000000100000009 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:28 00000001000000010000000A 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 00000001000000010000000B 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:27 00000001000000010000000C 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 00000001000000010000000D 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:27 00000001000000010000000E 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:27 00000001000000010000000F 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 000000010000000100000010 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:25 000000010000000100000011 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:25 000000010000000100000012 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:28 000000010000000100000013 2023-10-08T09:30:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 000000010000000100000014 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 000000010000000100000015 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:27 000000010000000100000016 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 000000010000000100000017 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 000000010000000100000018 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:28 000000010000000100000019 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 00000001000000010000001A 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:28 00000001000000010000001B 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:25 00000001000000010000001C 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 00000001000000010000001D 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 00000001000000010000001E 2023-10-08T09:30:36.764 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:27 00000001000000010000001F 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 8 09:26 000000010000000100000020 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 archive_status 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:total 0 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 8 09:15 . 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 8 09:30 .. 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:total 184 2023-10-08T09:30:36.765 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 8 09:15 . 2023-10-08T09:30:36.766 INFO:teuthology.orchestra.run.smithi049.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 8 09:30 .. 2023-10-08T09:30:36.766 INFO:teuthology.orchestra.run.smithi049.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 Oct 8 09:30 0000 2023-10-08T09:30:36.766 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-10-08T09:30:36.777 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:30:36.786 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-08T09:30:36.840 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-08T09:30:36.840 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_7b3143bae3b6870a749bfb9810ed6e7d0e89d7de/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-10-08T09:30:36.841 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-08T09:30:36.892 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:30:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 -- 172.21.15.49:0/2900586400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c0ff5a0 msgr2=0x7fae1c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 --2- 172.21.15.49:0/2900586400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c0ff9c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0099c0 tx=0x7fae0c009cd0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.019 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 -- 172.21.15.49:0/2900586400 shutdown_connections 2023-10-08T09:30:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 --2- 172.21.15.49:0/2900586400 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c101130 0x7fae1c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 --2- 172.21.15.49:0/2900586400 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c0ff9c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 --2- 172.21.15.49:0/2900586400 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c100790 0x7fae1c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 -- 172.21.15.49:0/2900586400 >> 172.21.15.49:0/2900586400 conn(0x7fae1c0fab10 msgr2=0x7fae1c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 -- 172.21.15.49:0/2900586400 shutdown_connections 2023-10-08T09:30:37.020 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.017+0000 7fae2082e700 1 -- 172.21.15.49:0/2900586400 wait complete. 2023-10-08T09:30:37.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.018+0000 7fae2082e700 1 Processor -- start 2023-10-08T09:30:37.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.018+0000 7fae2082e700 1 -- start start 2023-10-08T09:30:37.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae2082e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae2082e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c100790 0x7fae1c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.022 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae2082e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c101130 0x7fae1c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae2082e700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c19e120 con 0x7fae1c100790 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae2082e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c19e290 con 0x7fae1c0ff5a0 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae2082e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c19e450 con 0x7fae1c101130 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae19d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c100790 0x7fae1c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae1a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae1a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:44026/0 (socket says 172.21.15.49:44026) 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae1a59c700 1 -- 172.21.15.49:0/4278757440 learned_addr learned my addr 172.21.15.49:0/4278757440 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae1ad9d700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c101130 0x7fae1c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae19d9b700 1 -- 172.21.15.49:0/4278757440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c101130 msgr2=0x7fae1c19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae19d9b700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c101130 0x7fae1c19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae19d9b700 1 -- 172.21.15.49:0/4278757440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c0ff5a0 msgr2=0x7fae1c068350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae19d9b700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c068350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae19d9b700 1 -- 172.21.15.49:0/4278757440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0095f0 con 0x7fae1c100790 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae1ad9d700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c101130 0x7fae1c19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae1a59c700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae19d9b700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c100790 0x7fae1c19b390 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fae1000e9f0 tx=0x7fae1000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae0b7fe700 1 -- 172.21.15.49:0/4278757440 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000c9c0 con 0x7fae1c100790 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae0b7fe700 1 -- 172.21.15.49:0/4278757440 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae10010430 con 0x7fae1c100790 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.019+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c19e730 con 0x7fae1c100790 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.020+0000 7fae0b7fe700 1 -- 172.21.15.49:0/4278757440 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10018680 con 0x7fae1c100790 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.020+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c19ec90 con 0x7fae1c100790 2023-10-08T09:30:37.024 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.021+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c106a90 con 0x7fae1c100790 2023-10-08T09:30:37.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.022+0000 7fae0b7fe700 1 -- 172.21.15.49:0/4278757440 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fae10010860 con 0x7fae1c100790 2023-10-08T09:30:37.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.022+0000 7fae0b7fe700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fae040a1500 0x7fae040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.023+0000 7fae1a59c700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fae040a1500 0x7fae040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.025 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.023+0000 7fae0b7fe700 1 -- 172.21.15.49:0/4278757440 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fae10014070 con 0x7fae1c100790 2023-10-08T09:30:37.026 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.023+0000 7fae1a59c700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fae040a1500 0x7fae040a39c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fae0c004800 tx=0x7fae0c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:37.028 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.027+0000 7fae0b7fe700 1 -- 172.21.15.49:0/4278757440 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae10052100 con 0x7fae1c100790 2023-10-08T09:30:37.231 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.230+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae1c067450 con 0x7fae1c100790 2023-10-08T09:30:37.233 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.231+0000 7fae0b7fe700 1 -- 172.21.15.49:0/4278757440 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7fae10096400 con 0x7fae1c100790 2023-10-08T09:30:37.233 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:37.233 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:30:37.235 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fae040a1500 msgr2=0x7fae040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fae040a1500 0x7fae040a39c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fae0c004800 tx=0x7fae0c01a040 comp rx=0 tx=0).stop 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c100790 msgr2=0x7fae1c19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c100790 0x7fae1c19b390 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fae1000e9f0 tx=0x7fae1000ed00 comp rx=0 tx=0).stop 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 shutdown_connections 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fae040a1500 0x7fae040a39c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae1c101130 0x7fae1c19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae1c0ff5a0 0x7fae1c068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 --2- 172.21.15.49:0/4278757440 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae1c100790 0x7fae1c19b390 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 >> 172.21.15.49:0/4278757440 conn(0x7fae1c0fab10 msgr2=0x7fae1c103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 shutdown_connections 2023-10-08T09:30:37.236 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.234+0000 7fae2082e700 1 -- 172.21.15.49:0/4278757440 wait complete. 2023-10-08T09:30:37.248 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:30:37.248 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:30:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 -- 172.21.15.49:0/1737583787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b8101230 msgr2=0x7ff6b8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1737583787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b8101230 0x7ff6b8107ae0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b0d0 tx=0x7ff6b000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 -- 172.21.15.49:0/1737583787 shutdown_connections 2023-10-08T09:30:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1737583787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b80ff6a0 0x7ff6b80ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1737583787 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b8101230 0x7ff6b8107ae0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.381 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1737583787 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6b8100890 0x7ff6b8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 -- 172.21.15.49:0/1737583787 >> 172.21.15.49:0/1737583787 conn(0x7ff6b80fac10 msgr2=0x7ff6b80fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:37.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 -- 172.21.15.49:0/1737583787 shutdown_connections 2023-10-08T09:30:37.382 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.377+0000 7ff6c079a700 1 -- 172.21.15.49:0/1737583787 wait complete. 2023-10-08T09:30:37.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 Processor -- start 2023-10-08T09:30:37.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 -- start start 2023-10-08T09:30:37.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b80ff6a0 0x7ff6b8068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6b8100890 0x7ff6b819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b8101230 0x7ff6b819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.384 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b819e2d0 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b819e440 con 0x7ff6b8100890 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6c079a700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b819e600 con 0x7ff6b8101230 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b80ff6a0 0x7ff6b8068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b80ff6a0 0x7ff6b8068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54426/0 (socket says 172.21.15.49:54426) 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 -- 172.21.15.49:0/1515673234 learned_addr learned my addr 172.21.15.49:0/1515673234 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6bdd35700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6b8100890 0x7ff6b819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6bed37700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b8101230 0x7ff6b819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 -- 172.21.15.49:0/1515673234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b8101230 msgr2=0x7ff6b819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b8101230 0x7ff6b819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 -- 172.21.15.49:0/1515673234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6b8100890 msgr2=0x7ff6b819b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6b8100890 0x7ff6b819b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.385 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.378+0000 7ff6be536700 1 -- 172.21.15.49:0/1515673234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0009d00 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.379+0000 7ff6be536700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b80ff6a0 0x7ff6b8068350 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e970 tx=0x7ff6b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.379+0000 7ff6af7fe700 1 -- 172.21.15.49:0/1515673234 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4009790 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.379+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b819e8e0 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.379+0000 7ff6af7fe700 1 -- 172.21.15.49:0/1515673234 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6b400c7e0 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.379+0000 7ff6af7fe700 1 -- 172.21.15.49:0/1515673234 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4003c70 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.379+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b819edf0 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.380+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b8106ad0 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.381+0000 7ff6af7fe700 1 -- 172.21.15.49:0/1515673234 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff6b400cc50 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.382+0000 7ff6af7fe700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6a40a1480 0x7ff6a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.382+0000 7ff6af7fe700 1 -- 172.21.15.49:0/1515673234 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff6b4014070 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.382+0000 7ff6bdd35700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6a40a1480 0x7ff6a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.386 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.383+0000 7ff6bdd35700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6a40a1480 0x7ff6a40a3940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8068910 tx=0x7ff6a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:37.387 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.385+0000 7ff6af7fe700 1 -- 172.21.15.49:0/1515673234 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff6b4096660 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.588+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6b8067500 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.590 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.588+0000 7ff6af7fe700 1 -- 172.21.15.49:0/1515673234 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6b4099c80 con 0x7ff6b80ff6a0 2023-10-08T09:30:37.590 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:30:37.592 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6a40a1480 msgr2=0x7ff6a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6a40a1480 0x7ff6a40a3940 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8068910 tx=0x7ff6a8009260 comp rx=0 tx=0).stop 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b80ff6a0 msgr2=0x7ff6b8068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b80ff6a0 0x7ff6b8068350 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e970 tx=0x7ff6b400ec80 comp rx=0 tx=0).stop 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 shutdown_connections 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7ff6a40a1480 0x7ff6a40a3940 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6b8101230 0x7ff6b819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff6b80ff6a0 0x7ff6b8068350 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 --2- 172.21.15.49:0/1515673234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff6b8100890 0x7ff6b819b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.593 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 >> 172.21.15.49:0/1515673234 conn(0x7ff6b80fac10 msgr2=0x7ff6b8103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:37.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 shutdown_connections 2023-10-08T09:30:37.594 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.591+0000 7ff6c079a700 1 -- 172.21.15.49:0/1515673234 wait complete. 2023-10-08T09:30:37.605 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 -- 172.21.15.49:0/513141936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 msgr2=0x7f3a80107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 --2- 172.21.15.49:0/513141936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a80107ae0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b0d0 tx=0x7f3a7800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 -- 172.21.15.49:0/513141936 shutdown_connections 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 --2- 172.21.15.49:0/513141936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a800ff6a0 0x7f3a800ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 --2- 172.21.15.49:0/513141936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a80107ae0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 --2- 172.21.15.49:0/513141936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a80100890 0x7f3a80100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 -- 172.21.15.49:0/513141936 >> 172.21.15.49:0/513141936 conn(0x7f3a800fac10 msgr2=0x7f3a800fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 -- 172.21.15.49:0/513141936 shutdown_connections 2023-10-08T09:30:37.736 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.732+0000 7f3a866fb700 1 -- 172.21.15.49:0/513141936 wait complete. 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 Processor -- start 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 -- start start 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a800ff6a0 0x7f3a80196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a80100890 0x7f3a80196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a8019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8019e3a0 con 0x7f3a80101230 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8019e4e0 con 0x7f3a800ff6a0 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a866fb700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8019e670 con 0x7f3a80100890 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a84c98700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a8019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a7f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a80100890 0x7f3a80196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a7ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a800ff6a0 0x7f3a80196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.737 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a84c98700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a8019ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54444/0 (socket says 172.21.15.49:54444) 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a7f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a80100890 0x7f3a80196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41476/0 (socket says 172.21.15.49:41476) 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.733+0000 7f3a84c98700 1 -- 172.21.15.49:0/1821381839 learned_addr learned my addr 172.21.15.49:0/1821381839 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a84c98700 1 -- 172.21.15.49:0/1821381839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a80100890 msgr2=0x7f3a80196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a84c98700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a80100890 0x7f3a80196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a84c98700 1 -- 172.21.15.49:0/1821381839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a800ff6a0 msgr2=0x7f3a80196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a84c98700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a800ff6a0 0x7f3a80196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a84c98700 1 -- 172.21.15.49:0/1821381839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a78009d00 con 0x7f3a80101230 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a84c98700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a8019ddd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3a78007b60 tx=0x7f3a78008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a7d7fa700 1 -- 172.21.15.49:0/1821381839 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800e040 con 0x7f3a80101230 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a7d7fa700 1 -- 172.21.15.49:0/1821381839 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a7801d930 con 0x7f3a80101230 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8019e8f0 con 0x7f3a80101230 2023-10-08T09:30:37.738 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a7d7fa700 1 -- 172.21.15.49:0/1821381839 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7801b9f0 con 0x7f3a80101230 2023-10-08T09:30:37.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.734+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8019edc0 con 0x7f3a80101230 2023-10-08T09:30:37.739 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.735+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a80067530 con 0x7f3a80101230 2023-10-08T09:30:37.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.736+0000 7f3a7d7fa700 1 -- 172.21.15.49:0/1821381839 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3a7801daa0 con 0x7f3a80101230 2023-10-08T09:30:37.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.737+0000 7f3a7d7fa700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3a680a1500 0x7f3a680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:37.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.737+0000 7f3a7ffff700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3a680a1500 0x7f3a680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:37.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.737+0000 7f3a7d7fa700 1 -- 172.21.15.49:0/1821381839 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3a780cee00 con 0x7f3a80101230 2023-10-08T09:30:37.740 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.737+0000 7f3a7ffff700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3a680a1500 0x7f3a680a39c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3a801006f0 tx=0x7f3a70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:37.742 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.740+0000 7f3a7d7fa700 1 -- 172.21.15.49:0/1821381839 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3a78098160 con 0x7f3a80101230 2023-10-08T09:30:37.944 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.943+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a8019f0e0 con 0x7f3a80101230 2023-10-08T09:30:37.945 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.943+0000 7f3a7d7fa700 1 -- 172.21.15.49:0/1821381839 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a7809b780 con 0x7f3a80101230 2023-10-08T09:30:37.945 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3a680a1500 msgr2=0x7f3a680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3a680a1500 0x7f3a680a39c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3a801006f0 tx=0x7f3a70009260 comp rx=0 tx=0).stop 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 msgr2=0x7f3a8019ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a8019ddd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3a78007b60 tx=0x7f3a78008e40 comp rx=0 tx=0).stop 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 shutdown_connections 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f3a680a1500 0x7f3a680a39c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a80101230 0x7f3a8019ddd0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3a800ff6a0 0x7f3a80196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 --2- 172.21.15.49:0/1821381839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a80100890 0x7f3a80196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 >> 172.21.15.49:0/1821381839 conn(0x7f3a800fac10 msgr2=0x7f3a800fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:37.948 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 shutdown_connections 2023-10-08T09:30:37.949 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:37.946+0000 7f3a866fb700 1 -- 172.21.15.49:0/1821381839 wait complete. 2023-10-08T09:30:37.960 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:30:38.092 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 -- 172.21.15.49:0/2167470958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c0ff6a0 msgr2=0x7f500c0ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 --2- 172.21.15.49:0/2167470958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c0ff6a0 0x7f500c0ffac0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0099c0 tx=0x7f4ffc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 -- 172.21.15.49:0/2167470958 shutdown_connections 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 --2- 172.21.15.49:0/2167470958 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c101230 0x7f500c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 --2- 172.21.15.49:0/2167470958 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c0ff6a0 0x7f500c0ffac0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 --2- 172.21.15.49:0/2167470958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500c100890 0x7f500c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 -- 172.21.15.49:0/2167470958 >> 172.21.15.49:0/2167470958 conn(0x7f500c0fac10 msgr2=0x7f500c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.088+0000 7f5010dff700 1 -- 172.21.15.49:0/2167470958 shutdown_connections 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 -- 172.21.15.49:0/2167470958 wait complete. 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 Processor -- start 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 -- start start 2023-10-08T09:30:38.093 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c0ff6a0 0x7f500c068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c100890 0x7f500c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500c101230 0x7f500c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c19e220 con 0x7f500c100890 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c19e390 con 0x7f500c101230 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5010dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c19e550 con 0x7f500c0ff6a0 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5009d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c100890 0x7f500c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f500a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c0ff6a0 0x7f500c068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f500a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c0ff6a0 0x7f500c068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41488/0 (socket says 172.21.15.49:41488) 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f5009d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c100890 0x7f500c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54456/0 (socket says 172.21.15.49:54456) 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.089+0000 7f500a59c700 1 -- 172.21.15.49:0/550418937 learned_addr learned my addr 172.21.15.49:0/550418937 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f500ad9d700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500c101230 0x7f500c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.094 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f500a59c700 1 -- 172.21.15.49:0/550418937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500c101230 msgr2=0x7f500c19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f500a59c700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500c101230 0x7f500c19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f500a59c700 1 -- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c100890 msgr2=0x7f500c19b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f500a59c700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c100890 0x7f500c19b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f500a59c700 1 -- 172.21.15.49:0/550418937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0095f0 con 0x7f500c0ff6a0 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f5009d9b700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c100890 0x7f500c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f500a59c700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c0ff6a0 0x7f500c068350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc003730 tx=0x7f4ffc00f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f4ffb7fe700 1 -- 172.21.15.49:0/550418937 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc01d070 con 0x7f500c0ff6a0 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f4ffb7fe700 1 -- 172.21.15.49:0/550418937 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ffc0059a0 con 0x7f500c0ff6a0 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c19e7d0 con 0x7f500c0ff6a0 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f4ffb7fe700 1 -- 172.21.15.49:0/550418937 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc004990 con 0x7f500c0ff6a0 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.090+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c19ec80 con 0x7f500c0ff6a0 2023-10-08T09:30:38.095 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.091+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c106b40 con 0x7f500c0ff6a0 2023-10-08T09:30:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.092+0000 7f4ffb7fe700 1 -- 172.21.15.49:0/550418937 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4ffc005210 con 0x7f500c0ff6a0 2023-10-08T09:30:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.093+0000 7f4ffb7fe700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4ff40a1480 0x7f4ff40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.093+0000 7f4ffb7fe700 1 -- 172.21.15.49:0/550418937 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4ffc0cdfc0 con 0x7f500c0ff6a0 2023-10-08T09:30:38.096 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.093+0000 7f5009d9b700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4ff40a1480 0x7f4ff40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.097 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.093+0000 7f5009d9b700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4ff40a1480 0x7f4ff40a3940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f500c068910 tx=0x7f5000009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:38.098 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.097+0000 7f4ffb7fe700 1 -- 172.21.15.49:0/550418937 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4ffc097380 con 0x7f500c0ff6a0 2023-10-08T09:30:38.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:38 smithi132 ceph-mon[103389]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 735 op/s 2023-10-08T09:30:38.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/4278757440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:30:38.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1515673234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:30:38.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1821381839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:30:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.301+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f500c10aec0 con 0x7f500c0ff6a0 2023-10-08T09:30:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.302+0000 7f4ffb7fe700 1 -- 172.21.15.49:0/550418937 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ffc09a9a0 con 0x7f500c0ff6a0 2023-10-08T09:30:38.303 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:30:38.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4ff40a1480 msgr2=0x7f4ff40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4ff40a1480 0x7f4ff40a3940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f500c068910 tx=0x7f5000009260 comp rx=0 tx=0).stop 2023-10-08T09:30:38.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c0ff6a0 msgr2=0x7f500c068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c0ff6a0 0x7f500c068350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc003730 tx=0x7f4ffc00f670 comp rx=0 tx=0).stop 2023-10-08T09:30:38.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 shutdown_connections 2023-10-08T09:30:38.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f4ff40a1480 0x7f4ff40a3940 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.306 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f500c0ff6a0 0x7f500c068350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f500c101230 0x7f500c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 --2- 172.21.15.49:0/550418937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f500c100890 0x7f500c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 >> 172.21.15.49:0/550418937 conn(0x7f500c0fac10 msgr2=0x7f500c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:38.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 shutdown_connections 2023-10-08T09:30:38.307 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.304+0000 7f5010dff700 1 -- 172.21.15.49:0/550418937 wait complete. 2023-10-08T09:30:38.317 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:38 smithi049 ceph-mon[99485]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 735 op/s 2023-10-08T09:30:38.317 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/4278757440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:30:38.317 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1515673234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:30:38.318 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1821381839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:30:38.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:30:38.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 -- 172.21.15.49:0/626704139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c0ff6b0 msgr2=0x7fdc3c0ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 --2- 172.21.15.49:0/626704139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c0ff6b0 0x7fdc3c0ffad0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc300099c0 tx=0x7fdc30009cd0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 -- 172.21.15.49:0/626704139 shutdown_connections 2023-10-08T09:30:38.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 --2- 172.21.15.49:0/626704139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3c101240 0x7fdc3c107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 --2- 172.21.15.49:0/626704139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c0ff6b0 0x7fdc3c0ffad0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 --2- 172.21.15.49:0/626704139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc3c1008a0 0x7fdc3c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.448 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 -- 172.21.15.49:0/626704139 >> 172.21.15.49:0/626704139 conn(0x7fdc3c0fac40 msgr2=0x7fdc3c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:38.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 -- 172.21.15.49:0/626704139 shutdown_connections 2023-10-08T09:30:38.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 -- 172.21.15.49:0/626704139 wait complete. 2023-10-08T09:30:38.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.446+0000 7fdc416f0700 1 Processor -- start 2023-10-08T09:30:38.449 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc416f0700 1 -- start start 2023-10-08T09:30:38.450 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc416f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc3c0ff6b0 0x7fdc3c195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc416f0700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c1008a0 0x7fdc3c195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc416f0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3c101240 0x7fdc3c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc416f0700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c19d170 con 0x7fdc3c1008a0 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc416f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c19d2b0 con 0x7fdc3c0ff6b0 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc416f0700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c19d440 con 0x7fdc3c101240 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c1008a0 0x7fdc3c195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c1008a0 0x7fdc3c195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54470/0 (socket says 172.21.15.49:54470) 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 -- 172.21.15.49:0/1247298524 learned_addr learned my addr 172.21.15.49:0/1247298524 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:38.451 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3affd700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc3c0ff6b0 0x7fdc3c195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3b7fe700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3c101240 0x7fdc3c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 -- 172.21.15.49:0/1247298524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3c101240 msgr2=0x7fdc3c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3c101240 0x7fdc3c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 -- 172.21.15.49:0/1247298524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc3c0ff6b0 msgr2=0x7fdc3c195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc3c0ff6b0 0x7fdc3c195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 -- 172.21.15.49:0/1247298524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc300095f0 con 0x7fdc3c1008a0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.447+0000 7fdc3a7fc700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c1008a0 0x7fdc3c195b80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e920 tx=0x7fdc2c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.448+0000 7fdc23fff700 1 -- 172.21.15.49:0/1247298524 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c009790 con 0x7fdc3c1008a0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.448+0000 7fdc23fff700 1 -- 172.21.15.49:0/1247298524 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc2c00dcb0 con 0x7fdc3c1008a0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.448+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c068d70 con 0x7fdc3c1008a0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.448+0000 7fdc23fff700 1 -- 172.21.15.49:0/1247298524 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c016e70 con 0x7fdc3c1008a0 2023-10-08T09:30:38.452 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.448+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c069280 con 0x7fdc3c1008a0 2023-10-08T09:30:38.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.448+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c067980 con 0x7fdc3c1008a0 2023-10-08T09:30:38.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.450+0000 7fdc23fff700 1 -- 172.21.15.49:0/1247298524 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdc2c016510 con 0x7fdc3c1008a0 2023-10-08T09:30:38.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.450+0000 7fdc23fff700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdc240a1480 0x7fdc240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.453 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.451+0000 7fdc23fff700 1 -- 172.21.15.49:0/1247298524 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdc2c0ccc10 con 0x7fdc3c1008a0 2023-10-08T09:30:38.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.451+0000 7fdc3affd700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdc240a1480 0x7fdc240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.454 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.451+0000 7fdc3affd700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdc240a1480 0x7fdc240a3940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009990 tx=0x7fdc30000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:38.455 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.454+0000 7fdc23fff700 1 -- 172.21.15.49:0/1247298524 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdc2c095f40 con 0x7fdc3c1008a0 2023-10-08T09:30:38.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:38 smithi184 ceph-mon[103675]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 735 op/s 2023-10-08T09:30:38.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/4278757440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:30:38.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1515673234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:30:38.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1821381839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:30:38.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.658+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdc3c1065b0 con 0x7fdc3c1008a0 2023-10-08T09:30:38.660 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.659+0000 7fdc23fff700 1 -- 172.21.15.49:0/1247298524 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdc2c025020 con 0x7fdc3c1008a0 2023-10-08T09:30:38.660 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdc240a1480 msgr2=0x7fdc240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdc240a1480 0x7fdc240a3940 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009990 tx=0x7fdc30000bc0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c1008a0 msgr2=0x7fdc3c195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c1008a0 0x7fdc3c195b80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e920 tx=0x7fdc2c00ec30 comp rx=0 tx=0).stop 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 shutdown_connections 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdc240a1480 0x7fdc240a3940 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc3c101240 0x7fdc3c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc3c0ff6b0 0x7fdc3c195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.663 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 --2- 172.21.15.49:0/1247298524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdc3c1008a0 0x7fdc3c195b80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 >> 172.21.15.49:0/1247298524 conn(0x7fdc3c0fac40 msgr2=0x7fdc3c103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:38.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 shutdown_connections 2023-10-08T09:30:38.664 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.661+0000 7fdc416f0700 1 -- 172.21.15.49:0/1247298524 wait complete. 2023-10-08T09:30:38.675 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-08T09:30:38.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 -- 172.21.15.49:0/645710114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 msgr2=0x7f73cc107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 --2- 172.21.15.49:0/645710114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc107a10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f73c400b0d0 tx=0x7f73c400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 -- 172.21.15.49:0/645710114 shutdown_connections 2023-10-08T09:30:38.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 --2- 172.21.15.49:0/645710114 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc107a10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.805 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 --2- 172.21.15.49:0/645710114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc0ff5d0 0x7f73cc0ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 --2- 172.21.15.49:0/645710114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73cc1007c0 0x7f73cc100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 -- 172.21.15.49:0/645710114 >> 172.21.15.49:0/645710114 conn(0x7f73cc0fab40 msgr2=0x7f73cc0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 -- 172.21.15.49:0/645710114 shutdown_connections 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 -- 172.21.15.49:0/645710114 wait complete. 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 Processor -- start 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 -- start start 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc0ff5d0 0x7f73cc069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73cc1007c0 0x7f73cc19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.800+0000 7f73d089d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73d089d700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19e1a0 con 0x7f73cc101160 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73d089d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19e310 con 0x7f73cc0ff5d0 2023-10-08T09:30:38.806 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73d089d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19e4d0 con 0x7f73cc1007c0 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73ca59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc0ff5d0 0x7f73cc069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73c9d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73cc1007c0 0x7f73cc19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc19da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54486/0 (socket says 172.21.15.49:54486) 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 -- 172.21.15.49:0/2624978766 learned_addr learned my addr 172.21.15.49:0/2624978766 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 -- 172.21.15.49:0/2624978766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73cc1007c0 msgr2=0x7f73cc19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73cc1007c0 0x7f73cc19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 -- 172.21.15.49:0/2624978766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc0ff5d0 msgr2=0x7f73cc069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc0ff5d0 0x7f73cc069730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 -- 172.21.15.49:0/2624978766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4009d00 con 0x7f73cc101160 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73cad9d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc19da00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f73c4007b60 tx=0x7f73c4008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:38.807 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73bb7fe700 1 -- 172.21.15.49:0/2624978766 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400e040 con 0x7f73cc101160 2023-10-08T09:30:38.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73bb7fe700 1 -- 172.21.15.49:0/2624978766 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c401d930 con 0x7f73cc101160 2023-10-08T09:30:38.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc19e750 con 0x7f73cc101160 2023-10-08T09:30:38.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73bb7fe700 1 -- 172.21.15.49:0/2624978766 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c401ba00 con 0x7f73cc101160 2023-10-08T09:30:38.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.801+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc19ebd0 con 0x7f73cc101160 2023-10-08T09:30:38.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.802+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc106980 con 0x7f73cc101160 2023-10-08T09:30:38.808 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.804+0000 7f73bb7fe700 1 -- 172.21.15.49:0/2624978766 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f73c401daa0 con 0x7f73cc101160 2023-10-08T09:30:38.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.805+0000 7f73bb7fe700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73b40a1480 0x7f73b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:30:38.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.805+0000 7f73bb7fe700 1 -- 172.21.15.49:0/2624978766 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f73c40ce870 con 0x7f73cc101160 2023-10-08T09:30:38.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.805+0000 7f73ca59c700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73b40a1480 0x7f73b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:30:38.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.805+0000 7f73ca59c700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73b40a1480 0x7f73b40a3940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73cc100620 tx=0x7f73c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:30:38.809 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:38.808+0000 7f73bb7fe700 1 -- 172.21.15.49:0/2624978766 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f73c4097c30 con 0x7f73cc101160 2023-10-08T09:30:39.072 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.070+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f73cc0678a0 con 0x7f73cc101160 2023-10-08T09:30:39.080 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.079+0000 7f73bb7fe700 1 -- 172.21.15.49:0/2624978766 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733876 (secure 0 0 0) 0x7f73c409b250 con 0x7f73cc101160 2023-10-08T09:30:39.089 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:30:39.089 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 18, 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "active_gid": 14152, 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "active_name": "x", 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addrs": { 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1314466902 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1314466902 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "active_addr": "172.21.15.49:6801/1314466902", 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "active_change": "2023-10-08T09:06:35.771694+0000", 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "active_mgr_features": 4540138322906710015, 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "available": true, 2023-10-08T09:30:39.091 INFO:teuthology.orchestra.run.smithi049.stdout: "standbys": [ 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 24106, 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "z", 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540138322906710015, 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:39.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.094 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.095 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.097 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.099 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.100 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2023-10-08T09:30:39.101 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.102 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.103 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2023-10-08T09:30:39.104 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.105 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.106 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.107 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.108 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2023-10-08T09:30:39.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.112 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T09:30:39.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.115 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.119 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.120 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.121 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.122 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.123 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.124 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.125 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2023-10-08T09:30:39.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.127 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2023-10-08T09:30:39.128 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap" 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.130 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:39.132 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.133 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.134 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.136 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2023-10-08T09:30:39.137 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.138 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.139 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.140 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.141 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.142 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2023-10-08T09:30:39.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2023-10-08T09:30:39.144 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.146 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.147 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T09:30:39.148 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.149 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.150 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2023-10-08T09:30:39.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.153 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.155 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.158 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.160 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.161 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.163 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.168 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.169 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.170 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2023-10-08T09:30:39.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.173 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2023-10-08T09:30:39.174 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.175 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.176 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.177 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2023-10-08T09:30:39.179 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.181 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2023-10-08T09:30:39.182 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T09:30:39.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.185 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.186 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.187 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.188 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.189 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.190 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2023-10-08T09:30:39.191 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.192 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.194 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2023-10-08T09:30:39.195 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.197 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2023-10-08T09:30:39.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T09:30:39.201 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.202 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2023-10-08T09:30:39.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T09:30:39.205 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.207 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2023-10-08T09:30:39.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.209 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.210 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.211 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.213 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.214 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.215 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.216 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.217 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.218 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.219 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2023-10-08T09:30:39.220 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.221 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.222 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.224 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.225 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T09:30:39.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.228 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.229 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.230 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.231 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.233 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.234 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.235 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.236 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.237 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T09:30:39.238 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.240 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.241 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T09:30:39.242 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.244 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.245 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.246 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.247 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.248 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.249 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.250 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.251 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.252 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.254 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.255 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.256 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T09:30:39.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.259 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.260 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.261 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.263 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.264 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T09:30:39.265 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.267 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T09:30:39.268 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.269 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.270 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.270 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T09:30:39.270 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T09:30:39.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.270 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.272 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T09:30:39.273 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.274 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.275 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T09:30:39.276 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.277 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.280 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.281 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.282 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.283 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2023-10-08T09:30:39.284 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.285 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.286 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.288 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.289 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.290 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.291 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.292 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.294 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.295 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.296 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.297 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.298 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T09:30:39.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.300 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.301 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.302 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.303 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.304 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.306 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2023-10-08T09:30:39.308 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.310 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.312 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.313 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.314 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.316 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.317 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2023-10-08T09:30:39.318 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.319 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2023-10-08T09:30:39.320 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2023-10-08T09:30:39.322 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.323 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.324 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.326 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2023-10-08T09:30:39.327 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.329 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.330 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.331 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.332 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.333 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2023-10-08T09:30:39.334 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.335 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.336 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.337 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.339 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.340 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.341 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.342 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T09:30:39.343 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.344 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.345 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.346 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.347 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.348 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2023-10-08T09:30:39.351 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.352 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.353 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2023-10-08T09:30:39.354 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.355 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.356 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2023-10-08T09:30:39.357 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2023-10-08T09:30:39.358 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.359 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.360 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.361 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.362 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.363 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.364 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.365 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.366 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.368 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.369 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.370 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.371 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.372 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T09:30:39.373 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.374 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.375 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.376 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.377 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.378 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.379 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.381 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.382 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T09:30:39.383 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.384 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.386 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.388 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.389 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.390 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2023-10-08T09:30:39.391 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2023-10-08T09:30:39.392 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2023-10-08T09:30:39.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2023-10-08T09:30:39.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.395 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T09:30:39.396 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.397 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.398 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.399 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.400 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.402 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.403 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.406 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.407 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.408 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.409 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.410 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.411 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.413 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.414 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.415 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.416 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.417 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.419 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.420 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.422 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.424 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.425 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.428 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.429 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.430 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.432 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.435 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.436 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.439 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.441 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.441 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.441 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.441 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.441 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.445 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.447 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.448 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.450 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2023-10-08T09:30:39.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2023-10-08T09:30:39.454 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.455 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.456 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.458 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.459 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.463 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.464 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2023-10-08T09:30:39.466 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.467 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2023-10-08T09:30:39.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T09:30:39.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.470 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.471 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.472 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.474 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.477 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2023-10-08T09:30:39.479 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.480 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.481 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2023-10-08T09:30:39.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.484 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.485 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2023-10-08T09:30:39.486 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.487 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.488 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.490 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.491 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.494 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T09:30:39.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.497 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2023-10-08T09:30:39.498 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.499 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.500 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.501 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.504 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.505 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.505 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.505 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.505 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.505 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.505 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.505 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.506 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.508 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.509 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.510 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.511 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.512 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.513 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.514 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.515 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.516 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.517 INFO:teuthology.orchestra.run.smithi049.stdout: "drive_group_interval": { 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "drive_group_interval", 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300.0", 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T09:30:39.518 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.519 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.520 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.521 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.522 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.523 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.524 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.525 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.526 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.527 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.529 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.530 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.532 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.533 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.534 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2023-10-08T09:30:39.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.537 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.538 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2023-10-08T09:30:39.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.541 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.542 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2023-10-08T09:30:39.546 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2023-10-08T09:30:39.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.551 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.553 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.554 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.555 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.556 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.557 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.558 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.559 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.560 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.563 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.565 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.567 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.568 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.569 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.571 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.572 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.574 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.575 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.576 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2023-10-08T09:30:39.577 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.578 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.579 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.582 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.584 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2023-10-08T09:30:39.585 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.586 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2023-10-08T09:30:39.587 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.588 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.589 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2023-10-08T09:30:39.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.593 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.598 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.599 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.600 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.601 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.602 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.603 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.606 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.607 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.608 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.610 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.612 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.613 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.614 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.617 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.618 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.619 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.620 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.621 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.622 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.624 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.625 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.626 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.629 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.630 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.631 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.632 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.633 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2023-10-08T09:30:39.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.636 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.638 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.639 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.640 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.641 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.642 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.643 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.645 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.646 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.647 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.648 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.649 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2023-10-08T09:30:39.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: "gid": 24107, 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "y", 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_features": 4540138322906710015, 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2023-10-08T09:30:39.653 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.654 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.655 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.656 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.658 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.659 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.660 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.661 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.662 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.664 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2023-10-08T09:30:39.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.666 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2023-10-08T09:30:39.667 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.668 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.669 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T09:30:39.670 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.671 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2023-10-08T09:30:39.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.674 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.675 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2023-10-08T09:30:39.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.678 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T09:30:39.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2023-10-08T09:30:39.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2023-10-08T09:30:39.681 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2023-10-08T09:30:39.682 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.683 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.684 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.687 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.689 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T09:30:39.690 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap" 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.692 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.694 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2023-10-08T09:30:39.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.698 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2023-10-08T09:30:39.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.701 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.702 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2023-10-08T09:30:39.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.705 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.706 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.707 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T09:30:39.708 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.709 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.709 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.709 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2023-10-08T09:30:39.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2023-10-08T09:30:39.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.709 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T09:30:39.711 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T09:30:39.712 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2023-10-08T09:30:39.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2023-10-08T09:30:39.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.718 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.718 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2023-10-08T09:30:39.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2023-10-08T09:30:39.719 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T09:30:39.720 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2023-10-08T09:30:39.721 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.722 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T09:30:39.723 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.725 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.726 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.727 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T09:30:39.728 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2023-10-08T09:30:39.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.731 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T09:30:39.732 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2023-10-08T09:30:39.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T09:30:39.736 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.737 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.738 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.739 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.740 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2023-10-08T09:30:39.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.742 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.744 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.745 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.747 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.751 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.752 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.753 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.754 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.755 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2023-10-08T09:30:39.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.758 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.759 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.760 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.761 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T09:30:39.762 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2023-10-08T09:30:39.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2023-10-08T09:30:39.765 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.767 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.768 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.769 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.770 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2023-10-08T09:30:39.771 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T09:30:39.773 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.774 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.775 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.778 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.779 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.780 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.781 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.782 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.783 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.785 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.786 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.787 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.788 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T09:30:39.789 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.790 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.791 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.792 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T09:30:39.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.794 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.797 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.798 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.799 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.801 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.802 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.803 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T09:30:39.804 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.805 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.806 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.807 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:39.808 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T09:30:39.810 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.813 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.814 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.815 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T09:30:39.816 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.817 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.818 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.819 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.820 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T09:30:39.822 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.824 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.825 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T09:30:39.826 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.827 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.829 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.830 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.831 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T09:30:39.832 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.833 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.834 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.835 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T09:30:39.837 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.838 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.839 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.840 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.841 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2023-10-08T09:30:39.842 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2023-10-08T09:30:39.844 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.845 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.846 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.847 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.848 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.849 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.849 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.849 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.849 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.849 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.849 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.849 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.851 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.852 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.854 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.856 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:39.858 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.860 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.861 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2023-10-08T09:30:39.862 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2023-10-08T09:30:39.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.864 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.865 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2023-10-08T09:30:39.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.867 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.868 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.869 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.870 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.871 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.872 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.874 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.876 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2023-10-08T09:30:39.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.879 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.880 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.881 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T09:30:39.882 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2023-10-08T09:30:39.883 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.886 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.888 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.889 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.890 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.891 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.893 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.894 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.895 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.896 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.898 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.900 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.901 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.902 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2023-10-08T09:30:39.903 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.905 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.906 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.907 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.908 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.909 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.910 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.911 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.912 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.913 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.915 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2023-10-08T09:30:39.916 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.917 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.918 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.919 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.920 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.921 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.922 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.924 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.925 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.926 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.927 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2023-10-08T09:30:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.937 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2023-10-08T09:30:39.938 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.939 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:39.940 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.941 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.942 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.944 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.945 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.946 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.947 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.948 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2023-10-08T09:30:39.949 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T09:30:39.950 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.951 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.952 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.953 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.954 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.955 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.956 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.958 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.959 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.960 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.961 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.962 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.963 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.964 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.965 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.967 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.968 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.969 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.970 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.971 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.972 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.973 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.974 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.975 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.976 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.977 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.978 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.979 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.980 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.982 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.983 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.984 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.985 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.987 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.990 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.991 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.992 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:39.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.994 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.995 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:39.996 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.997 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:39.998 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2023-10-08T09:30:39.999 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.001 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.002 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.003 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.004 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.005 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.006 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.007 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2023-10-08T09:30:40.008 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.009 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.010 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.011 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.012 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.013 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.014 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.015 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.016 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.018 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.019 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.020 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.021 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.022 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.023 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2023-10-08T09:30:40.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T09:30:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.026 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.027 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.028 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.029 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.030 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.031 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.033 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.034 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.035 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.036 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.037 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.038 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.039 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.040 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.041 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.042 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.043 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.044 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.045 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.046 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.047 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.049 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2023-10-08T09:30:40.050 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.051 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.051 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.051 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.051 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.051 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.053 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.053 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2023-10-08T09:30:40.053 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2023-10-08T09:30:40.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.053 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.054 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.055 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T09:30:40.056 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.057 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2023-10-08T09:30:40.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2023-10-08T09:30:40.060 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.061 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.062 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.063 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.064 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.065 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.066 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.067 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.068 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.068 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.068 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.068 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.068 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.068 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.069 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T09:30:40.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.072 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.073 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.074 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.075 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.076 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.078 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.079 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.082 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.083 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.084 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.085 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2023-10-08T09:30:40.086 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.087 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.088 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.089 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.091 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.092 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2023-10-08T09:30:40.093 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "drive_group_interval": { 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "drive_group_interval", 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300.0", 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.094 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.095 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.096 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.097 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.098 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.101 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.102 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.103 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.105 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.106 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.107 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.108 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.109 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.110 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.111 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.112 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2023-10-08T09:30:40.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.114 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.115 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.116 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.117 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.119 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.120 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2023-10-08T09:30:40.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.123 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2023-10-08T09:30:40.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2023-10-08T09:30:40.126 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.127 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.129 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.131 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.132 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.134 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.135 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.136 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.137 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.138 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.139 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.140 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.141 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.142 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.143 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.144 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.146 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.147 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2023-10-08T09:30:40.148 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.149 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2023-10-08T09:30:40.150 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.151 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.152 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.153 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.155 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.156 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.157 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.158 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T09:30:40.159 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.160 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.161 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.162 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.163 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.164 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.165 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.166 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.168 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.169 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2023-10-08T09:30:40.170 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.171 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.172 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2023-10-08T09:30:40.173 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.175 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.177 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.178 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.179 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.180 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.181 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.182 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.183 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.184 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.185 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.186 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.187 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.188 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.190 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.191 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.192 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.193 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.194 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.195 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.197 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T09:30:40.198 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.199 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:40.200 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2023-10-08T09:30:40.201 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.202 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.203 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.205 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.206 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.207 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.208 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.209 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.210 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2023-10-08T09:30:40.211 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.212 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.213 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.214 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:40.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "modules": [ 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard", 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "iostat", 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "nfs", 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "restful" 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "available_modules": [ 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "alerts", 2023-10-08T09:30:40.216 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.217 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How frequently to reexamine health status", 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.218 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.219 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.220 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.221 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.223 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_destination": { 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_destination", 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email address to send alerts to", 2023-10-08T09:30:40.224 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_from_name": { 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_from_name", 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "Ceph", 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.225 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Email From: name", 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_host": { 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_host", 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.226 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP server", 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_password": { 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_password", 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.227 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Password to authenticate with", 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_port": { 2023-10-08T09:30:40.228 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_port", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "465", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP port", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.229 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_sender": { 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_sender", 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.230 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SMTP envelope sender", 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_ssl": { 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_ssl", 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.231 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "smtp_user": { 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "smtp_user", 2023-10-08T09:30:40.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "User to authenticate as", 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.233 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "balancer", 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: "active": { 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "active", 2023-10-08T09:30:40.234 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically balance PGs across cluster", 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.235 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_time": { 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_time", 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0000", 2023-10-08T09:30:40.236 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "beginning time of day to automatically balance", 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "begin_weekday": { 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "begin_weekday", 2023-10-08T09:30:40.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T09:30:40.238 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_max_iterations": { 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_max_iterations", 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "25", 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:40.239 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "250", 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_metrics": { 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_metrics", 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.240 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "pgs,objects,bytes", 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.241 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_compat_step": { 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crush_compat_step", 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.5", 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0.001", 2023-10-08T09:30:40.242 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "0.999", 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "aggressiveness of optimization", 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "end_time": { 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_time", 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2359", 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ending time of day to automatically balance", 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.244 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "end_weekday": { 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "end_weekday", 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "0", 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "6", 2023-10-08T09:30:40.245 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.246 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.247 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.248 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.249 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.250 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.251 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.252 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.253 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "min_score": { 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_score", 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.254 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.255 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "upmap", 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-compat", 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "none", 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap" 2023-10-08T09:30:40.256 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Balancer mode", 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_ids": { 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_ids", 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.257 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.258 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_deviation": { 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_deviation", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-08T09:30:40.260 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.261 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.261 INFO:teuthology.orchestra.run.smithi049.stdout: "upmap_max_optimizations": { 2023-10-08T09:30:40.261 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "upmap_max_optimizations", 2023-10-08T09:30:40.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:40.261 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.261 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm", 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_down_multiplier": { 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_down_multiplier", 2023-10-08T09:30:40.263 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-08T09:30:40.264 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_refresh_rate": { 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_refresh_rate", 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.265 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "20", 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "agent_starting_port": { 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "agent_starting_port", 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4721", 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.267 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_ptrace": { 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_ptrace", 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.268 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.269 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_interval": { 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_interval", 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.270 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "autotune_memory_target_ratio": { 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "autotune_memory_target_ratio", 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.271 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0.7", 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.272 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_log_destination": { 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cephadm_log_destination", 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "file", 2023-10-08T09:30:40.273 INFO:teuthology.orchestra.run.smithi049.stdout: "file,syslog", 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "syslog" 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "cgroups_split": { 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cgroups_split", 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.274 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "config_checks_enabled": { 2023-10-08T09:30:40.275 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_checks_enabled", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.276 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "config_dashboard": { 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_dashboard", 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-08T09:30:40.277 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_alertmanager": { 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_alertmanager", 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-08T09:30:40.278 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_base": { 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_base", 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.279 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Container image name, without the tag", 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_elasticsearch": { 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_elasticsearch", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "elasticsearch container image", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.281 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_grafana": { 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_grafana", 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.282 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_haproxy": { 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_haproxy", 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.283 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "HAproxy container image", 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_agent": { 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_agent", 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.284 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger agent container image", 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_collector": { 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_collector", 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.285 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger collector container image", 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.286 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_jaeger_query": { 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_jaeger_query", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Jaeger query container image", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_keepalived": { 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_keepalived", 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.288 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Keepalived container image", 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_loki": { 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_loki", 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.289 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Loki container image", 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_node_exporter": { 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_node_exporter", 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.290 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_nvmeof": { 2023-10-08T09:30:40.291 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_nvmeof", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Nvme-of container image", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_prometheus": { 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_prometheus", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Prometheus container image", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.293 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_promtail": { 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_promtail", 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.294 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Promtail container image", 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image_snmp_gateway": { 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_image_snmp_gateway", 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.295 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "SNMP Gateway container image", 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "container_init": { 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "container_init", 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.296 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Run podman/docker with `--init`", 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "daemon_cache_timeout": { 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "daemon_cache_timeout", 2023-10-08T09:30:40.297 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "default_cephadm_command_timeout": { 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_cephadm_command_timeout", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "900", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_registry": { 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "default_registry", 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "docker.io", 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.300 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "device_cache_timeout": { 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_cache_timeout", 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.301 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1800", 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache device inventory", 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "device_enhanced_scan": { 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_enhanced_scan", 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.302 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "facts_cache_timeout": { 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "facts_cache_timeout", 2023-10-08T09:30:40.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "seconds to cache host facts data", 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.304 INFO:teuthology.orchestra.run.smithi049.stdout: "host_check_interval": { 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "host_check_interval", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to perform a host check", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "inventory_list_all": { 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "inventory_list_all", 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.307 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.308 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "log_refresh_metadata": { 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_refresh_metadata", 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-08T09:30:40.309 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.310 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.311 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.312 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.313 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.314 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.315 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "*", 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.316 INFO:teuthology.orchestra.run.smithi049.stdout: "max_count_per_host": { 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_count_per_host", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of daemons per service per host", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.317 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd_draining_count": { 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_osd_draining_count", 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-08T09:30:40.318 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "migration_current": { 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "migration_current", 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.319 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "internal - do not modify", 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "mode": { 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mode", 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "root", 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.320 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm-package", 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "root" 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "mode for remote execution of cephadm", 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "prometheus_alerts_path": { 2023-10-08T09:30:40.321 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus_alerts_path", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.322 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_insecure": { 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_insecure", 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.323 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_password": { 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_password", 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.324 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_url": { 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_url", 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.325 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "registry_username": { 2023-10-08T09:30:40.326 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "registry_username", 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-08T09:30:40.327 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "secure_monitoring_stack": { 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "secure_monitoring_stack", 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.328 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "service_discovery_port": { 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "service_discovery_port", 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8765", 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.329 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "cephadm service discovery port", 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "ssh_config_file": { 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssh_config_file", 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.330 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "use_agent": { 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_agent", 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.331 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "use_repo_digest": { 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "use_repo_digest", 2023-10-08T09:30:40.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.333 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_failed_host_check": { 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_failed_host_check", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if the host check fails", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.334 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_daemons": { 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_daemons", 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-08T09:30:40.335 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_on_stray_hosts": { 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_on_stray_hosts", 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.336 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crash", 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.337 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.338 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.339 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.341 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.342 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "retain_interval": { 2023-10-08T09:30:40.343 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retain_interval", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "31536000", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_recent_interval": { 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_recent_interval", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1209600", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.345 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dashboard", 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-08T09:30:40.346 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_HOST": { 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_ENABLED": { 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_ENABLED", 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "ENABLE_BROWSABLE_API": { 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-08T09:30:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-08T09:30:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-08T09:30:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_PASSWORD": { 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_URL": { 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_URL", 2023-10-08T09:30:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_API_USERNAME": { 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-08T09:30:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_HOST": { 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-08T09:30:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-08T09:30:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-08T09:30:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_ENABLED": { 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8", 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "45", 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ACCESS_KEY": { 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-08T09:30:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "admin", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SECRET_KEY": { 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "RGW_API_SSL_VERIFY": { 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "UNSAFE_TLS_v1_2": { 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-08T09:30:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "cross_origin_url": { 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cross_origin_url", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "crt_file": { 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "crt_file", 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "debug": { 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "debug", 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Enable/disable debug options", 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "jwt_token_ttl": { 2023-10-08T09:30:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "jwt_token_ttl", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "28800", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "motd": { 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "motd", 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "The message of the day", 2023-10-08T09:30:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect_resolve_ip_addr": { 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "redirect_resolve_ip_addr", 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2023-10-08T09:30:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8080", 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl_server_port": { 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl_server_port", 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8443", 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "redirect", 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "redirect" 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2023-10-08T09:30:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "url_prefix": { 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url_prefix", 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_monitoring": { 2023-10-08T09:30:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_monitoring", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "monitor device health metrics", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "mark_out_threshold": { 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mark_out_threshold", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "2419200", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": { 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pool_name", 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "device_health_metrics", 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "retention_period": { 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "retention_period", 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15552000", 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long to retain device health metrics", 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_frequency": { 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_frequency", 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "self_heal": { 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "self_heal", 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how frequently to wake up and check device health", 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "warn_threshold": { 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "warn_threshold", 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7257600", 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-08T09:30:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "diskprediction_local", 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "predict_interval": { 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predict_interval", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "86400", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "predictor_model": { 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "predictor_model", 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "prophetstor", 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "600", 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "influx", 2023-10-08T09:30:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": false, 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "influxdb python module not found", 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "batch_size": { 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "batch_size", 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5000", 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "database": { 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "database", 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "ceph", 2023-10-08T09:30:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "hostname": { 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "hostname", 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server hostname", 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "30", 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "5", 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "password": { 2023-10-08T09:30:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "password", 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "password of InfluxDB server user", 2023-10-08T09:30:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "port": { 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "port", 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "8086", 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "InfluxDB server port", 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "ssl": { 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ssl", 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "false", 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "threads": { 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threads", 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "32", 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "username": { 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "username", 2023-10-08T09:30:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "username of InfluxDB server user", 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "verify_ssl": { 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "verify_ssl", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "true", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "insights", 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "iostat", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.475 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.477 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.478 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "k8sevents", 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_event_retention_days": { 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "ceph_event_retention_days", 2023-10-08T09:30:40.481 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "7", 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "config_check_secs": { 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "config_check_secs", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "10", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.483 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.484 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.485 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.485 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.485 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.485 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.485 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.485 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.485 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.486 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.487 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.488 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "localpool", 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "failure_domain": { 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "failure_domain", 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "host", 2023-10-08T09:30:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "failure domain for any created local pool", 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": { 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "min_size", 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default min_size for any created local pool", 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "num_rep": { 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "num_rep", 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3", 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default replica count for any created local pool", 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": { 2023-10-08T09:30:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_num", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "128", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "default pg_num for any created local pool", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "prefix": { 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prefix", 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "name prefix for any created local pool", 2023-10-08T09:30:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "subtree": { 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "subtree", 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "rack", 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mds_autoscaler", 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.504 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirroring", 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "nfs", 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "fail_fs": { 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "fail_fs", 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator": { 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "orchestrator", 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "rook", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "test_orchestrator" 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Orchestrator backend", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_perf_query", 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "osd_support", 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "pg_autoscaler", 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "threshold": { 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "threshold", 2023-10-08T09:30:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "3.0", 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1.0", 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "scaling threshold", 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "progress", 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_pg_recovery_event": { 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_pg_recovery_event", 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "max_completed_events": { 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_completed_events", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "50", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "number of past completed events to remember", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "sleep_interval": { 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "sleep_interval", 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "5", 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "how long the module is going to sleep", 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "prometheus", 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "cache": { 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "cache", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "exclude_perf_counters": { 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "exclude_perf_counters", 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.570 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.571 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.572 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.573 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.574 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.575 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools": { 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools", 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.576 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.577 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300", 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "scrape_interval": { 2023-10-08T09:30:40.578 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "scrape_interval", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15.0", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "::", 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.580 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.581 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "9283", 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "stale_cache_strategy": { 2023-10-08T09:30:40.582 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stale_cache_strategy", 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "log", 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.583 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_behaviour": { 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_behaviour", 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "default", 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.584 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: "default", 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: "error" 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "standby_error_status_code": { 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "standby_error_status_code", 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "500", 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "400", 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "599", 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.586 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2023-10-08T09:30:40.587 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.588 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.589 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.590 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.591 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.593 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_snap_create": { 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_snap_create", 2023-10-08T09:30:40.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10", 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.595 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "mirror_snapshot_schedule": { 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mirror_snapshot_schedule", 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.596 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "trash_purge_schedule": { 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "trash_purge_schedule", 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.597 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.598 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.598 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.598 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.598 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.598 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.598 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "restful", 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_auth": { 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enable_auth", 2023-10-08T09:30:40.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.600 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.601 INFO:teuthology.orchestra.run.smithi049.stdout: "key_file": { 2023-10-08T09:30:40.601 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "key_file", 2023-10-08T09:30:40.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.601 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.603 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.604 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.604 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.604 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.604 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.604 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.604 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.605 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.605 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.605 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.605 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.607 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.608 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.609 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "server_addr": { 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_addr", 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.610 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "server_port": { 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "server_port", 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.611 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.612 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rgw", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.613 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.614 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.615 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.616 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.618 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.619 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rook", 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "drive_group_interval": { 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "drive_group_interval", 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "300.0", 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.620 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.621 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.622 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.623 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.624 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.626 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "storage_class": { 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "storage_class", 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "local", 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.627 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "selftest", 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.628 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.629 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.630 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.631 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.633 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.634 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "roption1": { 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption1", 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.635 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.636 INFO:teuthology.orchestra.run.smithi049.stdout: "roption2": { 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "roption2", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "xyz", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption1": { 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption1", 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.638 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption2": { 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption2", 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.639 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption3": { 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption3", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "float", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption4": { 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption4", 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.642 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption5": { 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption5", 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.643 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption6": { 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption6", 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "rwoption7": { 2023-10-08T09:30:40.645 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rwoption7", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "1", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "42", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.646 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "testkey": { 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testkey", 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "testlkey": { 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testlkey", 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.648 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "testnewline": { 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "testnewline", 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.649 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snap_schedule", 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_m_granularity": { 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "allow_m_granularity", 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.651 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "allow minute scheduled snapshots", 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "dump_on_update": { 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "dump_on_update", 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "dump database to debug log on update", 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.654 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.656 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.657 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.658 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "stats", 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.660 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.661 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.662 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.663 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.665 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "status", 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.666 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.667 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.667 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.667 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.667 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.670 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.670 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.670 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.671 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.672 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.673 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.674 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.675 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.676 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.677 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telegraf", 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "address": { 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "address", 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.678 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.679 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "15", 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.680 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.682 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.683 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.684 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.685 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.686 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "telemetry", 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.689 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_basic": { 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_basic", 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-08T09:30:40.690 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_crash": { 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_crash", 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.691 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.692 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_device": { 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_device", 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "True", 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.693 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-08T09:30:40.694 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.694 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.694 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.694 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_ident": { 2023-10-08T09:30:40.694 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_ident", 2023-10-08T09:30:40.694 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-08T09:30:40.695 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "channel_perf": { 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "channel_perf", 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.696 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.697 INFO:teuthology.orchestra.run.smithi049.stdout: "contact": { 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "contact", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.698 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "description": { 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "description", 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.699 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "device_url": { 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "device_url", 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-08T09:30:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "enabled": { 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "enabled", 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.702 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "24", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "8", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.703 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "last_opt_revision": { 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "last_opt_revision", 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "1", 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.704 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard": { 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard", 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.705 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "leaderboard_description": { 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "leaderboard_description", 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.707 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.708 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.709 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.710 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.711 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.712 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.713 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "organization": { 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "organization", 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.714 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "proxy": { 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "proxy", 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "url": { 2023-10-08T09:30:40.716 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "url", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.717 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "test_orchestrator", 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.718 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.719 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.722 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.723 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.725 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.726 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.728 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.729 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.730 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "max_concurrent_clones": { 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "max_concurrent_clones", 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "4", 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.732 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "periodic_async_work": { 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "periodic_async_work", 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.733 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Periodically check for async work", 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "snapshot_clone_delay": { 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "snapshot_clone_delay", 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "0", 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix", 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "can_run": true, 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "error_string": "", 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "module_options": { 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "discovery_interval": { 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "discovery_interval", 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "uint", 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.736 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "100", 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "identifier": { 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "identifier", 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.737 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "interval": { 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "interval", 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "secs", 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "60", 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.739 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "log_level": { 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_level", 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.740 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster": { 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster", 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.742 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_cluster_level": { 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_cluster_level", 2023-10-08T09:30:40.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "info", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [ 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "critical", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "debug", 2023-10-08T09:30:40.744 INFO:teuthology.orchestra.run.smithi049.stdout: "error", 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "info", 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "warning" 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": { 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "log_to_file", 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bool", 2023-10-08T09:30:40.745 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "False", 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.746 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_host": { 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_host", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_port": { 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_port", 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "int", 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "10051", 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.748 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "zabbix_sender": { 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "zabbix_sender", 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "str", 2023-10-08T09:30:40.749 INFO:teuthology.orchestra.run.smithi049.stdout: "level": "advanced", 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 0, 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "min": "", 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "max": "", 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "enum_allowed": [], 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "desc": "", 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "long_desc": "", 2023-10-08T09:30:40.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tags": [], 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "see_also": [] 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "services": { 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "dashboard": "https://172.21.15.49:8443/" 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.751 INFO:teuthology.orchestra.run.smithi049.stdout: "always_on_modules": { 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "octopus": [ 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2023-10-08T09:30:40.752 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pacific": [ 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2023-10-08T09:30:40.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "quincy": [ 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2023-10-08T09:30:40.754 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2023-10-08T09:30:40.755 INFO:teuthology.orchestra.run.smithi049.stdout: "reef": [ 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "balancer", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "crash", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "devicehealth", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "orchestrator", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscaler", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "progress", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_support", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "status", 2023-10-08T09:30:40.756 INFO:teuthology.orchestra.run.smithi049.stdout: "telemetry", 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: "volumes" 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: "last_failure_osd_epoch": 3, 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: "active_clients": [ 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "devicehealth", 2023-10-08T09:30:40.757 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 567267847 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.758 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "libcephsqlite", 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3236157652 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.759 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "rbd_support", 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 889425091 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.760 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "volumes", 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:0", 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3838045796 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:40.761 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2023-10-08T09:30:40.762 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:30:40.763 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:39 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:40.763 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:39 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/550418937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:39 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1247298524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:40 smithi184 ceph-mon[103675]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 660 op/s 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:40 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2624978766' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:39 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:39 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/550418937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:39 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1247298524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:40 smithi132 ceph-mon[103389]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 660 op/s 2023-10-08T09:30:40.764 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:40 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2624978766' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T09:30:40.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73b40a1480 msgr2=0x7f73b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:40.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73b40a1480 0x7f73b40a3940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f73cc100620 tx=0x7f73c0009260 comp rx=0 tx=0).stop 2023-10-08T09:30:40.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 msgr2=0x7f73cc19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:30:40.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc19da00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f73c4007b60 tx=0x7f73c4008ec0 comp rx=0 tx=0).stop 2023-10-08T09:30:40.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 shutdown_connections 2023-10-08T09:30:40.765 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f73b40a1480 0x7f73b40a3940 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73cc101160 0x7f73cc19da00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73cc0ff5d0 0x7f73cc069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 --2- 172.21.15.49:0/2624978766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73cc1007c0 0x7f73cc19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:30:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 >> 172.21.15.49:0/2624978766 conn(0x7f73cc0fab40 msgr2=0x7f73cc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:30:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 shutdown_connections 2023-10-08T09:30:40.766 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:30:39.084+0000 7f73d089d700 1 -- 172.21.15.49:0/2624978766 wait complete. 2023-10-08T09:30:40.772 DEBUG:tasks.check_counter:Getting stats from a 2023-10-08T09:30:40.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-08T09:30:40.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:39 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:40.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:39 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/550418937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:30:40.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:39 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1247298524' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:30:40.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:40 smithi049 ceph-mon[99485]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 660 op/s 2023-10-08T09:30:40.773 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:40 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2624978766' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-08T09:30:41.359 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:30:42.126 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:42 smithi049 ceph-mon[99485]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 554 op/s 2023-10-08T09:30:42.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:42 smithi132 ceph-mon[103389]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 554 op/s 2023-10-08T09:30:42.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:42 smithi184 ceph-mon[103675]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 554 op/s 2023-10-08T09:30:43.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:43 smithi132 ceph-mon[103389]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2023-10-08T09:30:43.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:43 smithi049 ceph-mon[99485]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2023-10-08T09:30:43.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:43 smithi184 ceph-mon[103675]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2023-10-08T09:30:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:45 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:45.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:45 smithi132 ceph-mon[103389]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2023-10-08T09:30:45.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:45 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:45.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:45 smithi049 ceph-mon[99485]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2023-10-08T09:30:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:45 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:45 smithi184 ceph-mon[103675]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2023-10-08T09:30:47.069 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1338, 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1318, 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 963232, 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 491906, 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2023-10-08T09:30:47.070 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.154379601, 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.047531928, 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.046983521, 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.049535164, 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1317, 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.562486767, 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000427097 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.071 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 619840, 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 174064 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:30:47.072 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 964, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1181, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 237624, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 837042, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 7, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 4, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.103506307, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.032890965, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.022327908, 2023-10-08T09:30:47.073 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.039828319, 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1179, 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.405508623, 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000343942 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 537792 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1177, 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1177, 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 1415734, 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 365252, 2023-10-08T09:30:47.075 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 5, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.100368399, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.025797319, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.027662096, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.038618220, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1176, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.332989025, 2023-10-08T09:30:47.076 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000283153 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.077 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 5146, 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.152967766, 2023-10-08T09:30:47.078 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000029725 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2023-10-08T09:30:47.080 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 284, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 3, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 189, 2023-10-08T09:30:47.081 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 92, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 279, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 2, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 9, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2023-10-08T09:30:47.082 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2023-10-08T09:30:47.083 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:30:47.084 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:30:47.085 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2023-10-08T09:30:47.086 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2023-10-08T09:30:47.087 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2023-10-08T09:30:47.088 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2023-10-08T09:30:47.089 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 784, 2023-10-08T09:30:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2023-10-08T09:30:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2023-10-08T09:30:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2023-10-08T09:30:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2023-10-08T09:30:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2023-10-08T09:30:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2023-10-08T09:30:47.090 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 5302079, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 5300896, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.091 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 784 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 279, 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 281, 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 2, 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 17, 2023-10-08T09:30:47.092 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 17, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 0, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 284, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 286, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 2, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 52740, 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 207152 2023-10-08T09:30:47.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.094 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.095 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.096 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2023-10-08T09:30:47.097 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.098 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.100 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.101 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.102 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.103 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.104 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.105 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.106 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.107 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.108 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:30:47.109 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 1, 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 2, 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 1, 2023-10-08T09:30:47.110 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 1, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 2402, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2023-10-08T09:30:47.111 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:30:47.112 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:30:47.113 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2023-10-08T09:30:47.114 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 200178, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 322, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2023-10-08T09:30:47.115 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8792, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 168, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2023-10-08T09:30:47.116 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 851478, 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 7552, 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2023-10-08T09:30:47.117 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 3157, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 21, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 3, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 3157, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3157, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 11.451687363, 2023-10-08T09:30:47.118 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.003627395 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3157, 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 3158 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 3157, 2023-10-08T09:30:47.119 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 3157, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 1981, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 1174, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2023-10-08T09:30:47.120 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 1, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:30:47.121 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2023-10-08T09:30:47.122 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2023-10-08T09:30:47.123 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 90, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 4, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 5, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 7, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 2, 2023-10-08T09:30:47.124 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2023-10-08T09:30:47.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2023-10-08T09:30:47.126 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 3475, 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 2284607, 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:47.127 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 3475, 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3475, 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 2284607, 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 3157, 2023-10-08T09:30:47.129 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 1107780, 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3157, 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1107780, 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2023-10-08T09:30:47.130 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 3157, 2023-10-08T09:30:47.131 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 3157, 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3157, 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 3157, 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:30:47.132 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2023-10-08T09:30:47.133 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x557787118960": { 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:47.134 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2023-10-08T09:30:47.135 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:47.136 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:47.136 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:47.136 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:47.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:47.136 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:30:47.464 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-10-08T09:30:47.464 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-10-08T09:30:47.464 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2023-10-08T09:30:47.464 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-10-08T09:30:47.465 DEBUG:tasks.check_counter:Getting stats from d 2023-10-08T09:30:47.465 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-08T09:30:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:47 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:47.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:47 smithi184 ceph-mon[103675]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2023-10-08T09:30:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:47 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:47.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:47 smithi132 ceph-mon[103389]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2023-10-08T09:30:47.882 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:47 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:47.883 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:47 smithi049 ceph-mon[99485]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2023-10-08T09:30:48.041 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:30:49.387 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:30:49.387 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:30:49.387 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 295, 2023-10-08T09:30:49.387 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 292, 2023-10-08T09:30:49.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 494800, 2023-10-08T09:30:49.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 175360, 2023-10-08T09:30:49.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2023-10-08T09:30:49.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2023-10-08T09:30:49.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.043437905, 2023-10-08T09:30:49.388 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.016507348, 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.016171385, 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.008302281, 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 291, 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.128531520, 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000441689 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.389 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:49.390 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:49.390 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:49.390 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:49.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.390 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 494800, 2023-10-08T09:30:49.390 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 175360 2023-10-08T09:30:49.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:30:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 6, 2023-10-08T09:30:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 225, 2023-10-08T09:30:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3312, 2023-10-08T09:30:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 193072, 2023-10-08T09:30:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2023-10-08T09:30:49.391 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2023-10-08T09:30:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.016612938, 2023-10-08T09:30:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.012574013, 2023-10-08T09:30:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.002239033, 2023-10-08T09:30:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T09:30:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 224, 2023-10-08T09:30:49.392 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.115265607, 2023-10-08T09:30:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000514578 2023-10-08T09:30:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:49.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-08T09:30:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 193072 2023-10-08T09:30:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:30:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2023-10-08T09:30:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2023-10-08T09:30:49.394 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.000369179, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000282720, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T09:30:49.395 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:49.396 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:49.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.397 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:30:49.397 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:30:49.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.397 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2023-10-08T09:30:49.397 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2023-10-08T09:30:49.397 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2023-10-08T09:30:49.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.398 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2023-10-08T09:30:49.398 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:30:49.398 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2023-10-08T09:30:49.398 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:30:49.398 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:30:49.399 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:30:49.399 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:30:49.399 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:30:49.399 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:30:49.399 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:30:49.400 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:30:49.400 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:30:49.400 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:30:49.400 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:30:49.400 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:30:49.400 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:30:49.401 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2023-10-08T09:30:49.401 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:30:49.401 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2023-10-08T09:30:49.401 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:30:49.401 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:30:49.402 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:30:49.402 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:30:49.402 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:30:49.402 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:30:49.402 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:30:49.402 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:30:49.403 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:30:49.404 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:30:49.404 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:30:49.404 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148721, 2023-10-08T09:30:49.404 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2023-10-08T09:30:49.404 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:30:49.404 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2023-10-08T09:30:49.404 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2023-10-08T09:30:49.405 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2023-10-08T09:30:49.405 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:30:49.405 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2023-10-08T09:30:49.405 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:30:49.405 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2023-10-08T09:30:49.405 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2023-10-08T09:30:49.405 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2023-10-08T09:30:49.406 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:30:49.406 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:30:49.406 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2023-10-08T09:30:49.406 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2023-10-08T09:30:49.406 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2023-10-08T09:30:49.406 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2023-10-08T09:30:49.406 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2023-10-08T09:30:49.407 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:49.408 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 297, 2023-10-08T09:30:49.408 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 453100, 2023-10-08T09:30:49.408 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:49.408 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 297, 2023-10-08T09:30:49.408 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:49.408 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:49.408 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 297, 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 453100, 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:49.409 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:30:49.801 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-08T09:30:49.801 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-08T09:30:49.801 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-10-08T09:30:49.801 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-10-08T09:30:49.801 DEBUG:tasks.check_counter:Getting stats from g 2023-10-08T09:30:49.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-08T09:30:50.132 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:50 smithi049 ceph-mon[99485]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 638 KiB/s wr, 71 op/s 2023-10-08T09:30:50.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:50 smithi132 ceph-mon[103389]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 638 KiB/s wr, 71 op/s 2023-10-08T09:30:50.375 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:30:50.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:50 smithi184 ceph-mon[103675]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 638 KiB/s wr, 71 op/s 2023-10-08T09:30:51.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:51 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:51.203 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:51 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:30:51.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:51 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:51.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:51 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:30:51.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:51 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:51.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:51 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-08T09:30:51.686 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:30:51.686 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:30:51.686 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 1017, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 1019, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 341657, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 316288, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 7, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.118324708, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.038867647, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.029431448, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.039846750, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 1017, 2023-10-08T09:30:51.687 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.300531335, 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000295507 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:30:51.688 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2539, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2518, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 3171655, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 865476, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 3, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.202737937, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.056033293, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.060473084, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.074906743, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 2517, 2023-10-08T09:30:51.689 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.441751233, 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000175507 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 626496, 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 174688 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.690 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 9, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 225, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 4660, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 539360, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 7, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 4, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.017607419, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.012211313, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.003409366, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000088784, 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:51.691 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 224, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.081021393, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000361702 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 538416 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.692 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 1, 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-MDSRank": { 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 5288, 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.150272900, 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000028417 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.693 INFO:teuthology.orchestra.run.smithi049.stdout: "finisher-PurgeQueue": { 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "queue_len": 0, 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "complete_latency": { 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "mds": { 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "request": 0, 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "reply": 0, 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "reply_latency": { 2023-10-08T09:30:51.694 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_reply": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "forward": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_complete": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_fetch_keys": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_commit": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_split": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_merge": 0, 2023-10-08T09:30:51.695 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes": 482, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_top": 142, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_bottom": 310, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pin_tail": 30, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_pinned": 279, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_expired": 1, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "inodes_with_caps": 0, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "caps": 0, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "subtrees": 25, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse": 0, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_hit": 0, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_forward": 0, 2023-10-08T09:30:51.696 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_discover": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_remote_ino": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "traverse_lock": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "load_cent": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "q": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "exported": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "exported_inodes": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "imported": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "imported_inodes": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_dir_fetch": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_backtrace_fetch": 0, 2023-10-08T09:30:51.697 INFO:teuthology.orchestra.run.smithi049.stdout: "openino_peer_discover": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rfiles": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rbytes": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "root_rsnaps": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_set_tag": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_inodes": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T09:30:51.698 INFO:teuthology.orchestra.run.smithi049.stdout: "scrub_file_inodes": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_grant": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_caps_dirty": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_cap_release": 0, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "process_request_cap_release": 0 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.699 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache": { 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_update_receipt": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_try_discover": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_send_discover": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "dir_handle_discover": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_delayed": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_created": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_enqueued": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_reintegrated": 0, 2023-10-08T09:30:51.700 INFO:teuthology.orchestra.run.smithi049.stdout: "strays_migrated": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_processing": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_started": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_completed": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_exportdir": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_flush": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_fragstats": 0, 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: "ireq_inodestats": 0 2023-10-08T09:30:51.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log": { 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "evlrg": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "evadd": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "evex": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "evtrm": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "ev": 926, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "evexg": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "evexd": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "segadd": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "segex": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "segtrm": 0, 2023-10-08T09:30:51.702 INFO:teuthology.orchestra.run.smithi049.stdout: "seg": 2, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "segmjr": 2, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "segexg": 0, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "segexd": 0, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "expos": 4194304, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "wrpos": 6176777, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "rdpos": 6175646, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "jlat": { 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.703 INFO:teuthology.orchestra.run.smithi049.stdout: "replayed": 926 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mem": { 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "ino": 456, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "ino+": 593, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "ino-": 137, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "dir": 52, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "dir+": 56, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "dir-": 4, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "dn": 482, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "dn+": 622, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "dn-": 140, 2023-10-08T09:30:51.704 INFO:teuthology.orchestra.run.smithi049.stdout: "cap": 0, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "cap+": 0, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "cap-": 0, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "rss": 51052, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "heap": 207152 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_server": { 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_client_request": 0, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "dispatch_server_request": 0, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_request": 0, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_client_session": 0, 2023-10-08T09:30:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: "handle_peer_request": 0, 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "req_create_latency": { 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getattr_latency": { 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getfilelock_latency": { 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "req_link_latency": { 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookup_latency": { 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.707 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookuphash_latency": { 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupino_latency": { 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.708 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupname_latency": { 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupparent_latency": { 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.709 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lookupsnap_latency": { 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "req_lssnap_latency": { 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mkdir_latency": { 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.710 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mknod_latency": { 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "req_mksnap_latency": { 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.711 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "req_open_latency": { 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "req_readdir_latency": { 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rename_latency": { 2023-10-08T09:30:51.712 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "req_renamesnap_latency": { 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "req_snapdiff_latency": { 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.713 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmdir_latency": { 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmsnap_latency": { 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "req_rmxattr_latency": { 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setattr_latency": { 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.715 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setdirlayout_latency": { 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setfilelock_latency": { 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setlayout_latency": { 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.716 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "req_setxattr_latency": { 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "req_symlink_latency": { 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.717 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "req_unlink_latency": { 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "req_getvxattr_latency": { 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.718 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessions": { 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "session_count": 1, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "session_add": 2, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "session_remove": 1, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_open": 1, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "sessions_stale": 0, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "total_load": 0, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "average_load": 0, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "avg_session_uptime": 2405, 2023-10-08T09:30:51.719 INFO:teuthology.orchestra.run.smithi049.stdout: "mdthresh_evicted": 0 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:30:51.720 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:30:51.721 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2023-10-08T09:30:51.722 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 214080, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 503, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:30:51.723 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 8792, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 168, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 1432283, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 11958, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2023-10-08T09:30:51.724 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter": { 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "op_active": 0, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "op_laggy": 0, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send": 3241, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "op_send_bytes": 21, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "op_resend": 3, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "op_reply": 3241, 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "op_latency": { 2023-10-08T09:30:51.725 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3241, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 22.442387498, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.006924525 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "op_inflight": 0, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "oplen_avg": { 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 3241, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 3242 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "op": 3241, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "op_r": 3241, 2023-10-08T09:30:51.726 INFO:teuthology.orchestra.run.smithi049.stdout: "op_w": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "op_rmw": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "op_pg": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_stat": 1969, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_create": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_read": 1270, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_write": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writefull": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_writesame": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_append": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_zero": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_truncate": 0, 2023-10-08T09:30:51.727 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_delete": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_mapext": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_sparse_read": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_clonerange": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_getxattr": 1, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_setxattr": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_rmxattr": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_call": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_watch": 0, 2023-10-08T09:30:51.728 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_notify": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "osdop_other": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_active": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_send": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_resend": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "linger_ping": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_active": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_send": 0, 2023-10-08T09:30:51.729 INFO:teuthology.orchestra.run.smithi049.stdout: "poolop_resend": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_active": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_send": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "poolstat_resend": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_active": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_send": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "statfs_resend": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "command_active": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "command_send": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "command_resend": 0, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "map_epoch": 90, 2023-10-08T09:30:51.730 INFO:teuthology.orchestra.run.smithi049.stdout: "map_full": 0, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "map_inc": 4, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_sessions": 6, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_open": 8, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_session_close": 2, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_laggy": 0, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_wr": 0, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_rd": 2, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_del": 0 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "oft": { 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_objs": 0, 2023-10-08T09:30:51.731 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_updates": 0, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "omap_total_removes": 0 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "osd-slow-ops": { 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "slow_ops_count": 0 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "purge_queue": { 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops": 0, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing": 0, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executing_high_water": 0, 2023-10-08T09:30:51.732 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_executed": 0, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "pq_item_in_journal": 0 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 3561, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 3178150, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 3561, 2023-10-08T09:30:51.733 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3561, 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 3178150, 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_bytes": { 2023-10-08T09:30:51.734 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 3241, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 1982478, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3241, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 1982478, 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:51.735 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-objecter_ops": { 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 1024, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:51.736 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 3241, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 3241, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 3241, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 3241, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2023-10-08T09:30:51.738 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-write_buf_throttle-0x561ea21de960": { 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 3758096384, 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:51.739 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 0, 2023-10-08T09:30:51.741 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:51.742 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:51.742 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:51.742 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:51.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:51.742 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:30:52.157 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-10-08T09:30:52.157 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-10-08T09:30:52.157 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-10-08T09:30:52.157 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-10-08T09:30:52.157 DEBUG:tasks.check_counter:Getting stats from j 2023-10-08T09:30:52.157 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-08T09:30:52.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:52 smithi132 ceph-mon[103389]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 341 KiB/s wr, 39 op/s 2023-10-08T09:30:52.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:30:52.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:30:52.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:52 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:30:52.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:52 smithi049 ceph-mon[99485]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 341 KiB/s wr, 39 op/s 2023-10-08T09:30:52.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:30:52.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:30:52.383 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:52 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:30:52.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:52 smithi184 ceph-mon[103675]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 341 KiB/s wr, 39 op/s 2023-10-08T09:30:52.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-08T09:30:52.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-08T09:30:52.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:52 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' 2023-10-08T09:30:52.745 INFO:teuthology.orchestra.run.smithi049.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.a/config 2023-10-08T09:30:53.526 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:53 smithi049 ceph-mon[99485]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:53.526 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:53 smithi049 ceph-mon[99485]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 KiB/s wr, 18 op/s 2023-10-08T09:30:53.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:53 smithi132 ceph-mon[103389]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:53.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:53 smithi132 ceph-mon[103389]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 KiB/s wr, 18 op/s 2023-10-08T09:30:53.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:53 smithi184 ceph-mon[103675]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-08T09:30:53.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:53 smithi184 ceph-mon[103675]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 20 KiB/s wr, 18 op/s 2023-10-08T09:30:54.020 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2023-10-08T09:30:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 301, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 296, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 498336, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 177056, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 0, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.045079710, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.016705419, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.018446678, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.008007876, 2023-10-08T09:30:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 296, 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.097466419, 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000329278 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 498336, 2023-10-08T09:30:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 177056 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 2, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 224, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 320, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 192800, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 4, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 2, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.015517472, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.011797768, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.001880155, 2023-10-08T09:30:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 222, 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.045604126, 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000205423 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 192800 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_messages": 0, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages": 2, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_bytes": 0, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_bytes": 0, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_created_connections": 2, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_active_connections": 1, 2023-10-08T09:30:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_total_time": 0.000588956, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_send_time": 0.000000000, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_recv_time": 0.000500816, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "cct": { 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "total_workers": 0, 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "unhealthy_workers": 0 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool": { 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "bloom_filter_items": 0, 2023-10-08T09:30:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:30:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_items": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blob_items": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:30:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_txc_items": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_writing_items": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_bytes": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_items": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:30:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_bytes": 148744, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_anon_items": 30, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "buffer_meta_items": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bytes": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_items": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mapbl_items": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pglog_items": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_bytes": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_items": 0, 2023-10-08T09:30:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_bytes": 0, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "pgmap_items": 0, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_bytes": 928, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_co_items": 2, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_bytes": 0, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_1_items": 0, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_bytes": 0, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "unittest_2_items": 0 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2023-10-08T09:30:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "val": 0, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "max": 104857600, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "get_started": 0, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "get": 299, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "get_sum": 453360, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_fail": 0, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "get_or_fail_success": 299, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "take": 0, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "take_sum": 0, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "put": 299, 2023-10-08T09:30:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "put_sum": 453360, 2023-10-08T09:30:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: "wait": { 2023-10-08T09:30:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: "avgcount": 0, 2023-10-08T09:30:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: "sum": 0.000000000, 2023-10-08T09:30:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: "avgtime": 0.000000000 2023-10-08T09:30:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2023-10-08T09:30:54.034 INFO:teuthology.orchestra.run.smithi049.stdout:} 2023-10-08T09:30:54.416 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-08T09:30:54.417 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-08T09:30:54.417 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-10-08T09:30:54.417 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-10-08T09:30:54.417 DEBUG:tasks.check_counter:Getting stats from b 2023-10-08T09:30:54.417 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-08T09:30:56.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:56 smithi132 ceph-mon[103389]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2023-10-08T09:30:56.320 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:30:56.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:56 smithi049 ceph-mon[99485]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2023-10-08T09:30:56.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:56 smithi184 ceph-mon[103675]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 15 op/s 2023-10-08T09:30:58.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:30:58 smithi132 ceph-mon[103389]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-10-08T09:30:58.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:30:58 smithi049 ceph-mon[99485]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-10-08T09:30:58.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:30:58 smithi184 ceph-mon[103675]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-10-08T09:31:00.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:00 smithi132 ceph-mon[103389]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2023-10-08T09:31:00.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:00 smithi049 ceph-mon[99485]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2023-10-08T09:31:00.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:00 smithi184 ceph-mon[103675]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 16 op/s 2023-10-08T09:31:01.753 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 437233, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 439625, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 70798067, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 76958765, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 18, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 2, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 19.907953564, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 8.741439636, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 14.288036167, 2023-10-08T09:31:01.754 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.110624750, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 439624, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 12.539311546, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000028522 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 872574, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.123661411, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000141 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 681088, 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 185168 2023-10-08T09:31:01.755 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 3803, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 3728, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1457329, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 3172449, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 14, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 6, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.267126187, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.103388103, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.127880911, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.053298235, 2023-10-08T09:31:01.756 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3727, 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2.316034159, 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000621420 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 4492, 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.001013370, 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000225 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 49680, 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 593616 2023-10-08T09:31:01.757 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 394865, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 395501, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 63324820, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 72035168, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 215, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 8, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 14.952249719, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 5.706052491, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 10.705756530, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.096919934, 2023-10-08T09:31:01.758 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 395499, 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 16.152823963, 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000040841 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 786018, 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.111430735, 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000141 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.759 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2533, 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.655097749, 2023-10-08T09:31:01.760 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000653414 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 127, 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.006530680, 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000051422 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.761 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 1913, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 1694, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1694, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 7.644907699, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.004512932 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 219, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 4, 2023-10-08T09:31:01.762 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 1, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 540, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 3, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 162, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 2, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 140, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 20, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 159, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 1077, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 120, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 120, 2023-10-08T09:31:01.763 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 12, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 2166, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 1432, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 210, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 5, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 1, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 1106, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 46, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 1064, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 31, 2023-10-08T09:31:01.764 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 146, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 385, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 1037, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 1643088900, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 385, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 37, 2023-10-08T09:31:01.765 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 2, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 35, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 348, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 12, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 117, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 1167, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 970, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 830, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 14, 2023-10-08T09:31:01.766 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 142 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 148, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 282, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 142, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 288, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 337, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 32, 2023-10-08T09:31:01.767 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 31, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 1, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 2, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 46, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 3, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:01.768 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "evlrg": 0, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 2691, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 2663, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 2663, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 28, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 3, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 73, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 72, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 72, 2023-10-08T09:31:01.769 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 1, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "segmjr": 1, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 1, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 11075680, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 11206047, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 0, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 890, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 7.139271610, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.008021653 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.770 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 0 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 151, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 1191, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 1040, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 43, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 222, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 179, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 162, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 1553, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 1391, 2023-10-08T09:31:01.771 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 120, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 1173, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 1053, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 67836, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 207152 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 2009, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 20, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 1913, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 160, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 42, 2023-10-08T09:31:01.772 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1009, 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 6.684131040, 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.006624510 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 55, 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.400174544, 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.007275900 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.773 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000999645, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000999645 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.062164817, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.012432963 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.774 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.775 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.776 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 35, 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.042280853, 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001208024 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.777 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 495, 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.138768222, 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000280339 2023-10-08T09:31:01.778 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 42, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.148503837, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.003535805 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 23, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.098295920, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.004273735 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.779 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.780 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2, 2023-10-08T09:31:01.781 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.001502276, 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000751138 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 7, 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.020334924, 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002904989 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.783 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 20, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.047751621, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002387581 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:01.784 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 3, 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 2, 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 1, 2023-10-08T09:31:01.785 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 2, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 2, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 963, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 360, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 360, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:01.786 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:01.787 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:01.788 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 327149, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 260, 2023-10-08T09:31:01.789 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 168, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:01.790 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 569814, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 5674, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-10-08T09:31:01.791 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 4172, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 9188239, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 31, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 4172, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 4172, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 370.376535900, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.088776734 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 4172, 2023-10-08T09:31:01.792 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 8078 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 4172, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 406, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 3766, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 477, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 1889, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 14, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 902, 2023-10-08T09:31:01.793 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 240, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 3, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 91, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 392, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 2860, 2023-10-08T09:31:01.794 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-10-08T09:31:01.795 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-10-08T09:31:01.796 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 90, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 10, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 12, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 12, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 1159, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 10, 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 41 2023-10-08T09:31:01.797 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 1, 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:01.798 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 23, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 5, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 31, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 835897, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 60322418, 2023-10-08T09:31:01.799 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 835897, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 835897, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 60322418, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.800 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 4172, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 9191550, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 4172, 2023-10-08T09:31:01.801 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 9191550, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:01.802 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 4172, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 4172, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 4172, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 4172, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.803 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 31, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 3379, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 31, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 12, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 3379, 2023-10-08T09:31:01.804 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x558a023fe6e0": { 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 2691, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 7011743, 2023-10-08T09:31:01.805 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 2691, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 890, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 7011743, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:01.806 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-10-08T09:31:02.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:02 smithi132 ceph-mon[103389]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2023-10-08T09:31:02.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:02 smithi049 ceph-mon[99485]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2023-10-08T09:31:02.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:02 smithi184 ceph-mon[103675]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2023-10-08T09:31:03.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:03 smithi132 ceph-mon[103389]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2023-10-08T09:31:03.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:03 smithi049 ceph-mon[99485]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2023-10-08T09:31:03.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:03 smithi184 ceph-mon[103675]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 16 op/s 2023-10-08T09:31:05.526 INFO:tasks.check_counter:Daemon mds.b mds.exported=46 2023-10-08T09:31:05.526 INFO:tasks.check_counter:Daemon mds.b mds.imported=31 2023-10-08T09:31:05.526 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=148 2023-10-08T09:31:05.526 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=282 2023-10-08T09:31:05.527 DEBUG:tasks.check_counter:Getting stats from e 2023-10-08T09:31:05.527 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-08T09:31:06.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:06 smithi132 ceph-mon[103389]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 KiB/s wr, 13 op/s 2023-10-08T09:31:06.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:06 smithi049 ceph-mon[99485]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 KiB/s wr, 13 op/s 2023-10-08T09:31:06.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:06 smithi184 ceph-mon[103675]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.9 KiB/s wr, 13 op/s 2023-10-08T09:31:07.699 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:31:08.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:08 smithi132 ceph-mon[103389]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2023-10-08T09:31:08.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:08 smithi049 ceph-mon[99485]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2023-10-08T09:31:08.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:08 smithi184 ceph-mon[103675]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2023-10-08T09:31:10.264 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:10 smithi132 ceph-mon[103389]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2023-10-08T09:31:10.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:10 smithi049 ceph-mon[99485]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2023-10-08T09:31:10.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:10 smithi184 ceph-mon[103675]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2023-10-08T09:31:12.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:12 smithi132 ceph-mon[103389]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-10-08T09:31:12.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:12 smithi049 ceph-mon[99485]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-10-08T09:31:12.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:12 smithi184 ceph-mon[103675]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2023-10-08T09:31:13.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:13 smithi132 ceph-mon[103389]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-10-08T09:31:13.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:13 smithi049 ceph-mon[99485]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-10-08T09:31:13.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:13 smithi184 ceph-mon[103675]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2023-10-08T09:31:14.777 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-10-08T09:31:14.777 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1272, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 1274, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 2960270, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 395656, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 7, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 2, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.100790476, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.030660792, 2023-10-08T09:31:14.778 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.025365333, 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.035940470, 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1272, 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.278870720, 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000219237 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.779 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1301, 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 1274, 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 898144, 2023-10-08T09:31:14.780 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 481382, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 4, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.118692158, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.030640594, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.035428035, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.047192857, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1273, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.297926568, 2023-10-08T09:31:14.781 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000234035 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 660240, 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 180640 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:14.782 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1057, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 1280, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 356448, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 876013, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 5, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.103460100, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.031366803, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.023879100, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.038780621, 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:14.783 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1279, 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.295992965, 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000231425 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 549152 2023-10-08T09:31:14.784 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5542, 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.150772968, 2023-10-08T09:31:14.785 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000027205 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.786 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 0, 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 0, 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-10-08T09:31:14.787 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 0, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 0, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 0, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 382, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 3, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 331, 2023-10-08T09:31:14.788 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 48, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 377, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 1, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 0, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 0, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 7, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 0, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 0, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 0, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 0, 2023-10-08T09:31:14.789 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:14.790 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 0, 2023-10-08T09:31:14.791 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 0, 2023-10-08T09:31:14.792 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 0 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:14.793 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-10-08T09:31:14.794 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "evlrg": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 1522, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2023-10-08T09:31:14.795 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 0, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 3, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "segmjr": 2, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 4194304, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 6824159, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 6823585, 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-10-08T09:31:14.796 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 1522 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 183, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 402, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 219, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 16, 2023-10-08T09:31:14.797 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 36, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 20, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 382, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 410, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 28, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 0, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 0, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 0, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 50760, 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 207152 2023-10-08T09:31:14.798 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 0, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 0, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 0, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 0, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 0, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.799 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.800 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.801 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.802 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.803 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.804 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-10-08T09:31:14.805 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.806 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.807 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.808 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.809 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.810 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.811 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.812 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.813 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 1, 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 0, 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 1, 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-10-08T09:31:14.814 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 0, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 0, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 2542, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:14.815 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:14.816 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:14.817 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 196662, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 228, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:31:14.818 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 168, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:14.819 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 702134, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 5686, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-10-08T09:31:14.820 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 3290, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 21, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 3, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 3290, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3290, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 13.157674203, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.003999293 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-10-08T09:31:14.821 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3290, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 3291 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 3290, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 3290, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 0, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 2016, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 0, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 1272, 2023-10-08T09:31:14.822 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 1, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 0, 2023-10-08T09:31:14.823 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-10-08T09:31:14.824 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-10-08T09:31:14.825 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 90, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 4, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 5, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 6, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 1, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 0, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 2, 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 0 2023-10-08T09:31:14.826 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 0, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T09:31:14.827 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 0, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 0, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 3626, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 3868379, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:14.828 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 3626, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 3626, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 3868379, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.829 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 3290, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2629860, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 3290, 2023-10-08T09:31:14.830 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2629860, 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-10-08T09:31:14.831 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 3290, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 3290, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 3290, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 3290, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.832 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:14.833 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x559ce4abc960": { 2023-10-08T09:31:14.834 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 0, 2023-10-08T09:31:14.835 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:14.836 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:14.836 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:14.836 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:14.836 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.836 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:14.836 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-10-08T09:31:15.288 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-08T09:31:15.288 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-10-08T09:31:15.288 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-10-08T09:31:15.288 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-10-08T09:31:15.288 DEBUG:tasks.check_counter:Getting stats from h 2023-10-08T09:31:15.288 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-08T09:31:16.016 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:31:16.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:16 smithi132 ceph-mon[103389]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-10-08T09:31:16.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:16 smithi049 ceph-mon[99485]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-10-08T09:31:16.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:16 smithi184 ceph-mon[103675]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-10-08T09:31:18.184 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:18 smithi132 ceph-mon[103389]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-08T09:31:18.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:18 smithi049 ceph-mon[99485]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-08T09:31:18.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:18 smithi184 ceph-mon[103675]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2023-10-08T09:31:18.836 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-10-08T09:31:18.836 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:18.836 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 432916, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 432516, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 70765477, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 71522875, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 10, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 3, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 20.429813079, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 8.778489424, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 14.671481553, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.024721560, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 432518, 2023-10-08T09:31:18.837 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 8.012571568, 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000018525 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 863802, 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.114365595, 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000132 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.838 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2889102, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 2888678, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 469078203, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 456911698, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 8, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 6, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 130.686226069, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 56.364418925, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 94.673304046, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.015885771, 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:18.839 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2888679, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 63.744726265, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000022067 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5776079, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.686990526, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000118 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 666848, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 186656 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.840 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 1862, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 3037, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 584463, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 2007772, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 17, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 10, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.148909923, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.056572642, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.103061664, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.006810000, 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:18.841 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3030, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.902802381, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000297954 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3397, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000758685, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000223 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 470720 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.842 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 902, 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.625777507, 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000693766 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:18.843 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 588, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.036121238, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000061430 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 940, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 831, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 831, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 2.454568608, 2023-10-08T09:31:18.844 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002953752 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 109, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 11, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 0, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 202, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 8, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 183, 2023-10-08T09:31:18.845 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 11, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 190, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 14, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 173, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 173, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 15, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 1743, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 1486, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 109, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 30, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:18.846 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 70, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 168, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 0, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 19, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 8, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 24, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 181, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 51, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 1039, 2023-10-08T09:31:18.847 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 1603115627, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 51, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 51, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 4, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 620, 2023-10-08T09:31:18.848 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 453, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 304, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 89, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 804 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 155, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 337, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 178, 2023-10-08T09:31:18.849 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 300, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 159, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 198, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 198, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:18.850 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 19, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "evlrg": 0, 2023-10-08T09:31:18.851 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 1522, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 1522, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 3, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 3, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "segmjr": 2, 2023-10-08T09:31:18.852 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 4194304, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 6824159, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 0, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 366, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 8.402126889, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.022956630 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 0 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-10-08T09:31:18.853 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 205, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 411, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 206, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 48, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 188, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 140, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 202, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 419, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 217, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 173, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 530, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 357, 2023-10-08T09:31:18.854 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 55984, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 198960 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 1065, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 0, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 940, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 13, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 19, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 195, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.394457824, 2023-10-08T09:31:18.855 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002022860 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2, 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.008933574, 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.004466787 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-10-08T09:31:18.856 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 299, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.522620542, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.005092376 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.857 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.858 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.859 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.860 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 132, 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.092961763, 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000704255 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-10-08T09:31:18.861 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.005092518, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001018503 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 195, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.424410171, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002176462 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.862 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:18.863 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.864 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.865 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.866 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 3, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.006092216, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.002030738 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.867 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 1, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 0, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 1, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 92, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 92, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 979, 2023-10-08T09:31:18.868 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:18.869 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:18.870 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:18.871 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 290201, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 310, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 704, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 8, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:18.872 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 168, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 744140, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 7371, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.873 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 1164, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 2709469, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 28, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 1164, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1164, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 20.521491861, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.017630147 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.874 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1164, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1256 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 1164, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 114, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 1050, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 0, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 10, 2023-10-08T09:31:18.875 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 63, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 429, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 126, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 0, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 408, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 51, 2023-10-08T09:31:18.876 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 20, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2023-10-08T09:31:18.877 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-10-08T09:31:18.878 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 90, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 4, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 12, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 12, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 110, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 0, 2023-10-08T09:31:18.879 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 39 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 1, 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-10-08T09:31:18.880 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 10, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 5, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 198, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 3323879, 2023-10-08T09:31:18.881 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 241258187, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 3323879, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 3323879, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 241258187, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.882 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 1164, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2731051, 2023-10-08T09:31:18.883 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1164, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2731051, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:18.884 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 1164, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 1164, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1164, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1164, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.885 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 198, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 21582, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 198, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 63, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 21582, 2023-10-08T09:31:18.886 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x5610a9e9c6e0": { 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 1522, 2023-10-08T09:31:18.887 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 2629855, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 1522, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 366, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2629855, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:18.888 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-10-08T09:31:19.316 INFO:tasks.check_counter:Daemon mds.h mds.exported=19 2023-10-08T09:31:19.316 INFO:tasks.check_counter:Daemon mds.h mds.imported=24 2023-10-08T09:31:19.316 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=155 2023-10-08T09:31:19.316 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=337 2023-10-08T09:31:19.317 DEBUG:tasks.check_counter:Getting stats from k 2023-10-08T09:31:19.317 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-08T09:31:20.054 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.b/config 2023-10-08T09:31:20.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:20 smithi132 ceph-mon[103389]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-10-08T09:31:20.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:20 smithi049 ceph-mon[99485]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-10-08T09:31:20.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:20 smithi184 ceph-mon[103675]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-10-08T09:31:22.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:22 smithi132 ceph-mon[103389]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-10-08T09:31:22.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:22 smithi049 ceph-mon[99485]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-10-08T09:31:22.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:22 smithi184 ceph-mon[103675]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2023-10-08T09:31:23.171 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2919470, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 2920266, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 461601739, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 476182251, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 14, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 5, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 126.258705684, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 55.832791580, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 91.173206199, 2023-10-08T09:31:23.172 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.020968890, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2920265, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 97.725026973, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000033464 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 5837402, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.725975203, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000124 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 3744, 2023-10-08T09:31:23.173 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 1008 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2390110, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 2390989, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 378697092, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 388045276, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 12, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 9, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 84.867654697, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 32.646724419, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 61.164944525, 2023-10-08T09:31:23.174 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.013865219, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2390986, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 135.463751418, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000056656 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 4779495, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.583091392, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000121 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 652960, 2023-10-08T09:31:23.175 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 189456 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_messages": 2416, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages": 2885, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_bytes": 1215274, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_bytes": 1581032, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_created_connections": 11, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_active_connections": 4, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_total_time": 0.186043164, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_send_time": 0.088312705, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_recv_time": 0.104581632, 2023-10-08T09:31:23.176 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_running_fast_dispatch_time": 0.009395540, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2885, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1.304700424, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000452235 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 4386, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000987799, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000225 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: "msgr_send_encrypted_bytes": 470800 2023-10-08T09:31:23.177 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "cct": { 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "total_workers": 1, 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "unhealthy_workers": 0 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-MDSRank": { 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1309, 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.372995201, 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000284946 2023-10-08T09:31:23.178 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: "queue_len": 0, 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: "complete_latency": { 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 141, 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.006066584, 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000043025 2023-10-08T09:31:23.179 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "mds": { 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "request": 420, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "reply": 343, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "reply_latency": { 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 343, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.415949636, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001212681 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_reply": 0, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "forward": 77, 2023-10-08T09:31:23.180 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_complete": 0, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_fetch_keys": 0, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_commit": 11, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_split": 0, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_merge": 0, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes": 474, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_top": 3, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_bottom": 8, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pin_tail": 463, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_pinned": 469, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_expired": 135, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "inodes_with_caps": 448, 2023-10-08T09:31:23.181 INFO:teuthology.orchestra.run.smithi132.stdout: "caps": 448, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "subtrees": 24, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse": 790, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_hit": 645, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_forward": 71, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_discover": 18, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "traverse_lock": 10, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "load_cent": 1094, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "q": 1, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "exported": 9, 2023-10-08T09:31:23.182 INFO:teuthology.orchestra.run.smithi132.stdout: "exported_inodes": 138, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "imported": 13, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "imported_inodes": 581, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_backtrace_fetch": 291, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rfiles": 1039, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rbytes": 1603115627, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "root_rsnaps": 0, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_fetch": 291, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:23.183 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_inodes": 2, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_dirfrag_rstats": 2, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "scrub_file_inodes": 289, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_inode_file_caps": 6, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_revoke": 39, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_grant": 244, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:23.184 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps": 290, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_caps_dirty": 97, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_cap_release": 27, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "process_request_cap_release": 112 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_cache": { 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update": 66, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_update_receipt": 130, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_try_discover": 74, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_send_discover": 109, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "dir_handle_discover": 585, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays": 0, 2023-10-08T09:31:23.185 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_created": 20, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_enqueued": 20, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_reintegrated": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "strays_migrated": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_started": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "recovery_completed": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:31:23.186 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_exportdir": 9, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_flush": 0, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "ireq_inodestats": 0 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_log": { 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "evlrg": 0, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "evadd": 936, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "evex": 0, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "evtrm": 0, 2023-10-08T09:31:23.187 INFO:teuthology.orchestra.run.smithi132.stdout: "ev": 936, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "evexg": 0, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "evexd": 0, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "segadd": 2, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "segex": 0, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "segtrm": 0, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "seg": 2, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "segmjr": 2, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "segexg": 0, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "segexd": 0, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "expos": 4194304, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "wrpos": 6186391, 2023-10-08T09:31:23.188 INFO:teuthology.orchestra.run.smithi132.stdout: "rdpos": 0, 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "jlat": { 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 529, 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 16.795993515, 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.031750460 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "replayed": 0 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_mem": { 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "ino": 469, 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "ino+": 603, 2023-10-08T09:31:23.189 INFO:teuthology.orchestra.run.smithi132.stdout: "ino-": 134, 2023-10-08T09:31:23.190 INFO:teuthology.orchestra.run.smithi132.stdout: "dir": 63, 2023-10-08T09:31:23.190 INFO:teuthology.orchestra.run.smithi132.stdout: "dir+": 95, 2023-10-08T09:31:23.190 INFO:teuthology.orchestra.run.smithi132.stdout: "dir-": 32, 2023-10-08T09:31:23.190 INFO:teuthology.orchestra.run.smithi132.stdout: "dn": 474, 2023-10-08T09:31:23.190 INFO:teuthology.orchestra.run.smithi132.stdout: "dn+": 634, 2023-10-08T09:31:23.190 INFO:teuthology.orchestra.run.smithi132.stdout: "dn-": 160, 2023-10-08T09:31:23.190 INFO:teuthology.orchestra.run.smithi132.stdout: "cap": 448, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "cap+": 887, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "cap-": 439, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "rss": 56692, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "heap": 198960 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_server": { 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_client_request": 506, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "dispatch_server_request": 35, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_request": 420, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_client_session": 187, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "handle_peer_request": 87, 2023-10-08T09:31:23.191 INFO:teuthology.orchestra.run.smithi132.stdout: "req_create_latency": { 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 22, 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.032259013, 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.001466318 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getattr_latency": { 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 2, 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.001850190, 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000925095 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.192 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: "req_link_latency": { 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookup_latency": { 2023-10-08T09:31:23.193 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 239, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.220021520, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000920592 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupino_latency": { 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000235815, 2023-10-08T09:31:23.194 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000235815 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupname_latency": { 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.195 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "req_lssnap_latency": { 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mkdir_latency": { 2023-10-08T09:31:23.196 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mknod_latency": { 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "req_mksnap_latency": { 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.197 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "req_open_latency": { 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 25, 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.024269180, 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000970767 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "req_readdir_latency": { 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 31, 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.029654862, 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000956608 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.198 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rename_latency": { 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 13, 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.075556506, 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.005812038 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.199 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmdir_latency": { 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:23.200 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setattr_latency": { 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.201 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:23.202 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setlayout_latency": { 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "req_setxattr_latency": { 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.203 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "req_symlink_latency": { 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "req_unlink_latency": { 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 10, 2023-10-08T09:31:23.204 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.032102550, 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.003210255 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_sessions": { 2023-10-08T09:31:23.205 INFO:teuthology.orchestra.run.smithi132.stdout: "session_count": 1, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "session_add": 2, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "session_remove": 1, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_open": 1, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "sessions_stale": 0, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "total_load": 76, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "average_load": 76, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "avg_session_uptime": 973, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "mempool": { 2023-10-08T09:31:23.206 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bloom_filter_items": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:23.207 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:23.208 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_items": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:23.209 INFO:teuthology.orchestra.run.smithi132.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_bytes": 327746, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_anon_items": 583, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_bytes": 528, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "buffer_meta_items": 6, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_bytes": 0, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_items": 0, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:23.210 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:23.211 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:23.211 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_items": 168, 2023-10-08T09:31:23.211 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:23.211 INFO:teuthology.orchestra.run.smithi132.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:23.211 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:23.211 INFO:teuthology.orchestra.run.smithi132.stdout: "pgmap_items": 0, 2023-10-08T09:31:23.211 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_bytes": 1667013, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_co_items": 16391, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_1_items": 0, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "unittest_2_items": 0 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "objecter": { 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "op_active": 0, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "op_laggy": 0, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send": 1085, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "op_send_bytes": 2050435, 2023-10-08T09:31:23.212 INFO:teuthology.orchestra.run.smithi132.stdout: "op_resend": 33, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "op_reply": 1085, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "op_latency": { 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1085, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 24.662392702, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.022730315 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "op_inflight": 0, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "oplen_avg": { 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 1085, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 1189 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.213 INFO:teuthology.orchestra.run.smithi132.stdout: "op": 1085, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "op_r": 311, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "op_w": 774, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "op_rmw": 0, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "op_pg": 0, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_stat": 0, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_create": 10, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_read": 20, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_write": 549, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writefull": 81, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_writesame": 0, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_append": 0, 2023-10-08T09:31:23.214 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_zero": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_truncate": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_delete": 52, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_mapext": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_getxattr": 291, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_setxattr": 20, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_call": 0, 2023-10-08T09:31:23.215 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_watch": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_notify": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "osdop_other": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_active": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_send": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_resend": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "linger_ping": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_active": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_send": 0, 2023-10-08T09:31:23.216 INFO:teuthology.orchestra.run.smithi132.stdout: "poolop_resend": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_active": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_send": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "poolstat_resend": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_active": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_send": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "statfs_resend": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "command_active": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "command_send": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "command_resend": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "map_epoch": 90, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "map_full": 0, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "map_inc": 4, 2023-10-08T09:31:23.217 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_sessions": 12, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_open": 12, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_session_close": 0, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_laggy": 0, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_wr": 125, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_rd": 0, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_del": 41 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "oft": { 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_objs": 1, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:23.218 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_updates": 0, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "omap_total_removes": 0 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "osd-slow-ops": { 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "slow_ops_count": 0 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "purge_queue": { 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_ops_high_water": 4, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing": 0, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executing_high_water": 2, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_executed": 20, 2023-10-08T09:31:23.219 INFO:teuthology.orchestra.run.smithi132.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 5311998, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 363413561, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 5311999, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 5311998, 2023-10-08T09:31:23.220 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 363413493, 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 104857600, 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:23.221 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 1085, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 2052615, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1085, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2052615, 2023-10-08T09:31:23.222 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 1024, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 0, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 0, 2023-10-08T09:31:23.223 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 1085, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 1085, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 1085, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1085, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.224 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 20, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 2180, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 20, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 20, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 2180, 2023-10-08T09:31:23.225 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "throttle-write_buf_throttle-0x55845ca0a6e0": { 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "val": 0, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "max": 3758096384, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "get_started": 0, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "get": 936, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "get_sum": 1992087, 2023-10-08T09:31:23.226 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:23.227 INFO:teuthology.orchestra.run.smithi132.stdout: "get_or_fail_success": 936, 2023-10-08T09:31:23.227 INFO:teuthology.orchestra.run.smithi132.stdout: "take": 0, 2023-10-08T09:31:23.227 INFO:teuthology.orchestra.run.smithi132.stdout: "take_sum": 0, 2023-10-08T09:31:23.227 INFO:teuthology.orchestra.run.smithi132.stdout: "put": 529, 2023-10-08T09:31:23.227 INFO:teuthology.orchestra.run.smithi132.stdout: "put_sum": 1992087, 2023-10-08T09:31:23.227 INFO:teuthology.orchestra.run.smithi132.stdout: "wait": { 2023-10-08T09:31:23.228 INFO:teuthology.orchestra.run.smithi132.stdout: "avgcount": 0, 2023-10-08T09:31:23.228 INFO:teuthology.orchestra.run.smithi132.stdout: "sum": 0.000000000, 2023-10-08T09:31:23.228 INFO:teuthology.orchestra.run.smithi132.stdout: "avgtime": 0.000000000 2023-10-08T09:31:23.228 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.228 INFO:teuthology.orchestra.run.smithi132.stdout: } 2023-10-08T09:31:23.228 INFO:teuthology.orchestra.run.smithi132.stdout:} 2023-10-08T09:31:23.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:23 smithi132 ceph-mon[103389]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2023-10-08T09:31:23.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:23 smithi049 ceph-mon[99485]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2023-10-08T09:31:23.685 INFO:tasks.check_counter:Daemon mds.k mds.exported=9 2023-10-08T09:31:23.685 INFO:tasks.check_counter:Daemon mds.k mds.imported=13 2023-10-08T09:31:23.686 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=66 2023-10-08T09:31:23.686 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=130 2023-10-08T09:31:23.686 DEBUG:tasks.check_counter:Getting stats from c 2023-10-08T09:31:23.686 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-08T09:31:23.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:23 smithi184 ceph-mon[103675]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2023-10-08T09:31:24.383 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:31:25.793 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1209, 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1211, 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 4590035, 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 376009, 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.111374675, 2023-10-08T09:31:25.794 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.028201576, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.030790976, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.042124482, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1209, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.260706963, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000215638 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.795 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 949, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1176, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 233954, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 837639, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 7, 2023-10-08T09:31:25.796 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.092422068, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.031154794, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.018848819, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.035046120, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1173, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.239944852, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000204556 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:25.797 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 543056 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1605, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1576, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 3838752, 2023-10-08T09:31:25.798 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 575601, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 3, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.145276231, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.046212638, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.044414223, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.047258751, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1576, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.287420620, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000182373 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.799 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 663936, 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 182496 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2023-10-08T09:31:25.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 5685, 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.165367958, 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000029088 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:25.801 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2023-10-08T09:31:25.802 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2023-10-08T09:31:25.803 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 188, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 57, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 123, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 8, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 106, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 6, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 11, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2023-10-08T09:31:25.804 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2023-10-08T09:31:25.805 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 1037, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 1643101180, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:25.806 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2023-10-08T09:31:25.807 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:25.808 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:31:25.809 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2707, 2023-10-08T09:31:25.810 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 71, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 11075680, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 11237161, 2023-10-08T09:31:25.811 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 11231034, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 2707 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 147, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 1187, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 1040, 2023-10-08T09:31:25.812 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 29, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 84, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 55, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 188, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 1229, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 1041, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 52700, 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 207152 2023-10-08T09:31:25.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.814 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2023-10-08T09:31:25.815 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.816 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.817 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2023-10-08T09:31:25.818 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.819 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.821 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.822 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:25.823 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.824 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.826 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.827 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 3, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 2, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 1, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 2542, 2023-10-08T09:31:25.828 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:25.829 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:25.830 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:25.831 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 194903, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 180, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:25.832 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 168, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 499587, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 4029, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2023-10-08T09:31:25.833 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 3420, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 26, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 4, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 3420, 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2023-10-08T09:31:25.834 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3420, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 10.559790391, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.003087658 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3420, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 3421 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 3420, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 3420, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2023-10-08T09:31:25.835 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 2128, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1289, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2023-10-08T09:31:25.836 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 2, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2023-10-08T09:31:25.837 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2023-10-08T09:31:25.838 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 90, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 4, 2023-10-08T09:31:25.839 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 5, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 6, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 1, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2023-10-08T09:31:25.840 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:25.841 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 3759, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 8304138, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 3759, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:25.842 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3759, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 8304138, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:25.843 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3420, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 7042867, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3420, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 7042867, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.844 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3420, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3420, 2023-10-08T09:31:25.845 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3420, 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3420, 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:25.846 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.847 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55da4f176960": { 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:25.848 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:25.849 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-10-08T09:31:26.229 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:26 smithi184 ceph-mon[103675]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-08T09:31:26.254 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-10-08T09:31:26.254 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-10-08T09:31:26.254 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-10-08T09:31:26.254 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-10-08T09:31:26.254 DEBUG:tasks.check_counter:Getting stats from f 2023-10-08T09:31:26.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-08T09:31:26.296 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:26 smithi132 ceph-mon[103389]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-08T09:31:26.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:26 smithi049 ceph-mon[99485]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2023-10-08T09:31:26.888 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:31:28.222 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-10-08T09:31:28.222 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:28.222 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 3334, 2023-10-08T09:31:28.222 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 3334, 2023-10-08T09:31:28.222 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 4417884, 2023-10-08T09:31:28.222 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 1036085, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 6, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.239753146, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.064329298, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.060461939, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.098620258, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3333, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.472826328, 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000141862 2023-10-08T09:31:28.223 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 5, 2023-10-08T09:31:28.224 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 233, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1036, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 629948, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 7, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 3, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.011971924, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.007945332, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.002161285, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000082444, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 230, 2023-10-08T09:31:28.225 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.055948374, 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000243253 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-08T09:31:28.226 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 629264 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 338, 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 311, 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 661324, 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 182823, 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 4, 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 2, 2023-10-08T09:31:28.227 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.042814181, 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.014876546, 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.019967020, 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.008069790, 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 310, 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.082343278, 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000265623 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:28.228 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 661056, 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 182512 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2023-10-08T09:31:28.229 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 5528, 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.136499756, 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000024692 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.230 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 0, 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 0, 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.231 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 463, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 0, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 463, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 463, 2023-10-08T09:31:28.232 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 3, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2023-10-08T09:31:28.233 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2023-10-08T09:31:28.234 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 0, 2023-10-08T09:31:28.235 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 0, 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 0, 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 0, 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 0 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2023-10-08T09:31:28.236 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2023-10-08T09:31:28.237 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2023-10-08T09:31:28.238 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2016, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2023-10-08T09:31:28.239 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 3, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 7676397, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 7675823, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.240 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 2016 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 76, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 153, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 77, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 12, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 22, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 10, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 463, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 570, 2023-10-08T09:31:28.241 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 107, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 0, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 0, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 0, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 51540, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 207152 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 0, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 0, 2023-10-08T09:31:28.242 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 0, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 0, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.243 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.244 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.245 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.246 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2023-10-08T09:31:28.247 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.249 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.250 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.251 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.252 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.253 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2023-10-08T09:31:28.254 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.255 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.256 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.256 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2023-10-08T09:31:28.257 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.258 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 1, 2023-10-08T09:31:28.259 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 0, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 1, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 0, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 0, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 2545, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2023-10-08T09:31:28.260 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:28.261 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:28.262 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 184584, 2023-10-08T09:31:28.263 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 116, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 168, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:28.264 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 529718, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 17497, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2023-10-08T09:31:28.265 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 3333, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 21, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 3, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 3333, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3333, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 14.017721456, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.004205737 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2023-10-08T09:31:28.266 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 3333, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 3334 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 3333, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 3333, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 0, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 2037, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 0, 2023-10-08T09:31:28.267 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1294, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 1, 2023-10-08T09:31:28.268 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2023-10-08T09:31:28.269 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2023-10-08T09:31:28.270 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 90, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 4, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 4, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 6, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 2, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2023-10-08T09:31:28.271 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 0, 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 2, 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 0 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 0, 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2023-10-08T09:31:28.272 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 0, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 0, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 0, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:28.273 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 3673, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 4729332, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 3673, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3673, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 4729332, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.274 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:28.275 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3333, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3482098, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3333, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3482098, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:28.276 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 3333, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3333, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 3333, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3333, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.277 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:28.278 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x55c7c17cc960": { 2023-10-08T09:31:28.279 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 0, 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:28.280 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:28.281 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:28.281 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:28.281 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.281 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:28.281 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-10-08T09:31:28.281 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:28 smithi184 ceph-mon[103675]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-10-08T09:31:28.296 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:28 smithi132 ceph-mon[103389]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-10-08T09:31:28.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:28 smithi049 ceph-mon[99485]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2023-10-08T09:31:28.655 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-10-08T09:31:28.656 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-10-08T09:31:28.656 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-10-08T09:31:28.656 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-10-08T09:31:28.656 DEBUG:tasks.check_counter:Getting stats from i 2023-10-08T09:31:28.656 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-08T09:31:29.277 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:31:30.048 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:30 smithi184 ceph-mon[103675]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-10-08T09:31:30.298 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:30 smithi132 ceph-mon[103389]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-10-08T09:31:30.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:30 smithi049 ceph-mon[99485]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2766, 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 2693, 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1457488, 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 4548643, 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 10, 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 5, 2023-10-08T09:31:30.485 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.217310334, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.081720392, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.089071950, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.047867138, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2693, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.310572760, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000115325 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1840, 2023-10-08T09:31:30.486 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000318282, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000172 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 674400, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 188656 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 5930, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 6795, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 1781925, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 3979529, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 19, 2023-10-08T09:31:30.487 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 8, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.343112764, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.133301994, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.263110606, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.005971448, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 6792, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.725158877, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000106766 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:30.488 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 10818, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.002148425, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000198 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 480, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 636720 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1436, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1059, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 525029, 2023-10-08T09:31:30.489 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 503028, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 9, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 6, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.087777347, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.025872263, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.071992989, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.009057895, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1057, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.221409223, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000209469 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.490 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1891, 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000375493, 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000198 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 0 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.491 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2846, 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.239217287, 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000435424 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 16, 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000438986, 2023-10-08T09:31:30.492 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000027436 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 2044, 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 2018, 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2018, 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 5.338275061, 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.002645329 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2023-10-08T09:31:30.493 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 26, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 488, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 4, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 461, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 23, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 477, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 75, 2023-10-08T09:31:30.494 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 73, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 73, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 10, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 5330, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 4047, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 25, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 31, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 435, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 91, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 0, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 8, 2023-10-08T09:31:30.495 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 73, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 7, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 102, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 52, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 1039, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 1603115627, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 52, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:30.496 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 3, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 3, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 49, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 464, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 1140, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 1420, 2023-10-08T09:31:30.497 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 575, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 57, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 1231 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 202, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 513, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 270, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 462, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 205, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:30.498 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 288, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 1, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 287, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 8, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2023-10-08T09:31:30.499 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:31:30.501 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:30.501 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2023-10-08T09:31:30.501 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.501 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2023-10-08T09:31:30.501 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2023-10-08T09:31:30.501 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 2016, 2023-10-08T09:31:30.501 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 2016, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 3, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 3, 2023-10-08T09:31:30.502 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 7676397, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1385, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 7.318706250, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.005284264 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2023-10-08T09:31:30.503 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 103, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 175, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 72, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 42, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 270, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 228, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 488, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 592, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 104, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 73, 2023-10-08T09:31:30.504 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 378, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 305, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 58088, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 198960 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 3240, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 0, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 2044, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 555, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 31, 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2023-10-08T09:31:30.505 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 65, 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.059493575, 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000915285 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 9, 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.030129977, 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.003347775 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.506 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 287, 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.844088739, 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.002941075 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 929, 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1.230972880, 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001325051 2023-10-08T09:31:30.507 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1, 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000208330, 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000208330 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2023-10-08T09:31:30.508 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.509 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.510 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2023-10-08T09:31:30.511 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 133, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.163560685, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001229779 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 18, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.027060582, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001503365 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 288, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 2.521084786, 2023-10-08T09:31:30.512 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.008753766 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.513 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.514 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.515 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2023-10-08T09:31:30.516 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 288, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.461675507, 2023-10-08T09:31:30.517 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.001603039 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2023-10-08T09:31:30.518 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 1, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 0, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 1, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 264, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 264, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 983, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:31:30.519 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:30.520 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:30.521 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 294523, 2023-10-08T09:31:30.522 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 198, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 168, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:30.523 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 706071, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 22251, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2023-10-08T09:31:30.524 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 1693, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 3543071, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 28, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 1693, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1693, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 12.077759710, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.007133939 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2023-10-08T09:31:30.525 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1693, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 1896 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 1693, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 53, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 1640, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 1386, 2023-10-08T09:31:30.526 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 65, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 14, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 52, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:30.527 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2023-10-08T09:31:30.528 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2023-10-08T09:31:30.529 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 90, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 4, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 12, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 13, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 1, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 238, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 110 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.530 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 2, 2023-10-08T09:31:30.531 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 1, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 1, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 10128, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 2831270, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:30.532 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 10128, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 10128, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 2831270, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:30.533 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 1693, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 3543180, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1693, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3543180, 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:30.534 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:30.535 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 1693, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 1693, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1693, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1693, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:30.536 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 1, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 109, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 1, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 109, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.537 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x5584bf9146e0": { 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2016, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 3482093, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2016, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:30.538 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1385, 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 3482093, 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:30.539 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-10-08T09:31:30.934 INFO:tasks.check_counter:Daemon mds.i mds.exported=8 2023-10-08T09:31:30.934 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2023-10-08T09:31:30.934 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=202 2023-10-08T09:31:30.934 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=513 2023-10-08T09:31:30.934 DEBUG:tasks.check_counter:Getting stats from l 2023-10-08T09:31:30.935 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7b3143bae3b6870a749bfb9810ed6e7d0e89d7de shell --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-08T09:31:31.516 INFO:teuthology.orchestra.run.smithi184.stderr:Inferring config /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/mon.c/config 2023-10-08T09:31:32.297 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:32 smithi132 ceph-mon[103389]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:31:32.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:32 smithi049 ceph-mon[99485]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:31:32.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:32 smithi184 ceph-mon[103675]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-08T09:31:32.781 INFO:teuthology.orchestra.run.smithi184.stdout:{ 2023-10-08T09:31:32.781 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-0": { 2023-10-08T09:31:32.781 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 2852940, 2023-10-08T09:31:32.781 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 2852090, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 463021898, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 452345095, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 11, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 4, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 81.998453443, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 30.309499212, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 59.431379310, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.014483919, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2852091, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 61.150754309, 2023-10-08T09:31:32.782 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000021440 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 5703552, 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.489239032, 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000085 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 448 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-1": { 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 930, 2023-10-08T09:31:32.783 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 940, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 919196, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 477327, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 13, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 8, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.083108233, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.033043590, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.045954723, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.008792174, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 939, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.278657591, 2023-10-08T09:31:32.784 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000296759 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 969, 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000181443, 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000187 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 662992, 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 190320 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "AsyncMessenger::Worker-2": { 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_messages": 1119, 2023-10-08T09:31:32.785 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages": 1632, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_bytes": 427561, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_bytes": 1475727, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_created_connections": 14, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_active_connections": 7, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_total_time": 0.083020768, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_send_time": 0.034577607, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_recv_time": 0.057362700, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_running_fast_dispatch_time": 0.000739291, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_messages_queue_lat": { 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 1628, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.244219295, 2023-10-08T09:31:32.786 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000150011 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_handle_ack_lat": { 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2090, 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000453668, 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000217 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_recv_encrypted_bytes": 1232, 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "msgr_send_encrypted_bytes": 562928 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "cct": { 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "total_workers": 1, 2023-10-08T09:31:32.787 INFO:teuthology.orchestra.run.smithi184.stdout: "unhealthy_workers": 0 2023-10-08T09:31:32.788 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.788 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-MDSRank": { 2023-10-08T09:31:32.788 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:32.788 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:32.788 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 620, 2023-10-08T09:31:32.788 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.114633263, 2023-10-08T09:31:32.788 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000184892 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "finisher-PurgeQueue": { 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "queue_len": 0, 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "complete_latency": { 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 17, 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000538242, 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000031661 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "mds": { 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "request": 503, 2023-10-08T09:31:32.789 INFO:teuthology.orchestra.run.smithi184.stdout: "reply": 385, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "reply_latency": { 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 385, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.227893241, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000591930 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_reply": 0, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "forward": 118, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_complete": 0, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_fetch_keys": 0, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_commit": 11, 2023-10-08T09:31:32.790 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_split": 0, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_merge": 0, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes": 304, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_top": 8, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_bottom": 296, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pin_tail": 0, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_pinned": 289, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_expired": 14, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "inodes_with_caps": 269, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "caps": 269, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "subtrees": 18, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse": 1086, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_hit": 865, 2023-10-08T09:31:32.791 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_forward": 118, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_discover": 30, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_dir_fetch": 0, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_remote_ino": 0, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "traverse_lock": 0, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "load_cent": 1056, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "q": 1, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "exported": 0, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "exported_inodes": 0, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "imported": 7, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "imported_inodes": 273, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_dir_fetch": 0, 2023-10-08T09:31:32.792 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_backtrace_fetch": 129, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "openino_peer_discover": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rfiles": 1039, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rbytes": 1603115627, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "root_rsnaps": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_fetch": 129, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_set_tag": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_backtrace_repaired": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_inotable_repaired": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_inodes": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dir_base_inodes": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_dirfrag_rstats": 0, 2023-10-08T09:31:32.793 INFO:teuthology.orchestra.run.smithi184.stdout: "scrub_file_inodes": 129, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_inode_file_caps": 0, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_revoke": 3, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_grant": 51, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_trunc": 0, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps": 227, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_caps_dirty": 192, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_cap_release": 6, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "process_request_cap_release": 9 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.794 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_cache": { 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update": 208, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_update_receipt": 384, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_try_discover": 203, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_send_discover": 343, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "dir_handle_discover": 216, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays": 0, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_delayed": 0, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "num_strays_enqueuing": 0, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_created": 2, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_enqueued": 2, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_reintegrated": 0, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "strays_migrated": 0, 2023-10-08T09:31:32.795 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_processing": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_enqueued": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "num_recovering_prioritized": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_started": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "recovery_completed": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_enqueue_scrub": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_exportdir": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_flush": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragmentdir": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_fragstats": 0, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "ireq_inodestats": 0 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.796 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_log": { 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "evlrg": 0, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "evadd": 788, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "evex": 0, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "evtrm": 0, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "ev": 788, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "evexg": 0, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "evexd": 0, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "segadd": 2, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "segex": 0, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "segtrm": 0, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "seg": 2, 2023-10-08T09:31:32.797 INFO:teuthology.orchestra.run.smithi184.stdout: "segmjr": 2, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "segexg": 0, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "segexd": 0, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "expos": 4194304, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "wrpos": 5304375, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "rdpos": 0, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "jlat": { 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 221, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 9.216708657, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.041704564 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: "replayed": 0 2023-10-08T09:31:32.798 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_mem": { 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "ino": 304, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "ino+": 313, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "ino-": 9, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "dir": 49, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "dir+": 201, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "dir-": 152, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "dn": 304, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "dn+": 326, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "dn-": 22, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "cap": 269, 2023-10-08T09:31:32.799 INFO:teuthology.orchestra.run.smithi184.stdout: "cap+": 532, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "cap-": 263, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "rss": 58284, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "heap": 198960 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_server": { 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_client_request": 504, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "dispatch_server_request": 14, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_request": 503, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_client_session": 17, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "handle_peer_request": 14, 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "req_create_latency": { 2023-10-08T09:31:32.800 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getattr_latency": { 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getfilelock_latency": { 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.801 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "req_link_latency": { 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookup_latency": { 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 352, 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.197529521, 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000561163 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.802 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookuphash_latency": { 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupino_latency": { 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupname_latency": { 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.803 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupparent_latency": { 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lookupsnap_latency": { 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.804 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "req_lssnap_latency": { 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mkdir_latency": { 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.805 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mknod_latency": { 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "req_mksnap_latency": { 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "req_open_latency": { 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 24, 2023-10-08T09:31:32.806 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.022064134, 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000919338 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "req_readdir_latency": { 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 7, 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.006504344, 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000929192 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rename_latency": { 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.807 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "req_renamesnap_latency": { 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "req_snapdiff_latency": { 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmdir_latency": { 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.808 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmsnap_latency": { 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "req_rmxattr_latency": { 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.809 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setattr_latency": { 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setdirlayout_latency": { 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setfilelock_latency": { 2023-10-08T09:31:32.810 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setlayout_latency": { 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "req_setxattr_latency": { 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.811 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "req_symlink_latency": { 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "req_unlink_latency": { 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 2, 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.001795242, 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000897621 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.812 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_revoke_eviction": 0, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "cap_acquisition_throttle": 0, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "req_getvxattr_latency": { 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_sessions": { 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "session_count": 1, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "session_add": 2, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "session_remove": 1, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_open": 1, 2023-10-08T09:31:32.813 INFO:teuthology.orchestra.run.smithi184.stdout: "sessions_stale": 0, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "total_load": 44, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "average_load": 44, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "avg_session_uptime": 2549, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "mdthresh_evicted": 0 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "mempool": { 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_bytes": 0, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "bloom_filter_items": 0, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_bytes": 0, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_alloc_items": 0, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_bytes": 0, 2023-10-08T09:31:32.814 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_data_items": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_onode_items": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_meta_items": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_bytes": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_other_items": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_cache_buffer_items": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_bytes": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_extent_items": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_bytes": 0, 2023-10-08T09:31:32.815 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_blob_items": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_shared_blob_items": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_inline_bl_items": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_bytes": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_fsck_items": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_bytes": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_txc_items": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_deferred_items": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_bytes": 0, 2023-10-08T09:31:32.816 INFO:teuthology.orchestra.run.smithi184.stdout: "bluestore_writing_items": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_bytes": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_items": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_bytes": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_reader_items": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_bytes": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "bluefs_file_writer_items": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_bytes": 329157, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_anon_items": 412, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_bytes": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "buffer_meta_items": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_bytes": 0, 2023-10-08T09:31:32.817 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_items": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_bytes": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_mapbl_items": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_bytes": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_pglog_items": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_bytes": 8792, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_items": 168, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_bytes": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "osdmap_mapping_items": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_bytes": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "pgmap_items": 0, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_bytes": 1082592, 2023-10-08T09:31:32.818 INFO:teuthology.orchestra.run.smithi184.stdout: "mds_co_items": 11102, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_bytes": 0, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_1_items": 0, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_bytes": 0, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "unittest_2_items": 0 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "objecter": { 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "op_active": 0, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "op_laggy": 0, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send": 470, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "op_send_bytes": 1149411, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "op_resend": 33, 2023-10-08T09:31:32.819 INFO:teuthology.orchestra.run.smithi184.stdout: "op_reply": 470, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "op_latency": { 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 470, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 11.259778331, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.023956975 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "op_inflight": 0, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "oplen_avg": { 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 470, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 510 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "op": 470, 2023-10-08T09:31:32.820 INFO:teuthology.orchestra.run.smithi184.stdout: "op_r": 130, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "op_w": 340, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "op_rmw": 0, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "op_pg": 0, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_stat": 0, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_create": 10, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_read": 1, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_write": 222, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writefull": 62, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_writesame": 0, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_append": 0, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_zero": 0, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_truncate": 0, 2023-10-08T09:31:32.821 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_delete": 16, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_mapext": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_sparse_read": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_clonerange": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_getxattr": 129, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_setxattr": 20, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_cmpxattr": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_rmxattr": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_resetxattrs": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_call": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_watch": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_notify": 0, 2023-10-08T09:31:32.822 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_src_cmpxattr": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_pgls_filter": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "osdop_other": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_active": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_send": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_resend": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "linger_ping": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_active": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_send": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "poolop_resend": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_active": 0, 2023-10-08T09:31:32.823 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_send": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "poolstat_resend": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_active": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_send": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "statfs_resend": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "command_active": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "command_send": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "command_resend": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "map_epoch": 90, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "map_full": 0, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "map_inc": 4, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_sessions": 12, 2023-10-08T09:31:32.824 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_open": 12, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_session_close": 0, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "osd_laggy": 0, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_wr": 41, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_rd": 0, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_del": 9 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "oft": { 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_objs": 1, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_kv_pairs": 0, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_updates": 0, 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: "omap_total_removes": 0 2023-10-08T09:31:32.825 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "osd-slow-ops": { 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "slow_ops_count": 0 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "purge_queue": { 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops": 0, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_ops_high_water": 4, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing": 0, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executing_high_water": 2, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_executed": 2, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "pq_item_in_journal": 0 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.826 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 72, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2854988, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 207398178, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2854988, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 2854987, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 207398106, 2023-10-08T09:31:32.827 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_bytes": { 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 104857600, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:32.828 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 470, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 1149629, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 470, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1149629, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.829 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-objecter_ops": { 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 1024, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 0, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 0, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 0, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 470, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 470, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 470, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 470, 2023-10-08T09:31:32.830 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle": { 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 2, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 218, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:32.831 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 2, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 1, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 218, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: }, 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "throttle-write_buf_throttle-0x564a6311e6e0": { 2023-10-08T09:31:32.832 INFO:teuthology.orchestra.run.smithi184.stdout: "val": 0, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "max": 3758096384, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "get_started": 0, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "get": 788, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "get_sum": 1110071, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_fail": 0, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "get_or_fail_success": 788, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "take": 0, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "take_sum": 0, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "put": 221, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "put_sum": 1110071, 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "wait": { 2023-10-08T09:31:32.833 INFO:teuthology.orchestra.run.smithi184.stdout: "avgcount": 0, 2023-10-08T09:31:32.834 INFO:teuthology.orchestra.run.smithi184.stdout: "sum": 0.000000000, 2023-10-08T09:31:32.834 INFO:teuthology.orchestra.run.smithi184.stdout: "avgtime": 0.000000000 2023-10-08T09:31:32.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.834 INFO:teuthology.orchestra.run.smithi184.stdout: } 2023-10-08T09:31:32.834 INFO:teuthology.orchestra.run.smithi184.stdout:} 2023-10-08T09:31:33.167 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-10-08T09:31:33.167 INFO:tasks.check_counter:Daemon mds.l mds.imported=7 2023-10-08T09:31:33.167 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=208 2023-10-08T09:31:33.167 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=384 2023-10-08T09:31:33.168 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-08T09:31:33.179 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T09:31:33.179 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:31:33.179 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:31:33.230 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:31:33.231 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:31:33.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:31:33.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.470+0000 7f9c154dc700 1 -- 172.21.15.49:0/2846034054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 msgr2=0x7f9c10105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.471 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.470+0000 7f9c154dc700 1 --2- 172.21.15.49:0/2846034054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 0x7f9c10105ea0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009990 tx=0x7f9bf8009ca0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.470+0000 7f9c154dc700 1 -- 172.21.15.49:0/2846034054 shutdown_connections 2023-10-08T09:31:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.470+0000 7f9c154dc700 1 --2- 172.21.15.49:0/2846034054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c101063e0 0x7f9c101087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.470+0000 7f9c154dc700 1 --2- 172.21.15.49:0/2846034054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c10101180 0x7f9c10103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.470+0000 7f9c154dc700 1 --2- 172.21.15.49:0/2846034054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 0x7f9c10105ea0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.470+0000 7f9c154dc700 1 -- 172.21.15.49:0/2846034054 >> 172.21.15.49:0/2846034054 conn(0x7f9c100fab10 msgr2=0x7f9c100fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.471+0000 7f9c154dc700 1 -- 172.21.15.49:0/2846034054 shutdown_connections 2023-10-08T09:31:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.471+0000 7f9c154dc700 1 -- 172.21.15.49:0/2846034054 wait complete. 2023-10-08T09:31:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 Processor -- start 2023-10-08T09:31:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 -- start start 2023-10-08T09:31:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c10101180 0x7f9c10196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 0x7f9c10196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c101063e0 0x7f9c1019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1019e180 con 0x7f9c101063e0 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1019e2c0 con 0x7f9c10101180 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c154dc700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1019e450 con 0x7f9c10103ab0 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c0effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c10101180 0x7f9c10196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c0f7fe700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c101063e0 0x7f9c1019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c0effd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c10101180 0x7f9c10196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:44108/0 (socket says 172.21.15.49:44108) 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c0effd700 1 -- 172.21.15.49:0/1439021979 learned_addr learned my addr 172.21.15.49:0/1439021979 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.472+0000 7f9c0e7fc700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 0x7f9c10196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c0f7fe700 1 -- 172.21.15.49:0/1439021979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 msgr2=0x7f9c10196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c0f7fe700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 0x7f9c10196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c0f7fe700 1 -- 172.21.15.49:0/1439021979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c10101180 msgr2=0x7f9c10196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.475 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c0f7fe700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c10101180 0x7f9c10196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c0f7fe700 1 -- 172.21.15.49:0/1439021979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf80095f0 con 0x7f9c101063e0 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c0effd700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c10101180 0x7f9c10196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c0f7fe700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c101063e0 0x7f9c1019dbb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800efc0 tx=0x7f9c0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c07fff700 1 -- 172.21.15.49:0/1439021979 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08018070 con 0x7f9c101063e0 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1019e6d0 con 0x7f9c101063e0 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c07fff700 1 -- 172.21.15.49:0/1439021979 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c0800f040 con 0x7f9c101063e0 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c07fff700 1 -- 172.21.15.49:0/1439021979 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800cd40 con 0x7f9c101063e0 2023-10-08T09:31:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.473+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1019eb90 con 0x7f9c101063e0 2023-10-08T09:31:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.474+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c100fc690 con 0x7f9c101063e0 2023-10-08T09:31:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.475+0000 7f9c07fff700 1 -- 172.21.15.49:0/1439021979 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9c08003a00 con 0x7f9c101063e0 2023-10-08T09:31:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.476+0000 7f9c07fff700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bfc0a1480 0x7f9bfc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.476+0000 7f9c07fff700 1 -- 172.21.15.49:0/1439021979 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9c080cd5f0 con 0x7f9c101063e0 2023-10-08T09:31:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.476+0000 7f9c0effd700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bfc0a1480 0x7f9bfc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.476+0000 7f9c0effd700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bfc0a1480 0x7f9bfc0a3940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011550 tx=0x7f9c00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:33.480 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.479+0000 7f9c07fff700 1 -- 172.21.15.49:0/1439021979 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9c08096950 con 0x7f9c101063e0 2023-10-08T09:31:33.497 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:33 smithi184 ceph-mon[103675]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-10-08T09:31:33.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:33 smithi132 ceph-mon[103389]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-10-08T09:31:33.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:33 smithi049 ceph-mon[99485]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-10-08T09:31:33.685 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.685+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c10067450 con 0x7f9c101063e0 2023-10-08T09:31:33.687 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.687+0000 7f9c07fff700 1 -- 172.21.15.49:0/1439021979 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f9c08099f70 con 0x7f9c101063e0 2023-10-08T09:31:33.687 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:31:33.688 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:31:33.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.688+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bfc0a1480 msgr2=0x7f9bfc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bfc0a1480 0x7f9bfc0a3940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011550 tx=0x7f9c00009260 comp rx=0 tx=0).stop 2023-10-08T09:31:33.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c101063e0 msgr2=0x7f9c1019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.689 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c101063e0 0x7f9c1019dbb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800efc0 tx=0x7f9c0800c3a0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 shutdown_connections 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f9bfc0a1480 0x7f9bfc0a3940 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9c101063e0 0x7f9c1019dbb0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c10101180 0x7f9c10196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 --2- 172.21.15.49:0/1439021979 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9c10103ab0 0x7f9c10196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 >> 172.21.15.49:0/1439021979 conn(0x7f9c100fab10 msgr2=0x7f9c10103040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 shutdown_connections 2023-10-08T09:31:33.690 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.689+0000 7f9c154dc700 1 -- 172.21.15.49:0/1439021979 wait complete. 2023-10-08T09:31:33.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T09:09:10.311953+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T09:13:01.183216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T09:13:02.157813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T09:13:06.277641+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T09:31:33.703 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T09:31:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 -- 172.21.15.49:0/2760490689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 msgr2=0x7f7b1c0ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2760490689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c0ffad0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009990 tx=0x7f7b0c009ca0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 -- 172.21.15.49:0/2760490689 shutdown_connections 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2760490689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c0ffad0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2760490689 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1c101240 0x7f7b1c107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2760490689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b1c1008a0 0x7f7b1c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 -- 172.21.15.49:0/2760490689 >> 172.21.15.49:0/2760490689 conn(0x7f7b1c0fac40 msgr2=0x7f7b1c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 -- 172.21.15.49:0/2760490689 shutdown_connections 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.832+0000 7f7b220f5700 1 -- 172.21.15.49:0/2760490689 wait complete. 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 Processor -- start 2023-10-08T09:31:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 -- start start 2023-10-08T09:31:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b1c1008a0 0x7f7b1c196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1c101240 0x7f7b1c19dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c19e280 con 0x7f7b1c101240 2023-10-08T09:31:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c19e3c0 con 0x7f7b1c1008a0 2023-10-08T09:31:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b220f5700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c19e550 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.836 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b1b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b1bfff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1c101240 0x7f7b1c19dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b1affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b1c1008a0 0x7f7b1c196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b1b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41542/0 (socket says 172.21.15.49:41542) 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.833+0000 7f7b1b7fe700 1 -- 172.21.15.49:0/2553351650 learned_addr learned my addr 172.21.15.49:0/2553351650 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b1b7fe700 1 -- 172.21.15.49:0/2553351650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b1c1008a0 msgr2=0x7f7b1c196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b1b7fe700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b1c1008a0 0x7f7b1c196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b1b7fe700 1 -- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1c101240 msgr2=0x7f7b1c19dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b1b7fe700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1c101240 0x7f7b1c19dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b1b7fe700 1 -- 172.21.15.49:0/2553351650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b14009c40 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b1b7fe700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c196750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c000c00 tx=0x7f7b0c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:33.837 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b18ff9700 1 -- 172.21.15.49:0/2553351650 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c01d070 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b18ff9700 1 -- 172.21.15.49:0/2553351650 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b0c021410 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c0095f0 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b18ff9700 1 -- 172.21.15.49:0/2553351650 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c017650 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.834+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c19ea60 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.838 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.835+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c106b80 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.836+0000 7f7b18ff9700 1 -- 172.21.15.49:0/2553351650 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7b0c021580 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.837+0000 7f7b18ff9700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b040a1480 0x7f7b040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.837+0000 7f7b1affd700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b040a1480 0x7f7b040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.837+0000 7f7b18ff9700 1 -- 172.21.15.49:0/2553351650 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7b0c0ceb30 con 0x7f7b1c0ff6b0 2023-10-08T09:31:33.839 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.838+0000 7f7b1affd700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b040a1480 0x7f7b040a3940 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009c60 tx=0x7f7b10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:33.842 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:33.842+0000 7f7b18ff9700 1 -- 172.21.15.49:0/2553351650 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7b0c097e90 con 0x7f7b1c0ff6b0 2023-10-08T09:31:34.043 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.043+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b1c10c140 con 0x7f7b1c0ff6b0 2023-10-08T09:31:34.044 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.043+0000 7f7b18ff9700 1 -- 172.21.15.49:0/2553351650 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b0c09b4b0 con 0x7f7b1c0ff6b0 2023-10-08T09:31:34.044 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2023-10-08T09:31:34.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b040a1480 msgr2=0x7f7b040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b040a1480 0x7f7b040a3940 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7b10009c60 tx=0x7f7b10009260 comp rx=0 tx=0).stop 2023-10-08T09:31:34.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 msgr2=0x7f7b1c196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c196750 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c000c00 tx=0x7f7b0c004730 comp rx=0 tx=0).stop 2023-10-08T09:31:34.046 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 shutdown_connections 2023-10-08T09:31:34.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f7b040a1480 0x7f7b040a3940 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1c0ff6b0 0x7f7b1c196750 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b1c101240 0x7f7b1c19dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 --2- 172.21.15.49:0/2553351650 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b1c1008a0 0x7f7b1c196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.045+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 >> 172.21.15.49:0/2553351650 conn(0x7f7b1c0fac40 msgr2=0x7f7b1c103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:34.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.046+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 shutdown_connections 2023-10-08T09:31:34.047 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.046+0000 7f7b220f5700 1 -- 172.21.15.49:0/2553351650 wait complete. 2023-10-08T09:31:34.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 -- 172.21.15.49:0/3949941594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6581007b0 msgr2=0x7fb658100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/3949941594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6581007b0 0x7fb658100c10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb6480099c0 tx=0x7fb648009cd0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 -- 172.21.15.49:0/3949941594 shutdown_connections 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/3949941594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 0x7fb6580ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/3949941594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb658101150 0x7fb658107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/3949941594 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6581007b0 0x7fb658100c10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 -- 172.21.15.49:0/3949941594 >> 172.21.15.49:0/3949941594 conn(0x7fb6580fab30 msgr2=0x7fb6580fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 -- 172.21.15.49:0/3949941594 shutdown_connections 2023-10-08T09:31:34.189 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.188+0000 7fb65cbf2700 1 -- 172.21.15.49:0/3949941594 wait complete. 2023-10-08T09:31:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 Processor -- start 2023-10-08T09:31:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 -- start start 2023-10-08T09:31:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 0x7fb6581954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6581007b0 0x7fb658195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb658101150 0x7fb65819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65819d020 con 0x7fb658101150 2023-10-08T09:31:34.191 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb658067830 con 0x7fb6581007b0 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65cbf2700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580679c0 con 0x7fb6580ff5c0 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 0x7fb6581954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 0x7fb6581954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41556/0 (socket says 172.21.15.49:41556) 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb65659c700 1 -- 172.21.15.49:0/2641146988 learned_addr learned my addr 172.21.15.49:0/2641146988 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb655d9b700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6581007b0 0x7fb658195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.189+0000 7fb656d9d700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb658101150 0x7fb65819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65659c700 1 -- 172.21.15.49:0/2641146988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6581007b0 msgr2=0x7fb658195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65659c700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6581007b0 0x7fb658195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.193 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65659c700 1 -- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb658101150 msgr2=0x7fb65819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65659c700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb658101150 0x7fb65819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65659c700 1 -- 172.21.15.49:0/2641146988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480095f0 con 0x7fb6580ff5c0 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb655d9b700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6581007b0 0x7fb658195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65659c700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 0x7fb6581954f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e970 tx=0x7fb64c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb6477fe700 1 -- 172.21.15.49:0/2641146988 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c009790 con 0x7fb6580ff5c0 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb658067ca0 con 0x7fb6580ff5c0 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb6477fe700 1 -- 172.21.15.49:0/2641146988 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb64c00c7e0 con 0x7fb6580ff5c0 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb6477fe700 1 -- 172.21.15.49:0/2641146988 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c003c70 con 0x7fb6580ff5c0 2023-10-08T09:31:34.194 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.190+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6580681b0 con 0x7fb6580ff5c0 2023-10-08T09:31:34.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.191+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb658106a90 con 0x7fb6580ff5c0 2023-10-08T09:31:34.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.193+0000 7fb6477fe700 1 -- 172.21.15.49:0/2641146988 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb64c00cc50 con 0x7fb6580ff5c0 2023-10-08T09:31:34.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.193+0000 7fb6477fe700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6400a1470 0x7fb6400a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.195 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.193+0000 7fb655d9b700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6400a1470 0x7fb6400a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.193+0000 7fb6477fe700 1 -- 172.21.15.49:0/2641146988 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fb64c014070 con 0x7fb6580ff5c0 2023-10-08T09:31:34.196 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.194+0000 7fb655d9b700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6400a1470 0x7fb6400a3930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb648009990 tx=0x7fb64801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:34.197 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.197+0000 7fb6477fe700 1 -- 172.21.15.49:0/2641146988 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb64c096660 con 0x7fb6580ff5c0 2023-10-08T09:31:34.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:34 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1439021979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:31:34.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:34 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2553351650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:31:34.400 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.400+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb65810b020 con 0x7fb6580ff5c0 2023-10-08T09:31:34.401 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.400+0000 7fb6477fe700 1 -- 172.21.15.49:0/2641146988 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb64c003dd0 con 0x7fb6580ff5c0 2023-10-08T09:31:34.401 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:31:34.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.402+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6400a1470 msgr2=0x7fb6400a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.402+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6400a1470 0x7fb6400a3930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb648009990 tx=0x7fb64801a040 comp rx=0 tx=0).stop 2023-10-08T09:31:34.403 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.402+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 msgr2=0x7fb6581954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.402+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 0x7fb6581954f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00e970 tx=0x7fb64c00ec80 comp rx=0 tx=0).stop 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 shutdown_connections 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fb6400a1470 0x7fb6400a3930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6580ff5c0 0x7fb6581954f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb658101150 0x7fb65819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 --2- 172.21.15.49:0/2641146988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb6581007b0 0x7fb658195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 >> 172.21.15.49:0/2641146988 conn(0x7fb6580fab30 msgr2=0x7fb658103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 shutdown_connections 2023-10-08T09:31:34.404 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.403+0000 7fb65cbf2700 1 -- 172.21.15.49:0/2641146988 wait complete. 2023-10-08T09:31:34.416 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T09:31:34.547 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 -- 172.21.15.49:0/3278209117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 msgr2=0x7fa2bc100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3278209117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc100d20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009990 tx=0x7fa2b0009ca0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 -- 172.21.15.49:0/3278209117 shutdown_connections 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3278209117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 0x7fa2bc107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3278209117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2bc0ff6d0 0x7fa2bc0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3278209117 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc100d20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 -- 172.21.15.49:0/3278209117 >> 172.21.15.49:0/3278209117 conn(0x7fa2bc0fac40 msgr2=0x7fa2bc0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.544+0000 7fa2c2585700 1 -- 172.21.15.49:0/3278209117 shutdown_connections 2023-10-08T09:31:34.548 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 -- 172.21.15.49:0/3278209117 wait complete. 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 Processor -- start 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 -- start start 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2bc0ff6d0 0x7fa2bc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 0x7fa2bc19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc19e200 con 0x7fa2bc101260 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc19e370 con 0x7fa2bc0ff6d0 2023-10-08T09:31:34.549 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.545+0000 7fa2c2585700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc19e530 con 0x7fa2bc1008c0 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 0x7fa2bc19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2bbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2bc0ff6d0 0x7fa2bc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2bb7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2bb7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.49:41572/0 (socket says 172.21.15.49:41572) 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2bbfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2bc0ff6d0 0x7fa2bc0694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:44150/0 (socket says 172.21.15.49:44150) 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 0x7fa2bc19da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54540/0 (socket says 172.21.15.49:54540) 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2bb7fe700 1 -- 172.21.15.49:0/3879782293 learned_addr learned my addr 172.21.15.49:0/3879782293 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 -- 172.21.15.49:0/3879782293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 msgr2=0x7fa2bc19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.550 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 -- 172.21.15.49:0/3879782293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2bc0ff6d0 msgr2=0x7fa2bc0694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2bc0ff6d0 0x7fa2bc0694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 -- 172.21.15.49:0/3879782293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00095f0 con 0x7fa2bc101260 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2bb7fe700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c0b22700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 0x7fa2bc19da60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400efc0 tx=0x7fa2b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2b97fa700 1 -- 172.21.15.49:0/3879782293 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4018070 con 0x7fa2bc101260 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc19e7b0 con 0x7fa2bc101260 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2b97fa700 1 -- 172.21.15.49:0/3879782293 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2b400f040 con 0x7fa2bc101260 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2b97fa700 1 -- 172.21.15.49:0/3879782293 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400cd40 con 0x7fa2bc101260 2023-10-08T09:31:34.551 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.546+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc19ec70 con 0x7fa2bc101260 2023-10-08T09:31:34.552 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:34 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1439021979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:31:34.552 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:34 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2553351650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:31:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.547+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc106af0 con 0x7fa2bc101260 2023-10-08T09:31:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.549+0000 7fa2b97fa700 1 -- 172.21.15.49:0/3879782293 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa2b4003a00 con 0x7fa2bc101260 2023-10-08T09:31:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.549+0000 7fa2b97fa700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2a40a1480 0x7fa2a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.549+0000 7fa2b97fa700 1 -- 172.21.15.49:0/3879782293 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa2b40cd590 con 0x7fa2bc101260 2023-10-08T09:31:34.553 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.549+0000 7fa2bbfff700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2a40a1480 0x7fa2a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.550+0000 7fa2bbfff700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2a40a1480 0x7fa2a40a3940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc100720 tx=0x7fa2ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:34.554 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.552+0000 7fa2b97fa700 1 -- 172.21.15.49:0/3879782293 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa2b4096950 con 0x7fa2bc101260 2023-10-08T09:31:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:34 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1439021979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:31:34.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:34 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2553351650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-08T09:31:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.757+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2bc10aff0 con 0x7fa2bc101260 2023-10-08T09:31:34.757 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.757+0000 7fa2b97fa700 1 -- 172.21.15.49:0/3879782293 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa2b4099f70 con 0x7fa2bc101260 2023-10-08T09:31:34.758 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2023-10-08T09:31:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2a40a1480 msgr2=0x7fa2a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2a40a1480 0x7fa2a40a3940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc100720 tx=0x7fa2ac009260 comp rx=0 tx=0).stop 2023-10-08T09:31:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 msgr2=0x7fa2bc19da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 0x7fa2bc19da60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400efc0 tx=0x7fa2b400c3a0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 shutdown_connections 2023-10-08T09:31:34.760 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa2a40a1480 0x7fa2a40a3940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa2bc101260 0x7fa2bc19da60 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2bc0ff6d0 0x7fa2bc0694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 --2- 172.21.15.49:0/3879782293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2bc1008c0 0x7fa2bc19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 >> 172.21.15.49:0/3879782293 conn(0x7fa2bc0fac40 msgr2=0x7fa2bc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 shutdown_connections 2023-10-08T09:31:34.761 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.759+0000 7fa2c2585700 1 -- 172.21.15.49:0/3879782293 wait complete. 2023-10-08T09:31:34.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.901+0000 7fdbe6499700 1 -- 172.21.15.49:0/1674413434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe0102370 msgr2=0x7fdbe0108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.901+0000 7fdbe6499700 1 --2- 172.21.15.49:0/1674413434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe0102370 0x7fdbe0108c20 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800b0d0 tx=0x7fdbd800b3e0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.901+0000 7fdbe6499700 1 -- 172.21.15.49:0/1674413434 shutdown_connections 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.901+0000 7fdbe6499700 1 --2- 172.21.15.49:0/1674413434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe01007e0 0x7fdbe0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.901+0000 7fdbe6499700 1 --2- 172.21.15.49:0/1674413434 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe0102370 0x7fdbe0108c20 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.901+0000 7fdbe6499700 1 --2- 172.21.15.49:0/1674413434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe01019d0 0x7fdbe0101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.901+0000 7fdbe6499700 1 -- 172.21.15.49:0/1674413434 >> 172.21.15.49:0/1674413434 conn(0x7fdbe00fbd30 msgr2=0x7fdbe00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:34.904 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 -- 172.21.15.49:0/1674413434 shutdown_connections 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 -- 172.21.15.49:0/1674413434 wait complete. 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 Processor -- start 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 -- start start 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe01007e0 0x7fdbe0196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe01019d0 0x7fdbe0196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe0102370 0x7fdbe019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe019e340 con 0x7fdbe01019d0 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe019e480 con 0x7fdbe0102370 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbe6499700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe019e610 con 0x7fdbe01007e0 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe01019d0 0x7fdbe0196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbdffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe01007e0 0x7fdbe0196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbd7fff700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe01019d0 0x7fdbe0196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54552/0 (socket says 172.21.15.49:54552) 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.902+0000 7fdbd7fff700 1 -- 172.21.15.49:0/3761596936 learned_addr learned my addr 172.21.15.49:0/3761596936 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbe4a36700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe0102370 0x7fdbe019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbd7fff700 1 -- 172.21.15.49:0/3761596936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe01007e0 msgr2=0x7fdbe0196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbd7fff700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe01007e0 0x7fdbe0196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbd7fff700 1 -- 172.21.15.49:0/3761596936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe0102370 msgr2=0x7fdbe019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbd7fff700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe0102370 0x7fdbe019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbd7fff700 1 -- 172.21.15.49:0/3761596936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd8009d00 con 0x7fdbe01019d0 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbdffff700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe01007e0 0x7fdbe0196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbe4a36700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe0102370 0x7fdbe019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbd7fff700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe01019d0 0x7fdbe0196d50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800e9a0 tx=0x7fdbc800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbddffb700 1 -- 172.21.15.49:0/3761596936 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8009790 con 0x7fdbe01019d0 2023-10-08T09:31:34.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbddffb700 1 -- 172.21.15.49:0/3761596936 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc8010430 con 0x7fdbe01019d0 2023-10-08T09:31:34.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe019e8f0 con 0x7fdbe01019d0 2023-10-08T09:31:34.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbddffb700 1 -- 172.21.15.49:0/3761596936 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8018680 con 0x7fdbe01019d0 2023-10-08T09:31:34.907 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.903+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe019ee00 con 0x7fdbe01019d0 2023-10-08T09:31:34.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.904+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe0107b90 con 0x7fdbe01019d0 2023-10-08T09:31:34.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.906+0000 7fdbddffb700 1 -- 172.21.15.49:0/3761596936 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdbc80105a0 con 0x7fdbe01019d0 2023-10-08T09:31:34.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.907+0000 7fdbddffb700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbcc0a1480 0x7fdbcc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:34.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.907+0000 7fdbddffb700 1 -- 172.21.15.49:0/3761596936 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fdbc8014070 con 0x7fdbe01019d0 2023-10-08T09:31:34.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.907+0000 7fdbdffff700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbcc0a1480 0x7fdbcc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:34.908 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.907+0000 7fdbdffff700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbcc0a1480 0x7fdbcc0a3940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0101830 tx=0x7fdbd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:34.910 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:34.910+0000 7fdbddffb700 1 -- 172.21.15.49:0/3761596936 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdbc8096700 con 0x7fdbe01019d0 2023-10-08T09:31:35.115 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.115+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdbe010c000 con 0x7fdbe01019d0 2023-10-08T09:31:35.116 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.115+0000 7fdbddffb700 1 -- 172.21.15.49:0/3761596936 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdbc8099d20 con 0x7fdbe01019d0 2023-10-08T09:31:35.116 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2023-10-08T09:31:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbcc0a1480 msgr2=0x7fdbcc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbcc0a1480 0x7fdbcc0a3940 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0101830 tx=0x7fdbd0009260 comp rx=0 tx=0).stop 2023-10-08T09:31:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe01019d0 msgr2=0x7fdbe0196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe01019d0 0x7fdbe0196d50 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800e9a0 tx=0x7fdbc800ecb0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.118 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 shutdown_connections 2023-10-08T09:31:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fdbcc0a1480 0x7fdbcc0a3940 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe01007e0 0x7fdbe0196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbe0102370 0x7fdbe019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 --2- 172.21.15.49:0/3761596936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbe01019d0 0x7fdbe0196d50 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 >> 172.21.15.49:0/3761596936 conn(0x7fdbe00fbd30 msgr2=0x7fdbe0104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 shutdown_connections 2023-10-08T09:31:35.119 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.117+0000 7fdbe6499700 1 -- 172.21.15.49:0/3761596936 wait complete. 2023-10-08T09:31:35.131 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T09:31:35.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 -- 172.21.15.49:0/1263795081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0ff6d0 msgr2=0x7fa10c0ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:35.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 --2- 172.21.15.49:0/1263795081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0ff6d0 0x7fa10c0ffaf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0099c0 tx=0x7fa0fc009cd0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.260 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 -- 172.21.15.49:0/1263795081 shutdown_connections 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 --2- 172.21.15.49:0/1263795081 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c101260 0x7fa10c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 --2- 172.21.15.49:0/1263795081 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c0ff6d0 0x7fa10c0ffaf0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 --2- 172.21.15.49:0/1263795081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1008c0 0x7fa10c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 -- 172.21.15.49:0/1263795081 >> 172.21.15.49:0/1263795081 conn(0x7fa10c0fac40 msgr2=0x7fa10c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 -- 172.21.15.49:0/1263795081 shutdown_connections 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 -- 172.21.15.49:0/1263795081 wait complete. 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.258+0000 7fa114008700 1 Processor -- start 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 -- start start 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0ff6d0 0x7fa10c1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1008c0 0x7fa10c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c101260 0x7fa10c19dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:35.261 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c19e2d0 con 0x7fa10c101260 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c19e410 con 0x7fa10c0ff6d0 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c19e5a0 con 0x7fa10c1008c0 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c101260 0x7fa10c19dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa111da4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0ff6d0 0x7fa10c1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c101260 0x7fa10c19dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54568/0 (socket says 172.21.15.49:54568) 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 -- 172.21.15.49:0/71775965 learned_addr learned my addr 172.21.15.49:0/71775965 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1115a3700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1008c0 0x7fa10c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 -- 172.21.15.49:0/71775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1008c0 msgr2=0x7fa10c196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1008c0 0x7fa10c196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 -- 172.21.15.49:0/71775965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0ff6d0 msgr2=0x7fa10c1967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:35.262 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0ff6d0 0x7fa10c1967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 -- 172.21.15.49:0/71775965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0095f0 con 0x7fa10c101260 2023-10-08T09:31:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa1125a5700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c101260 0x7fa10c19dd00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa10400ef30 tx=0x7fa10400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa102ffd700 1 -- 172.21.15.49:0/71775965 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104018070 con 0x7fa10c101260 2023-10-08T09:31:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa102ffd700 1 -- 172.21.15.49:0/71775965 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10400f040 con 0x7fa10c101260 2023-10-08T09:31:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c19e820 con 0x7fa10c101260 2023-10-08T09:31:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.259+0000 7fa102ffd700 1 -- 172.21.15.49:0/71775965 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10400cd00 con 0x7fa10c101260 2023-10-08T09:31:35.263 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.260+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c19ed80 con 0x7fa10c101260 2023-10-08T09:31:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.260+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c066ca0 con 0x7fa10c101260 2023-10-08T09:31:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.262+0000 7fa102ffd700 1 -- 172.21.15.49:0/71775965 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa104003a00 con 0x7fa10c101260 2023-10-08T09:31:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.262+0000 7fa102ffd700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0f80a1500 0x7fa0f80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.262+0000 7fa102ffd700 1 -- 172.21.15.49:0/71775965 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa1040cd450 con 0x7fa10c101260 2023-10-08T09:31:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.262+0000 7fa111da4700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0f80a1500 0x7fa0f80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:35.264 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.263+0000 7fa111da4700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0f80a1500 0x7fa0f80a39c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009990 tx=0x7fa0fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:35.266 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.266+0000 7fa102ffd700 1 -- 172.21.15.49:0/71775965 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa104096810 con 0x7fa10c101260 2023-10-08T09:31:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:35 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/2641146988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:35 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3879782293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:35 smithi049 ceph-mon[99485]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-10-08T09:31:35.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:35 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3761596936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.487+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa10c10af30 con 0x7fa10c101260 2023-10-08T09:31:35.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.487+0000 7fa102ffd700 1 -- 172.21.15.49:0/71775965 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+587 (secure 0 0 0) 0x7fa104099e30 con 0x7fa10c101260 2023-10-08T09:31:35.488 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2700533077 2023-10-09T09:06:35.771578+0000 2023-10-08T09:31:35.488 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/656654162 2023-10-09T09:06:35.771578+0000 2023-10-08T09:31:35.488 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/4176953235 2023-10-09T09:06:35.771578+0000 2023-10-08T09:31:35.488 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/907336252 2023-10-09T09:06:08.179468+0000 2023-10-08T09:31:35.488 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/890731290 2023-10-09T09:06:35.771578+0000 2023-10-08T09:31:35.489 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1281967666 2023-10-09T09:06:08.179468+0000 2023-10-08T09:31:35.489 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/1812466550 2023-10-09T09:06:08.179468+0000 2023-10-08T09:31:35.489 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6801/907336252 2023-10-09T09:06:08.179468+0000 2023-10-08T09:31:35.489 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:0/2211252163 2023-10-09T09:06:08.179468+0000 2023-10-08T09:31:35.489 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.15.49:6800/890731290 2023-10-09T09:06:35.771578+0000 2023-10-08T09:31:35.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0f80a1500 msgr2=0x7fa0f80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:35.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0f80a1500 0x7fa0f80a39c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009990 tx=0x7fa0fc000bc0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c101260 msgr2=0x7fa10c19dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c101260 0x7fa10c19dd00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa10400ef30 tx=0x7fa10400c3a0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 shutdown_connections 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fa0f80a1500 0x7fa0f80a39c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa10c101260 0x7fa10c19dd00 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa10c0ff6d0 0x7fa10c1967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 --2- 172.21.15.49:0/71775965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa10c1008c0 0x7fa10c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 >> 172.21.15.49:0/71775965 conn(0x7fa10c0fac40 msgr2=0x7fa10c0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 shutdown_connections 2023-10-08T09:31:35.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:35.489+0000 7fa114008700 1 -- 172.21.15.49:0/71775965 wait complete. 2023-10-08T09:31:35.492 INFO:teuthology.orchestra.run.smithi049.stderr:listed 10 entries 2023-10-08T09:31:35.503 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T09:31:35.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:31:35.503 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /tmp/cephfs 2023-10-08T09:31:35.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:35 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/2641146988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.553 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:35 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3879782293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.553 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:35 smithi132 ceph-mon[103389]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-10-08T09:31:35.553 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:35 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3761596936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.623 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:31:35.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:31:35.623 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-10-08T09:31:35.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:31:35.653 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-10-08T09:31:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:35 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/2641146988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:35 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3879782293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:35 smithi184 ceph-mon[103675]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-10-08T09:31:35.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:35 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3761596936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-08T09:31:35.767 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-10-08T09:31:35.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:31:35.767 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-10-08T09:31:35.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:31:35.804 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-10-08T09:31:35.923 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T09:31:35.923 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2023-10-08T09:31:35.923 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T09:31:35.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284204k,nr_inodes=4071051,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19209 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/aae671c9bfdd41db3a1384a4325637d92ff6fe73ea717772e716d806d2a75e36/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/aae671c9bfdd41db3a1384a4325637d92ff6fe73ea717772e716d806d2a75e36/diff,workdir=/var/lib/containers/storage/overlay/aae671c9bfdd41db3a1384a4325637d92ff6fe73ea717772e716d806d2a75e36/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9a40fe4ce77576d3cc430a50c70c4a7bc548927390e9a76e644892c3d11a5dc9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/9a40fe4ce77576d3cc430a50c70c4a7bc548927390e9a76e644892c3d11a5dc9/diff,workdir=/var/lib/containers/storage/overlay/9a40fe4ce77576d3cc430a50c70c4a7bc548927390e9a76e644892c3d11a5dc9/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260540k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/13e03bee4ecfa3554a51432b5f4da9585010836f5a4e44034259d1eca14ad3b1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/13e03bee4ecfa3554a51432b5f4da9585010836f5a4e44034259d1eca14ad3b1/diff,workdir=/var/lib/containers/storage/overlay/13e03bee4ecfa3554a51432b5f4da9585010836f5a4e44034259d1eca14ad3b1/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/448ab56d6c1c10dd8104807e7d5822b3e63af0eab9eb5649cb63e2a0e52c96c0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/448ab56d6c1c10dd8104807e7d5822b3e63af0eab9eb5649cb63e2a0e52c96c0/diff,workdir=/var/lib/containers/storage/overlay/448ab56d6c1c10dd8104807e7d5822b3e63af0eab9eb5649cb63e2a0e52c96c0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0cb2676e16a521c84671eb21e2cac6ede13921d455d4ebafb52a6b6de942b990/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/0cb2676e16a521c84671eb21e2cac6ede13921d455d4ebafb52a6b6de942b990/diff,workdir=/var/lib/containers/storage/overlay/0cb2676e16a521c84671eb21e2cac6ede13921d455d4ebafb52a6b6de942b990/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a3d1d298d04649df05c40d8233ebbfd19918fe655a73ae5371b9fca2436fcf21/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/a3d1d298d04649df05c40d8233ebbfd19918fe655a73ae5371b9fca2436fcf21/diff,workdir=/var/lib/containers/storage/overlay/a3d1d298d04649df05c40d8233ebbfd19918fe655a73ae5371b9fca2436fcf21/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a98d8f0a56a249aa607050ce752dc51b184b106cd044a96dc22beef5ea8cfefa/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/a98d8f0a56a249aa607050ce752dc51b184b106cd044a96dc22beef5ea8cfefa/diff,workdir=/var/lib/containers/storage/overlay/a98d8f0a56a249aa607050ce752dc51b184b106cd044a96dc22beef5ea8cfefa/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/70d36ee1014f6cf8f0b02d001b287bf12f7bc809120a80dbe2ef3826099db98a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/70d36ee1014f6cf8f0b02d001b287bf12f7bc809120a80dbe2ef3826099db98a/diff,workdir=/var/lib/containers/storage/overlay/70d36ee1014f6cf8f0b02d001b287bf12f7bc809120a80dbe2ef3826099db98a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/655e3cd98062bb2c06227530851bd537aca2a03abcfd3cad4cdb43298ead26ef/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/655e3cd98062bb2c06227530851bd537aca2a03abcfd3cad4cdb43298ead26ef/diff,workdir=/var/lib/containers/storage/overlay/655e3cd98062bb2c06227530851bd537aca2a03abcfd3cad4cdb43298ead26ef/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3e77ff115712886bc64a575566e0fa1d0e608902e90f1c121c11956dce6cab2d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3OYH22ANEYKJW7K34ZPIBCPCPX:/var/lib/containers/storage/overlay/l/NWYEQCBVF4HLZ4VUB7NHZ42XSE,upperdir=/var/lib/containers/storage/overlay/3e77ff115712886bc64a575566e0fa1d0e608902e90f1c121c11956dce6cab2d/diff,workdir=/var/lib/containers/storage/overlay/3e77ff115712886bc64a575566e0fa1d0e608902e90f1c121c11956dce6cab2d/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-08T09:31:35.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:31:35.943 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2023-10-08T09:31:36.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T09:31:36.000 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2023-10-08T09:31:36.224 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2023-10-08T09:31:36.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:31:36.226 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-08T09:31:36.239 INFO:tasks.cephadm:Teardown begin 2023-10-08T09:31:36.239 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T09:31:36.239 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:31:36.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 -- 172.21.15.49:0/1401446470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86541006c0 msgr2=0x7f8654100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 --2- 172.21.15.49:0/1401446470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86541006c0 0x7f8654100ae0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f86480099c0 tx=0x7f8648009cd0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.368 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 -- 172.21.15.49:0/1401446470 shutdown_connections 2023-10-08T09:31:36.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 --2- 172.21.15.49:0/1401446470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8654102250 0x7f8654108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 --2- 172.21.15.49:0/1401446470 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86541006c0 0x7f8654100ae0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 --2- 172.21.15.49:0/1401446470 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86541018b0 0x7f8654101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 -- 172.21.15.49:0/1401446470 >> 172.21.15.49:0/1401446470 conn(0x7f86540fbc50 msgr2=0x7f86540fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:36.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 -- 172.21.15.49:0/1401446470 shutdown_connections 2023-10-08T09:31:36.369 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.367+0000 7f865ae44700 1 -- 172.21.15.49:0/1401446470 wait complete. 2023-10-08T09:31:36.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 Processor -- start 2023-10-08T09:31:36.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 -- start start 2023-10-08T09:31:36.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86541006c0 0x7f8654198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86541018b0 0x7f8654198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8654102250 0x7f865419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a0380 con 0x7f8654102250 2023-10-08T09:31:36.371 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a04c0 con 0x7f86541018b0 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.368+0000 7f865ae44700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a0650 con 0x7f86541006c0 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8653fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86541018b0 0x7f8654198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f86593e1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8654102250 0x7f865419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8658be0700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86541006c0 0x7f8654198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8653fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86541018b0 0x7f8654198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.49:44192/0 (socket says 172.21.15.49:44192) 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8653fff700 1 -- 172.21.15.49:0/1879615380 learned_addr learned my addr 172.21.15.49:0/1879615380 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f86593e1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8654102250 0x7f865419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54580/0 (socket says 172.21.15.49:54580) 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8658be0700 1 -- 172.21.15.49:0/1879615380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86541018b0 msgr2=0x7f8654198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8658be0700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86541018b0 0x7f8654198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.372 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8658be0700 1 -- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8654102250 msgr2=0x7f865419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8658be0700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8654102250 0x7f865419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8658be0700 1 -- 172.21.15.49:0/1879615380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480095f0 con 0x7f86541006c0 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f86593e1700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8654102250 0x7f865419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8658be0700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86541006c0 0x7f8654198850 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86480058e0 tx=0x7f864800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8651ffb700 1 -- 172.21.15.49:0/1879615380 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648025070 con 0x7f86541006c0 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8651ffb700 1 -- 172.21.15.49:0/1879615380 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8648005050 con 0x7f86541006c0 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8654102cc0 con 0x7f86541006c0 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.369+0000 7f8651ffb700 1 -- 172.21.15.49:0/1879615380 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648004950 con 0x7f86541006c0 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.370+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8654103140 con 0x7f86541006c0 2023-10-08T09:31:36.373 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.370+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8654066bc0 con 0x7f86541006c0 2023-10-08T09:31:36.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.372+0000 7f8651ffb700 1 -- 172.21.15.49:0/1879615380 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8648005350 con 0x7f86541006c0 2023-10-08T09:31:36.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.372+0000 7f8651ffb700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86440a1480 0x7f86440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.372+0000 7f8651ffb700 1 -- 172.21.15.49:0/1879615380 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f86480d6b20 con 0x7f86541006c0 2023-10-08T09:31:36.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.372+0000 7f8653fff700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86440a1480 0x7f86440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.374 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.373+0000 7f8653fff700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86440a1480 0x7f86440a3940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8640009d20 tx=0x7f8640009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:36.376 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.376+0000 7f8651ffb700 1 -- 172.21.15.49:0/1879615380 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f864809fee0 con 0x7f86541006c0 2023-10-08T09:31:36.382 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:36 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/71775965' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:31:36.547 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:36 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/71775965' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:31:36.624 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.624+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f865410bee0 con 0x7f86541006c0 2023-10-08T09:31:36.625 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.624+0000 7f8651ffb700 1 -- 172.21.15.49:0/1879615380 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f86480a3500 con 0x7f86541006c0 2023-10-08T09:31:36.625 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:31:36.625 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:30:51.368687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86440a1480 msgr2=0x7f86440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86440a1480 0x7f86440a3940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8640009d20 tx=0x7f8640009260 comp rx=0 tx=0).stop 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86541006c0 msgr2=0x7f8654198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86541006c0 0x7f8654198850 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f86480058e0 tx=0x7f864800f690 comp rx=0 tx=0).stop 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 shutdown_connections 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f86440a1480 0x7f86440a3940 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86541006c0 0x7f8654198850 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8654102250 0x7f865419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 --2- 172.21.15.49:0/1879615380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f86541018b0 0x7f8654198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.628 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 >> 172.21.15.49:0/1879615380 conn(0x7f86540fbc50 msgr2=0x7f86541070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:36.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 shutdown_connections 2023-10-08T09:31:36.629 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.627+0000 7f865ae44700 1 -- 172.21.15.49:0/1879615380 wait complete. 2023-10-08T09:31:36.629 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 47 2023-10-08T09:31:36.641 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T09:31:36.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:36 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/71775965' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-08T09:31:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 -- 172.21.15.49:0/3253126709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 msgr2=0x7f54141079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 --2- 172.21.15.49:0/3253126709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f54141079e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f540c00b0d0 tx=0x7f540c00b3e0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 -- 172.21.15.49:0/3253126709 shutdown_connections 2023-10-08T09:31:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 --2- 172.21.15.49:0/3253126709 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f54141079e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 --2- 172.21.15.49:0/3253126709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54140ff5a0 0x7f54140ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 --2- 172.21.15.49:0/3253126709 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5414100790 0x7f5414100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.769 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 -- 172.21.15.49:0/3253126709 >> 172.21.15.49:0/3253126709 conn(0x7f54140fab10 msgr2=0x7f54140fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 -- 172.21.15.49:0/3253126709 shutdown_connections 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.766+0000 7f541b202700 1 -- 172.21.15.49:0/3253126709 wait complete. 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 Processor -- start 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 -- start start 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54140ff5a0 0x7f54140683b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5414100790 0x7f541419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f541419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541419e1a0 con 0x7f5414101130 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541419e310 con 0x7f5414100790 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541b202700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541419e4d0 con 0x7f54140ff5a0 2023-10-08T09:31:36.770 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541979f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f541419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f5418f9e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54140ff5a0 0x7f54140683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541979f700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f541419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54594/0 (socket says 172.21.15.49:54594) 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f5413fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5414100790 0x7f541419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.767+0000 7f541979f700 1 -- 172.21.15.49:0/108169880 learned_addr learned my addr 172.21.15.49:0/108169880 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541979f700 1 -- 172.21.15.49:0/108169880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54140ff5a0 msgr2=0x7f54140683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541979f700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54140ff5a0 0x7f54140683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541979f700 1 -- 172.21.15.49:0/108169880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5414100790 msgr2=0x7f541419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541979f700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5414100790 0x7f541419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541979f700 1 -- 172.21.15.49:0/108169880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c009d00 con 0x7f5414101130 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f5413fff700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5414100790 0x7f541419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f5418f9e700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54140ff5a0 0x7f54140683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:36.771 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541979f700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f541419da00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f540c007b60 tx=0x7f540c008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f5411ffb700 1 -- 172.21.15.49:0/108169880 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c00e040 con 0x7f5414101130 2023-10-08T09:31:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f5411ffb700 1 -- 172.21.15.49:0/108169880 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f540c01d930 con 0x7f5414101130 2023-10-08T09:31:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541419e750 con 0x7f5414101130 2023-10-08T09:31:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f5411ffb700 1 -- 172.21.15.49:0/108169880 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c01ba00 con 0x7f5414101130 2023-10-08T09:31:36.772 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.768+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541419ebd0 con 0x7f5414101130 2023-10-08T09:31:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.769+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5414106950 con 0x7f5414101130 2023-10-08T09:31:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.770+0000 7f5411ffb700 1 -- 172.21.15.49:0/108169880 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f540c01daa0 con 0x7f5414101130 2023-10-08T09:31:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.771+0000 7f5411ffb700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54040a1480 0x7f54040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.771+0000 7f5418f9e700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54040a1480 0x7f54040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.771+0000 7f5411ffb700 1 -- 172.21.15.49:0/108169880 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f540c0ce490 con 0x7f5414101130 2023-10-08T09:31:36.773 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.771+0000 7f5418f9e700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54040a1480 0x7f54040a3940 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f54141005f0 tx=0x7f5408009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:36.775 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:36.775+0000 7f5411ffb700 1 -- 172.21.15.49:0/108169880 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f540c0977f0 con 0x7f5414101130 2023-10-08T09:31:37.023 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.023+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5414067450 con 0x7f5414101130 2023-10-08T09:31:37.358 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.023+0000 7f5411ffb700 1 -- 172.21.15.49:0/108169880 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+11389 (secure 0 0 0) 0x7f540c09ae10 con 0x7f5414101130 2023-10-08T09:31:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:31:37.359 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14661,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/3458420640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6834","nonce":3458420640},{"type":"v1","addr":"172.21.15.49:6835","nonce":3458420640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/1210414617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":1210414617},{"type":"v1","addr":"172.21.15.49:6837","nonce":1210414617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-08T09:13:02.804952+0000","modified":"2023-10-08T09:30:51.368687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24616,"mds_2":24608,"mds_3":24553,"mds_4":24545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14721":{"gid":14721,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6839/3863085428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6838","nonce":3863085428},{"type":"v1","addr":"172.21.15.49:6839","nonce":3863085428}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14733":{"gid":14733,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.49:6841/1275166284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6840","nonce":1275166284},{"type":"v1","addr":"172.21.15.49:6841","nonce":1275166284}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24545":{"gid":24545,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":16,"addr":"172.21.15.132:6833/3058040154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6832","nonce":3058040154},{"type":"v1","addr":"172.21.15.132:6833","nonce":3058040154}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24553":{"gid":24553,"name":"l","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.184:6833/3616311663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3616311663},{"type":"v1","addr":"172.21.15.184:6833","nonce":3616311663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24575":{"gid":24575,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.132:6835/738661164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6834","nonce":738661164},{"type":"v1","addr":"172.21.15.132:6835","nonce":738661164}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24592":{"gid":24592,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6835/356478734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6834","nonce":356478734},{"type":"v1","addr":"172.21.15.184:6835","nonce":356478734}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24604":{"gid":24604,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.184:6837/3605975147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6836","nonce":3605975147},{"type":"v1","addr":"172.21.15.184:6837","nonce":3605975147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24608":{"gid":24608,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.132:6837/2064856316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6836","nonce":2064856316},{"type":"v1","addr":"172.21.15.132:6837","nonce":2064856316}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24616":{"gid":24616,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":12,"addr":"172.21.15.184:6839/2795080121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6838","nonce":2795080121},{"type":"v1","addr":"172.21.15.184:6839","nonce":2795080121}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.132:6839/1629770277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6838","nonce":1629770277},{"type":"v1","addr":"172.21.15.132:6839","nonce":1629770277}]},"join_fscid":-1,"export_targets":[2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T09:31:37.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54040a1480 msgr2=0x7f54040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54040a1480 0x7f54040a3940 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f54141005f0 tx=0x7f5408009260 comp rx=0 tx=0).stop 2023-10-08T09:31:37.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 msgr2=0x7f541419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.360 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f541419da00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f540c007b60 tx=0x7f540c008ec0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 shutdown_connections 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f54040a1480 0x7f54040a3940 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54140ff5a0 0x7f54140683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5414101130 0x7f541419da00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 --2- 172.21.15.49:0/108169880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5414100790 0x7f541419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 >> 172.21.15.49:0/108169880 conn(0x7f54140fab10 msgr2=0x7f5414103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 shutdown_connections 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.026+0000 7f541b202700 1 -- 172.21.15.49:0/108169880 wait complete. 2023-10-08T09:31:37.361 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 47 2023-10-08T09:31:37.362 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T09:31:37.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.498+0000 7f314321c700 1 -- 172.21.15.49:0/2508229784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0fef70 msgr2=0x7f313c0ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.498+0000 7f314321c700 1 --2- 172.21.15.49:0/2508229784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0fef70 0x7f313c0ff390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f31300098f0 tx=0x7f3130009c00 comp rx=0 tx=0).stop 2023-10-08T09:31:37.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.498+0000 7f314321c700 1 -- 172.21.15.49:0/2508229784 shutdown_connections 2023-10-08T09:31:37.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.498+0000 7f314321c700 1 --2- 172.21.15.49:0/2508229784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0fef70 0x7f313c0ff390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.498+0000 7f314321c700 1 --2- 172.21.15.49:0/2508229784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313c100290 0x7f313c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.498+0000 7f314321c700 1 --2- 172.21.15.49:0/2508229784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0ff8d0 0x7f313c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.498+0000 7f314321c700 1 -- 172.21.15.49:0/2508229784 >> 172.21.15.49:0/2508229784 conn(0x7f313c0fab10 msgr2=0x7f313c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 -- 172.21.15.49:0/2508229784 shutdown_connections 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 -- 172.21.15.49:0/2508229784 wait complete. 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 Processor -- start 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 -- start start 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0fef70 0x7f313c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0ff8d0 0x7f313c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313c100290 0x7f313c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1a0380 con 0x7f313c0fef70 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1a04c0 con 0x7f313c100290 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.499+0000 7f314321c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1a0650 con 0x7f313c0ff8d0 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f3140fb8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0fef70 0x7f313c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:37.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f313bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0ff8d0 0x7f313c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f3140fb8700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0fef70 0x7f313c198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54604/0 (socket says 172.21.15.49:54604) 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f3140fb8700 1 -- 172.21.15.49:0/3317575908 learned_addr learned my addr 172.21.15.49:0/3317575908 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f31417b9700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313c100290 0x7f313c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f313bfff700 1 -- 172.21.15.49:0/3317575908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313c100290 msgr2=0x7f313c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f313bfff700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313c100290 0x7f313c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f313bfff700 1 -- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0fef70 msgr2=0x7f313c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f313bfff700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0fef70 0x7f313c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f313bfff700 1 -- 172.21.15.49:0/3317575908 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128009520 con 0x7f313c0ff8d0 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f3140fb8700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0fef70 0x7f313c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f313bfff700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0ff8d0 0x7f313c198d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9f0 tx=0x7f312800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:37.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f3139ffb700 1 -- 172.21.15.49:0/3317575908 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800cb70 con 0x7f313c0ff8d0 2023-10-08T09:31:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f3139ffb700 1 -- 172.21.15.49:0/3317575908 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f312800ee90 con 0x7f313c0ff8d0 2023-10-08T09:31:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3134009d00 con 0x7f313c0ff8d0 2023-10-08T09:31:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.500+0000 7f3139ffb700 1 -- 172.21.15.49:0/3317575908 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128003cf0 con 0x7f313c0ff8d0 2023-10-08T09:31:37.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.501+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3130009520 con 0x7f313c0ff8d0 2023-10-08T09:31:37.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.501+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c109fd0 con 0x7f313c0ff8d0 2023-10-08T09:31:37.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.503+0000 7f3139ffb700 1 -- 172.21.15.49:0/3317575908 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f312800ccd0 con 0x7f313c0ff8d0 2023-10-08T09:31:37.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.503+0000 7f3139ffb700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f312c0a1480 0x7f312c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:37.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.503+0000 7f3139ffb700 1 -- 172.21.15.49:0/3317575908 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3128014070 con 0x7f313c0ff8d0 2023-10-08T09:31:37.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.503+0000 7f3140fb8700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f312c0a1480 0x7f312c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:37.505 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.504+0000 7f3140fb8700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f312c0a1480 0x7f312c0a3940 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3130009fd0 tx=0x7f3130019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:37.508 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.508+0000 7f3139ffb700 1 -- 172.21.15.49:0/3317575908 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3128096a00 con 0x7f313c0ff8d0 2023-10-08T09:31:37.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:37 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1879615380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:31:37.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:37 smithi049 ceph-mon[99485]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-10-08T09:31:37.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:37 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/108169880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:31:37.715 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.714+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f313c067450 con 0x7f313c0ff8d0 2023-10-08T09:31:37.716 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.716+0000 7f3139ffb700 1 -- 172.21.15.49:0/3317575908 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f312809a020 con 0x7f313c0ff8d0 2023-10-08T09:31:37.717 INFO:teuthology.orchestra.run.smithi049.stdout: 2023-10-08T09:31:37.717 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":90,"fsid":"b8694e94-65b9-11ee-8db5-212e2dc638e7","created":"2023-10-08T09:05:34.760723+0000","modified":"2023-10-08T09:14:38.423693+0000","last_up_change":"2023-10-08T09:12:19.933373+0000","last_in_change":"2023-10-08T09:12:02.612518+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T09:09:10.311953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T09:13:01.183216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T09:13:02.157813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T09:13:06.277641+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"78f3f71f-7b0b-4d29-997d-3bca2fd04fd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6803","nonce":3366428162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6805","nonce":3366428162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6809","nonce":3366428162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":3366428162},{"type":"v1","addr":"172.21.15.49:6807","nonce":3366428162}]},"public_addr":"172.21.15.49:6803/3366428162","cluster_addr":"172.21.15.49:6805/3366428162","heartbeat_back_addr":"172.21.15.49:6809/3366428162","heartbeat_front_addr":"172.21.15.49:6807/3366428162","state":["exists","up"]},{"osd":1,"uuid":"7f3cdae9-b298-43b7-a0d0-703f6f7ab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6801","nonce":2026160799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6803","nonce":2026160799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6807","nonce":2026160799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2026160799},{"type":"v1","addr":"172.21.15.132:6805","nonce":2026160799}]},"public_addr":"172.21.15.132:6801/2026160799","cluster_addr":"172.21.15.132:6803/2026160799","heartbeat_back_addr":"172.21.15.132:6807/2026160799","heartbeat_front_addr":"172.21.15.132:6805/2026160799","state":["exists","up"]},{"osd":2,"uuid":"95cbb767-d840-4487-8b70-2f8381136ef7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6801","nonce":2213406776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6803","nonce":2213406776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6807","nonce":2213406776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":2213406776},{"type":"v1","addr":"172.21.15.184:6805","nonce":2213406776}]},"public_addr":"172.21.15.184:6801/2213406776","cluster_addr":"172.21.15.184:6803/2213406776","heartbeat_back_addr":"172.21.15.184:6807/2213406776","heartbeat_front_addr":"172.21.15.184:6805/2213406776","state":["exists","up"]},{"osd":3,"uuid":"cc9fe096-1fd1-4de3-b21c-58a18d6cbec6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6811","nonce":3100042133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6813","nonce":3100042133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6817","nonce":3100042133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":3100042133},{"type":"v1","addr":"172.21.15.49:6815","nonce":3100042133}]},"public_addr":"172.21.15.49:6811/3100042133","cluster_addr":"172.21.15.49:6813/3100042133","heartbeat_back_addr":"172.21.15.49:6817/3100042133","heartbeat_front_addr":"172.21.15.49:6815/3100042133","state":["exists","up"]},{"osd":4,"uuid":"e852c5d9-cf88-4ddf-98e7-e22d644b4a30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6809","nonce":2969628103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6811","nonce":2969628103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6815","nonce":2969628103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":2969628103},{"type":"v1","addr":"172.21.15.132:6813","nonce":2969628103}]},"public_addr":"172.21.15.132:6809/2969628103","cluster_addr":"172.21.15.132:6811/2969628103","heartbeat_back_addr":"172.21.15.132:6815/2969628103","heartbeat_front_addr":"172.21.15.132:6813/2969628103","state":["exists","up"]},{"osd":5,"uuid":"95803835-c3f8-469c-bee8-95249eccbe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6809","nonce":3484166138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6811","nonce":3484166138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6815","nonce":3484166138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":3484166138},{"type":"v1","addr":"172.21.15.184:6813","nonce":3484166138}]},"public_addr":"172.21.15.184:6809/3484166138","cluster_addr":"172.21.15.184:6811/3484166138","heartbeat_back_addr":"172.21.15.184:6815/3484166138","heartbeat_front_addr":"172.21.15.184:6813/3484166138","state":["exists","up"]},{"osd":6,"uuid":"811a7eaa-3df2-45a3-9cf3-b6a559db1546","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6819","nonce":3113806039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6821","nonce":3113806039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6825","nonce":3113806039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":3113806039},{"type":"v1","addr":"172.21.15.49:6823","nonce":3113806039}]},"public_addr":"172.21.15.49:6819/3113806039","cluster_addr":"172.21.15.49:6821/3113806039","heartbeat_back_addr":"172.21.15.49:6825/3113806039","heartbeat_front_addr":"172.21.15.49:6823/3113806039","state":["exists","up"]},{"osd":7,"uuid":"d84fc995-e0a5-4655-831b-cacfceee4afa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6817","nonce":1427224102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6819","nonce":1427224102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6823","nonce":1427224102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1427224102},{"type":"v1","addr":"172.21.15.132:6821","nonce":1427224102}]},"public_addr":"172.21.15.132:6817/1427224102","cluster_addr":"172.21.15.132:6819/1427224102","heartbeat_back_addr":"172.21.15.132:6823/1427224102","heartbeat_front_addr":"172.21.15.132:6821/1427224102","state":["exists","up"]},{"osd":8,"uuid":"ef5ef15c-b1fb-4eaf-946f-e4bdfae4bd42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6817","nonce":936308582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6819","nonce":936308582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6823","nonce":936308582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":936308582},{"type":"v1","addr":"172.21.15.184:6821","nonce":936308582}]},"public_addr":"172.21.15.184:6817/936308582","cluster_addr":"172.21.15.184:6819/936308582","heartbeat_back_addr":"172.21.15.184:6823/936308582","heartbeat_front_addr":"172.21.15.184:6821/936308582","state":["exists","up"]},{"osd":9,"uuid":"b1a0dc66-f039-4112-93d3-73f6f3a6869e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6827","nonce":2343500504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6829","nonce":2343500504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6833","nonce":2343500504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":2343500504},{"type":"v1","addr":"172.21.15.49:6831","nonce":2343500504}]},"public_addr":"172.21.15.49:6827/2343500504","cluster_addr":"172.21.15.49:6829/2343500504","heartbeat_back_addr":"172.21.15.49:6833/2343500504","heartbeat_front_addr":"172.21.15.49:6831/2343500504","state":["exists","up"]},{"osd":10,"uuid":"9f90e2ab-e7e8-42b3-966b-7c0d40a89925","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6825","nonce":3556600534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6827","nonce":3556600534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6830","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6831","nonce":3556600534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6828","nonce":3556600534},{"type":"v1","addr":"172.21.15.132:6829","nonce":3556600534}]},"public_addr":"172.21.15.132:6825/3556600534","cluster_addr":"172.21.15.132:6827/3556600534","heartbeat_back_addr":"172.21.15.132:6831/3556600534","heartbeat_front_addr":"172.21.15.132:6829/3556600534","state":["exists","up"]},{"osd":11,"uuid":"4139ccc6-3973-496b-828c-069a9a0735fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6825","nonce":613863047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6827","nonce":613863047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6831","nonce":613863047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":613863047},{"type":"v1","addr":"172.21.15.184:6829","nonce":613863047}]},"public_addr":"172.21.15.184:6825/613863047","cluster_addr":"172.21.15.184:6827/613863047","heartbeat_back_addr":"172.21.15.184:6831/613863047","heartbeat_front_addr":"172.21.15.184:6829/613863047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:26.073691+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:08:47.162103+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:08.012268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:29.175745+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:09:50.231502+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:11.013470+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:32.513338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:10:53.548674+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:14.208445+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:36.206550+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:11:57.415395+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T09:12:18.272140+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.49:0/2700533077":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/656654162":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/4176953235":"2023-10-09T09:06:35.771578+0000","172.21.15.49:6800/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/890731290":"2023-10-09T09:06:35.771578+0000","172.21.15.49:0/1281967666":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/1812466550":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6801/907336252":"2023-10-09T09:06:08.179468+0000","172.21.15.49:0/2211252163":"2023-10-09T09:06:08.179468+0000","172.21.15.49:6800/890731290":"2023-10-09T09:06:35.771578+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T09:31:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f312c0a1480 msgr2=0x7f312c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f312c0a1480 0x7f312c0a3940 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3130009fd0 tx=0x7f3130019040 comp rx=0 tx=0).stop 2023-10-08T09:31:37.719 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0ff8d0 msgr2=0x7f313c198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0ff8d0 0x7f313c198d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9f0 tx=0x7f312800ed00 comp rx=0 tx=0).stop 2023-10-08T09:31:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 shutdown_connections 2023-10-08T09:31:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f312c0a1480 0x7f312c0a3940 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f313c100290 0x7f313c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f313c0fef70 0x7f313c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.720 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 --2- 172.21.15.49:0/3317575908 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f313c0ff8d0 0x7f313c198d90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 >> 172.21.15.49:0/3317575908 conn(0x7f313c0fab10 msgr2=0x7f313c0fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:37.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 shutdown_connections 2023-10-08T09:31:37.721 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.718+0000 7f314321c700 1 -- 172.21.15.49:0/3317575908 wait complete. 2023-10-08T09:31:37.734 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T09:31:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:37 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1879615380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:31:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:37 smithi184 ceph-mon[103675]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-10-08T09:31:37.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:37 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/108169880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:31:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:37 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1879615380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:31:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:37 smithi132 ceph-mon[103389]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-10-08T09:31:37.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:37 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/108169880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-08T09:31:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 -- 172.21.15.49:0/1383578778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c103b60 msgr2=0x7f2e1c105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 --2- 172.21.15.49:0/1383578778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c103b60 0x7f2e1c105f50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009990 tx=0x7f2e10009ca0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 -- 172.21.15.49:0/1383578778 shutdown_connections 2023-10-08T09:31:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 --2- 172.21.15.49:0/1383578778 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c106490 0x7f2e1c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 --2- 172.21.15.49:0/1383578778 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c101230 0x7f2e1c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 --2- 172.21.15.49:0/1383578778 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c103b60 0x7f2e1c105f50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:37.873 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 -- 172.21.15.49:0/1383578778 >> 172.21.15.49:0/1383578778 conn(0x7f2e1c0fac40 msgr2=0x7f2e1c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 -- 172.21.15.49:0/1383578778 shutdown_connections 2023-10-08T09:31:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.872+0000 7f2e2114c700 1 -- 172.21.15.49:0/1383578778 wait complete. 2023-10-08T09:31:37.874 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 Processor -- start 2023-10-08T09:31:37.876 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 -- start start 2023-10-08T09:31:38.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c101230 0x7f2e1c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c103b60 0x7f2e1c0ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c106490 0x7f2e1c0ff720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.032 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0ffec0 con 0x7f2e1c106490 2023-10-08T09:31:38.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c100030 con 0x7f2e1c101230 2023-10-08T09:31:38.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e2114c700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c1a0740 con 0x7f2e1c103b60 2023-10-08T09:31:38.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e1b59e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c106490 0x7f2e1c0ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e1ad9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c101230 0x7f2e1c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e1b59e700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c106490 0x7f2e1c0ff720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54620/0 (socket says 172.21.15.49:54620) 2023-10-08T09:31:38.033 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e1b59e700 1 -- 172.21.15.49:0/3339065935 learned_addr learned my addr 172.21.15.49:0/3339065935 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:38.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.873+0000 7f2e1a59c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c103b60 0x7f2e1c0ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1a59c700 1 -- 172.21.15.49:0/3339065935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c101230 msgr2=0x7f2e1c100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:38.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1a59c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c101230 0x7f2e1c100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1a59c700 1 -- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c106490 msgr2=0x7f2e1c0ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:38.034 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1a59c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c106490 0x7f2e1c0ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1a59c700 1 -- 172.21.15.49:0/3339065935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100095f0 con 0x7f2e1c103b60 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1ad9d700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c101230 0x7f2e1c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1b59e700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c106490 0x7f2e1c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e1a59c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c103b60 0x7f2e1c0ff1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e10000c00 tx=0x7f2e10004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e03fff700 1 -- 172.21.15.49:0/3339065935 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1001d070 con 0x7f2e1c103b60 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c1a0970 con 0x7f2e1c103b60 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e03fff700 1 -- 172.21.15.49:0/3339065935 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e10021410 con 0x7f2e1c103b60 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e03fff700 1 -- 172.21.15.49:0/3339065935 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e10021ea0 con 0x7f2e1c103b60 2023-10-08T09:31:38.035 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.874+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c1a0d90 con 0x7f2e1c103b60 2023-10-08T09:31:38.036 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.875+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c0fc7c0 con 0x7f2e1c103b60 2023-10-08T09:31:38.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.876+0000 7f2e03fff700 1 -- 172.21.15.49:0/3339065935 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2e10021580 con 0x7f2e1c103b60 2023-10-08T09:31:38.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.877+0000 7f2e03fff700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e040a1480 0x7f2e040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.037 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.877+0000 7f2e03fff700 1 -- 172.21.15.49:0/3339065935 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2e100ce5e0 con 0x7f2e1c103b60 2023-10-08T09:31:38.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.877+0000 7f2e1ad9d700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e040a1480 0x7f2e040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.877+0000 7f2e1ad9d700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e040a1480 0x7f2e040a3940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011490 tx=0x7f2e0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:38.038 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:37.881+0000 7f2e03fff700 1 -- 172.21.15.49:0/3339065935 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2e100979a0 con 0x7f2e1c103b60 2023-10-08T09:31:38.134 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.134+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2e1c10c0a0 con 0x7f2e1c103b60 2023-10-08T09:31:38.142 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.142+0000 7f2e03fff700 1 -- 172.21.15.49:0/3339065935 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2e1009afc0 con 0x7f2e1c103b60 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e040a1480 msgr2=0x7f2e040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e040a1480 0x7f2e040a3940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c011490 tx=0x7f2e0c009260 comp rx=0 tx=0).stop 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c103b60 msgr2=0x7f2e1c0ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c103b60 0x7f2e1c0ff1e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e10000c00 tx=0x7f2e10004730 comp rx=0 tx=0).stop 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 shutdown_connections 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f2e040a1480 0x7f2e040a3940 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2e1c106490 0x7f2e1c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.147 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e1c101230 0x7f2e1c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 --2- 172.21.15.49:0/3339065935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e1c103b60 0x7f2e1c0ff1e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 >> 172.21.15.49:0/3339065935 conn(0x7f2e1c0fac40 msgr2=0x7f2e1c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:38.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 shutdown_connections 2023-10-08T09:31:38.148 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.146+0000 7f2e2114c700 1 -- 172.21.15.49:0/3339065935 wait complete. 2023-10-08T09:31:38.148 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T09:31:38.161 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T09:31:38.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 -- 172.21.15.49:0/562582574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc101230 msgr2=0x7feebc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:38.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 --2- 172.21.15.49:0/562582574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc101230 0x7feebc107ae0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7feeb400b0d0 tx=0x7feeb400b3e0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 -- 172.21.15.49:0/562582574 shutdown_connections 2023-10-08T09:31:38.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 --2- 172.21.15.49:0/562582574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc101230 0x7feebc107ae0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 --2- 172.21.15.49:0/562582574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feebc0ff6a0 0x7feebc0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 --2- 172.21.15.49:0/562582574 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc100890 0x7feebc100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 -- 172.21.15.49:0/562582574 >> 172.21.15.49:0/562582574 conn(0x7feebc0fac10 msgr2=0x7feebc0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:38.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 -- 172.21.15.49:0/562582574 shutdown_connections 2023-10-08T09:31:38.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.297+0000 7feec0e94700 1 -- 172.21.15.49:0/562582574 wait complete. 2023-10-08T09:31:38.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 Processor -- start 2023-10-08T09:31:38.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 -- start start 2023-10-08T09:31:38.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc0ff6a0 0x7feebc196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc100890 0x7feebc196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feebc101230 0x7feebc19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc19e3b0 con 0x7feebc100890 2023-10-08T09:31:38.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc19e4f0 con 0x7feebc101230 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feec0e94700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc19e680 con 0x7feebc0ff6a0 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feeb9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc100890 0x7feebc196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feeb9d9b700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc100890 0x7feebc196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54658/0 (socket says 172.21.15.49:54658) 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feeb9d9b700 1 -- 172.21.15.49:0/1095445618 learned_addr learned my addr 172.21.15.49:0/1095445618 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.298+0000 7feeba59c700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc0ff6a0 0x7feebc196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feebad9d700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feebc101230 0x7feebc19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeba59c700 1 -- 172.21.15.49:0/1095445618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feebc101230 msgr2=0x7feebc19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeba59c700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feebc101230 0x7feebc19dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeba59c700 1 -- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc100890 msgr2=0x7feebc196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeba59c700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc100890 0x7feebc196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:38.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeba59c700 1 -- 172.21.15.49:0/1095445618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb4009d00 con 0x7feebc0ff6a0 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeb9d9b700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc100890 0x7feebc196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeba59c700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc0ff6a0 0x7feebc196880 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e9c0 tx=0x7feeac00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeab7fe700 1 -- 172.21.15.49:0/1095445618 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00cbd0 con 0x7feebc0ff6a0 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeab7fe700 1 -- 172.21.15.49:0/1095445618 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feeac00eeb0 con 0x7feebc0ff6a0 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc19e960 con 0x7feebc0ff6a0 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feeab7fe700 1 -- 172.21.15.49:0/1095445618 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac003c70 con 0x7feebc0ff6a0 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.299+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc19eec0 con 0x7feebc0ff6a0 2023-10-08T09:31:38.303 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.300+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc067530 con 0x7feebc0ff6a0 2023-10-08T09:31:38.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.302+0000 7feeab7fe700 1 -- 172.21.15.49:0/1095445618 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7feeac003dd0 con 0x7feebc0ff6a0 2023-10-08T09:31:38.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.302+0000 7feeab7fe700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feea40a1500 0x7feea40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:38.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.302+0000 7feeab7fe700 1 -- 172.21.15.49:0/1095445618 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9479+0+0 (secure 0 0 0) 0x7feeac014070 con 0x7feebc0ff6a0 2023-10-08T09:31:38.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.302+0000 7feeb9d9b700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feea40a1500 0x7feea40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:38.304 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.303+0000 7feeb9d9b700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feea40a1500 0x7feea40a39c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feeb0005950 tx=0x7feeb000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:38.311 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.311+0000 7feeab7fe700 1 -- 172.21.15.49:0/1095445618 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7feeac0965f0 con 0x7feebc0ff6a0 2023-10-08T09:31:38.447 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-a[155168]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.447 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-a[155168]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.447 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mds[155192]: -- 172.21.15.49:0/2355703767 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555b4faf71e0 con 0x555b4faccc00 2023-10-08T09:31:38.447 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-a[155168]: starting mds.a at 2023-10-08T09:31:38.447 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g[154775]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g[154775]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g[154775]: starting mds.g at 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g[154775]: 2023-10-08T09:31:38.218+0000 7f1ea24eb700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6838/0: (98) Address already in use 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-a[99461]: 2023-10-08T09:31:38.136+0000 7f023fda8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3317575908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3339065935' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T09:31:38.448 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T09:31:38.449 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T09:31:38.449 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T09:31:38.449 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:38 smithi049 ceph-mon[99485]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-08T09:31:38.498 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-c[141603]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.498 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-c[141603]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.498 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mds[141627]: -- 172.21.15.184:0/1560720384 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653129771e0 con 0x56531294d000 2023-10-08T09:31:38.498 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-c[141603]: starting mds.c at 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-f[141266]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-f[141266]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mds[141290]: -- 172.21.15.184:0/491310138 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562bb08e71e0 con 0x562bb08bc800 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-f[141266]: starting mds.f at 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-i[141944]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-i[141944]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mds[141969]: -- 172.21.15.184:0/3712509854 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aa2f3ef1e0 con 0x55aa2f3c4800 2023-10-08T09:31:38.499 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-i[141944]: starting mds.i at 2023-10-08T09:31:38.500 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-l[140926]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.500 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-l[140926]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.500 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mds[140951]: -- 172.21.15.184:0/2523841118 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5aac371e0 con 0x55b5aac0cc00 2023-10-08T09:31:38.500 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-l[140926]: starting mds.l at 2023-10-08T09:31:38.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3317575908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:31:38.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3339065935' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T09:31:38.500 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T09:31:38.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T09:31:38.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T09:31:38.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T09:31:38.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T09:31:38.501 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:38 smithi184 ceph-mon[103675]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-08T09:31:38.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mds[139447]: -- 172.21.15.132:0/689603020 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5598f05931e0 con 0x5598f0569000 2023-10-08T09:31:38.547 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: starting mds.b at 2023-10-08T09:31:38.548 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-e[138745]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.548 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-e[138745]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.548 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mds[138769]: -- 172.21.15.132:0/2727384176 <== mon.1 v2:172.21.15.184:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a8e90d1e0 con 0x555a8e8e3000 2023-10-08T09:31:38.548 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-e[138745]: starting mds.e at 2023-10-08T09:31:38.548 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-h[139084]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.548 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-h[139084]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.549 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mds[139108]: -- 172.21.15.132:0/329542778 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dc7739b1e0 con 0x55dc77370800 2023-10-08T09:31:38.549 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-h[139084]: starting mds.h at 2023-10-08T09:31:38.549 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-k[138409]: ignoring --setuser ceph since I am not root 2023-10-08T09:31:38.549 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-k[138409]: ignoring --setgroup ceph since I am not root 2023-10-08T09:31:38.549 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mds[138433]: -- 172.21.15.132:0/3967769319 <== mon.2 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5605f72931e0 con 0x5605f7268c00 2023-10-08T09:31:38.549 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-k[138409]: starting mds.k at 2023-10-08T09:31:38.549 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3317575908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-08T09:31:38.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3339065935' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T09:31:38.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-08T09:31:38.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-08T09:31:38.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T09:31:38.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-08T09:31:38.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: osdmap e91: 12 total, 12 up, 12 in 2023-10-08T09:31:38.550 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:38 smithi132 ceph-mon[103389]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-08T09:31:38.575 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:38.575+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feebc10af00 con 0x7feebc0ff6a0 2023-10-08T09:31:39.143 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.142+0000 7feeab7fe700 1 -- 172.21.15.49:0/1095445618 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7feebc10af00 con 0x7feebc0ff6a0 2023-10-08T09:31:39.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.149+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feea40a1500 msgr2=0x7feea40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:39.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.149+0000 7feec0e94700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feea40a1500 0x7feea40a39c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feeb0005950 tx=0x7feeb000b500 comp rx=0 tx=0).stop 2023-10-08T09:31:39.150 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.149+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc0ff6a0 msgr2=0x7feebc196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.149+0000 7feec0e94700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc0ff6a0 0x7feebc196880 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e9c0 tx=0x7feeac00ecd0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 shutdown_connections 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7feea40a1500 0x7feea40a39c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feebc0ff6a0 0x7feebc196880 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feebc101230 0x7feebc19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 --2- 172.21.15.49:0/1095445618 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feebc100890 0x7feebc196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 >> 172.21.15.49:0/1095445618 conn(0x7feebc0fac10 msgr2=0x7feebc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 shutdown_connections 2023-10-08T09:31:39.151 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.150+0000 7feec0e94700 1 -- 172.21.15.49:0/1095445618 wait complete. 2023-10-08T09:31:39.164 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T09:31:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 -- 172.21.15.49:0/282623660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6694100890 msgr2=0x7f6694100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 --2- 172.21.15.49:0/282623660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6694100890 0x7f6694100cf0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f66840099c0 tx=0x7f6684009cd0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 -- 172.21.15.49:0/282623660 shutdown_connections 2023-10-08T09:31:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 --2- 172.21.15.49:0/282623660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6694101230 0x7f6694107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 --2- 172.21.15.49:0/282623660 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66940ff6a0 0x7f66940ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 --2- 172.21.15.49:0/282623660 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6694100890 0x7f6694100cf0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.298 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 -- 172.21.15.49:0/282623660 >> 172.21.15.49:0/282623660 conn(0x7f66940fac10 msgr2=0x7f66940fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 -- 172.21.15.49:0/282623660 shutdown_connections 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.295+0000 7f669bab7700 1 -- 172.21.15.49:0/282623660 wait complete. 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 Processor -- start 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 -- start start 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66940ff6a0 0x7f6694196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6694100890 0x7f6694196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6694101230 0x7f669419dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669419e290 con 0x7f66940ff6a0 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669419e3d0 con 0x7f6694100890 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669bab7700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669419e560 con 0x7f6694101230 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f6699853700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66940ff6a0 0x7f6694196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:39.299 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f6699853700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66940ff6a0 0x7f6694196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54672/0 (socket says 172.21.15.49:54672) 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f6699853700 1 -- 172.21.15.49:0/3802358297 learned_addr learned my addr 172.21.15.49:0/3802358297 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f6699052700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6694100890 0x7f6694196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.296+0000 7f669a054700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6694101230 0x7f669419dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f6699853700 1 -- 172.21.15.49:0/3802358297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6694101230 msgr2=0x7f669419dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f6699853700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6694101230 0x7f669419dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f6699853700 1 -- 172.21.15.49:0/3802358297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6694100890 msgr2=0x7f6694196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f6699853700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6694100890 0x7f6694196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f6699853700 1 -- 172.21.15.49:0/3802358297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840095f0 con 0x7f66940ff6a0 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f6699853700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66940ff6a0 0x7f6694196760 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f669000d980 tx=0x7f669000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f668affd700 1 -- 172.21.15.49:0/3802358297 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690004020 con 0x7f66940ff6a0 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f668affd700 1 -- 172.21.15.49:0/3802358297 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f669000f690 con 0x7f66940ff6a0 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669419e840 con 0x7f66940ff6a0 2023-10-08T09:31:39.300 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f668affd700 1 -- 172.21.15.49:0/3802358297 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690003a60 con 0x7f66940ff6a0 2023-10-08T09:31:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.297+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669419eda0 con 0x7f66940ff6a0 2023-10-08T09:31:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.298+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6694067530 con 0x7f66940ff6a0 2023-10-08T09:31:39.301 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.299+0000 7f668affd700 1 -- 172.21.15.49:0/3802358297 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f669000fb00 con 0x7f66940ff6a0 2023-10-08T09:31:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.300+0000 7f668affd700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66800a1500 0x7f66800a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.300+0000 7f6699052700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66800a1500 0x7f66800a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.300+0000 7f668affd700 1 -- 172.21.15.49:0/3802358297 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9479+0+0 (secure 0 0 0) 0x7f66900cd090 con 0x7f66940ff6a0 2023-10-08T09:31:39.302 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.301+0000 7f6699052700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66800a1500 0x7f66800a39c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6684009990 tx=0x7f6684005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:39.305 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.304+0000 7f668affd700 1 -- 172.21.15.49:0/3802358297 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6690096290 con 0x7f66940ff6a0 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1095445618' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: Cluster is now healthy 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:31:39.452 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6832/3979438948,v1:172.21.15.184:6833/3979438948] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6838/2350782736,v1:172.21.15.184:6839/2350782736] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6834/1244272754,v1:172.21.15.184:6835/1244272754] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.184:6836/3982208370,v1:172.21.15.184:6837/3982208370] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.49:6839/480620954,v1:172.21.15.49:6841/480620954] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6833/3269507946,v1:172.21.15.132:6835/3269507946] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6836/2415633749,v1:172.21.15.132:6838/2415633749] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6837/2935404060,v1:172.21.15.132:6839/2935404060] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.49:6838/2234447919,v1:172.21.15.49:6840/2234447919] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: mds.? [v2:172.21.15.132:6832/769148375,v1:172.21.15.132:6834/769148375] up:boot 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: fsmap 12 up:standby 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:31:39.453 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:31:39.454 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:39 smithi049 ceph-mon[99485]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:31:39.513 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:39.513+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f669410aeb0 con 0x7f66940ff6a0 2023-10-08T09:31:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1095445618' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:31:39.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: Cluster is now healthy 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6832/3979438948,v1:172.21.15.184:6833/3979438948] up:boot 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6838/2350782736,v1:172.21.15.184:6839/2350782736] up:boot 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6834/1244272754,v1:172.21.15.184:6835/1244272754] up:boot 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.184:6836/3982208370,v1:172.21.15.184:6837/3982208370] up:boot 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.49:6839/480620954,v1:172.21.15.49:6841/480620954] up:boot 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6833/3269507946,v1:172.21.15.132:6835/3269507946] up:boot 2023-10-08T09:31:39.748 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6836/2415633749,v1:172.21.15.132:6838/2415633749] up:boot 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6837/2935404060,v1:172.21.15.132:6839/2935404060] up:boot 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.49:6838/2234447919,v1:172.21.15.49:6840/2234447919] up:boot 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: mds.? [v2:172.21.15.132:6832/769148375,v1:172.21.15.132:6834/769148375] up:boot 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: fsmap 12 up:standby 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:31:39.749 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:31:39.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:31:39.750 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:39 smithi184 ceph-mon[103675]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:31:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1095445618' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:31:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-08T09:31:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2023-10-08T09:31:39.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: Cluster is now healthy 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6832/3979438948,v1:172.21.15.184:6833/3979438948] up:boot 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6838/2350782736,v1:172.21.15.184:6839/2350782736] up:boot 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6834/1244272754,v1:172.21.15.184:6835/1244272754] up:boot 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.184:6836/3982208370,v1:172.21.15.184:6837/3982208370] up:boot 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.49:6839/480620954,v1:172.21.15.49:6841/480620954] up:boot 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6833/3269507946,v1:172.21.15.132:6835/3269507946] up:boot 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6836/2415633749,v1:172.21.15.132:6838/2415633749] up:boot 2023-10-08T09:31:39.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6837/2935404060,v1:172.21.15.132:6839/2935404060] up:boot 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.49:6838/2234447919,v1:172.21.15.49:6840/2234447919] up:boot 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: mds.? [v2:172.21.15.132:6832/769148375,v1:172.21.15.132:6834/769148375] up:boot 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: fsmap 12 up:standby 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-08T09:31:39.799 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-08T09:31:39.800 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:39 smithi132 ceph-mon[103389]: from='mgr.14152 172.21.15.49:0/4159119078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-08T09:31:40.325 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.325+0000 7f668affd700 1 -- 172.21.15.49:0/3802358297 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7f66900998b0 con 0x7f66940ff6a0 2023-10-08T09:31:40.330 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66800a1500 msgr2=0x7f66800a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66800a1500 0x7f66800a39c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6684009990 tx=0x7f6684005ad0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66940ff6a0 msgr2=0x7f6694196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66940ff6a0 0x7f6694196760 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f669000d980 tx=0x7f669000dc90 comp rx=0 tx=0).stop 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 shutdown_connections 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f66800a1500 0x7f66800a39c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6694101230 0x7f669419dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f66940ff6a0 0x7f6694196760 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 --2- 172.21.15.49:0/3802358297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6694100890 0x7f6694196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.329+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 >> 172.21.15.49:0/3802358297 conn(0x7f66940fac10 msgr2=0x7f6694067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.330+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 shutdown_connections 2023-10-08T09:31:40.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.330+0000 7f669bab7700 1 -- 172.21.15.49:0/3802358297 wait complete. 2023-10-08T09:31:40.332 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2023-10-08T09:31:40.347 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T09:31:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 -- 172.21.15.49:0/3798835238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f801007c0 msgr2=0x7f0f80100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/3798835238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f801007c0 0x7f0f80100be0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0f700099c0 tx=0x7f0f70009cd0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 -- 172.21.15.49:0/3798835238 shutdown_connections 2023-10-08T09:31:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/3798835238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f80102350 0x7f0f80108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/3798835238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f801007c0 0x7f0f80100be0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/3798835238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f801019b0 0x7f0f80101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 -- 172.21.15.49:0/3798835238 >> 172.21.15.49:0/3798835238 conn(0x7f0f800fbd30 msgr2=0x7f0f800fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 -- 172.21.15.49:0/3798835238 shutdown_connections 2023-10-08T09:31:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.484+0000 7f0f84bc1700 1 -- 172.21.15.49:0/3798835238 wait complete. 2023-10-08T09:31:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 Processor -- start 2023-10-08T09:31:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 -- start start 2023-10-08T09:31:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f801007c0 0x7f0f80198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f801019b0 0x7f0f80198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102350 0x7f0f8019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801a0430 con 0x7f0f80102350 2023-10-08T09:31:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801a0570 con 0x7f0f801019b0 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f84bc1700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801a0700 con 0x7f0f801007c0 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f7ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102350 0x7f0f8019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f7ed9d700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102350 0x7f0f8019fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54690/0 (socket says 172.21.15.49:54690) 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.485+0000 7f0f7ed9d700 1 -- 172.21.15.49:0/1376012828 learned_addr learned my addr 172.21.15.49:0/1376012828 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7dd9b700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f801019b0 0x7f0f80198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7e59c700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f801007c0 0x7f0f80198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7ed9d700 1 -- 172.21.15.49:0/1376012828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f801007c0 msgr2=0x7f0f80198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7ed9d700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f801007c0 0x7f0f80198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7ed9d700 1 -- 172.21.15.49:0/1376012828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f801019b0 msgr2=0x7f0f80198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7ed9d700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f801019b0 0x7f0f80198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7ed9d700 1 -- 172.21.15.49:0/1376012828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700095f0 con 0x7f0f80102350 2023-10-08T09:31:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f7ed9d700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102350 0x7f0f8019fe60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800ada0 tx=0x7f0f7800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f777fe700 1 -- 172.21.15.49:0/1376012828 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800f070 con 0x7f0f80102350 2023-10-08T09:31:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f777fe700 1 -- 172.21.15.49:0/1376012828 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f78007b30 con 0x7f0f80102350 2023-10-08T09:31:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f801a09e0 con 0x7f0f80102350 2023-10-08T09:31:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f777fe700 1 -- 172.21.15.49:0/1376012828 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78005280 con 0x7f0f80102350 2023-10-08T09:31:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.486+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f801a0ea0 con 0x7f0f80102350 2023-10-08T09:31:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.487+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f80067530 con 0x7f0f80102350 2023-10-08T09:31:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.488+0000 7f0f777fe700 1 -- 172.21.15.49:0/1376012828 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0f7800a090 con 0x7f0f80102350 2023-10-08T09:31:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.489+0000 7f0f777fe700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f6c0a1480 0x7f0f6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.489+0000 7f0f777fe700 1 -- 172.21.15.49:0/1376012828 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8983+0+0 (secure 0 0 0) 0x7f0f780ccae0 con 0x7f0f80102350 2023-10-08T09:31:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.489+0000 7f0f7e59c700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f6c0a1480 0x7f0f6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.489+0000 7f0f7e59c700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f6c0a1480 0x7f0f6c0a3940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f70003820 tx=0x7f0f70005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.492+0000 7f0f777fe700 1 -- 172.21.15.49:0/1376012828 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0f78099970 con 0x7f0f80102350 2023-10-08T09:31:40.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:40 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3802358297' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:40.695 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:40.695+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f8010bfe0 con 0x7f0f80102350 2023-10-08T09:31:40.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:40 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3802358297' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:40.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:40 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3802358297' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:41.331 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.330+0000 7f0f777fe700 1 -- 172.21.15.49:0/1376012828 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0f78099500 con 0x7f0f80102350 2023-10-08T09:31:41.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.335+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f6c0a1480 msgr2=0x7f0f6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:41.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.335+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f6c0a1480 0x7f0f6c0a3940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f70003820 tx=0x7f0f70005040 comp rx=0 tx=0).stop 2023-10-08T09:31:41.335 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.335+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102350 msgr2=0x7f0f8019fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:41.336 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.335+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102350 0x7f0f8019fe60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800ada0 tx=0x7f0f7800c330 comp rx=0 tx=0).stop 2023-10-08T09:31:41.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.337+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 shutdown_connections 2023-10-08T09:31:41.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.337+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7f0f6c0a1480 0x7f0f6c0a3940 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.337+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f801007c0 0x7f0f80198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.337+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0f80102350 0x7f0f8019fe60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.337+0000 7f0f84bc1700 1 --2- 172.21.15.49:0/1376012828 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f801019b0 0x7f0f80198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.337+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 >> 172.21.15.49:0/1376012828 conn(0x7f0f800fbd30 msgr2=0x7f0f80072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.338+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 shutdown_connections 2023-10-08T09:31:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.338+0000 7f0f84bc1700 1 -- 172.21.15.49:0/1376012828 wait complete. 2023-10-08T09:31:41.339 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2023-10-08T09:31:41.357 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-08T09:31:41.498 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 -- 172.21.15.49:0/3833585581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc68101260 msgr2=0x7fcc68107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:41.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3833585581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc68101260 0x7fcc68107b10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000b0d0 tx=0x7fcc6000b3e0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 -- 172.21.15.49:0/3833585581 shutdown_connections 2023-10-08T09:31:41.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3833585581 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc68101260 0x7fcc68107b10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3833585581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc680ff6d0 0x7fcc680ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.499 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3833585581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc681008c0 0x7fcc68100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 -- 172.21.15.49:0/3833585581 >> 172.21.15.49:0/3833585581 conn(0x7fcc680fac40 msgr2=0x7fcc680fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.492+0000 7fcc6f236700 1 -- 172.21.15.49:0/3833585581 shutdown_connections 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 -- 172.21.15.49:0/3833585581 wait complete. 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 Processor -- start 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 -- start start 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc680ff6d0 0x7fcc68069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc681008c0 0x7fcc6819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:41.500 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc68101260 0x7fcc6819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819e3b0 con 0x7fcc680ff6d0 2023-10-08T09:31:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819e520 con 0x7fcc68101260 2023-10-08T09:31:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6f236700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6819e6e0 con 0x7fcc681008c0 2023-10-08T09:31:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6cfd2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc680ff6d0 0x7fcc68069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6cfd2700 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc680ff6d0 0x7fcc68069550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:54700/0 (socket says 172.21.15.49:54700) 2023-10-08T09:31:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.493+0000 7fcc6cfd2700 1 -- 172.21.15.49:0/3875137238 learned_addr learned my addr 172.21.15.49:0/3875137238 (peer_addr_for_me v2:172.21.15.49:0/0) 2023-10-08T09:31:41.501 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc67fff700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc681008c0 0x7fcc6819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6d7d3700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc68101260 0x7fcc6819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6cfd2700 1 -- 172.21.15.49:0/3875137238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc681008c0 msgr2=0x7fcc6819b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6cfd2700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc681008c0 0x7fcc6819b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6cfd2700 1 -- 172.21.15.49:0/3875137238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc68101260 msgr2=0x7fcc6819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6cfd2700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc68101260 0x7fcc6819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6cfd2700 1 -- 172.21.15.49:0/3875137238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60009d00 con 0x7fcc680ff6d0 2023-10-08T09:31:41.502 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6d7d3700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc68101260 0x7fcc6819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6cfd2700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc680ff6d0 0x7fcc68069550 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00d980 tx=0x7fcc5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc65ffb700 1 -- 172.21.15.49:0/3875137238 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c004020 con 0x7fcc680ff6d0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc65ffb700 1 -- 172.21.15.49:0/3875137238 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc5c009730 con 0x7fcc680ff6d0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6819e960 con 0x7fcc680ff6d0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc65ffb700 1 -- 172.21.15.49:0/3875137238 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c003a80 con 0x7fcc680ff6d0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.494+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6819ee70 con 0x7fcc680ff6d0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.495+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc68106a80 con 0x7fcc680ff6d0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.496+0000 7fcc65ffb700 1 -- 172.21.15.49:0/3875137238 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcc5c0098a0 con 0x7fcc680ff6d0 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.497+0000 7fcc65ffb700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcc580a1480 0x7fcc580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T09:31:41.503 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.497+0000 7fcc65ffb700 1 -- 172.21.15.49:0/3875137238 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8424+0+0 (secure 0 0 0) 0x7fcc5c0ccad0 con 0x7fcc680ff6d0 2023-10-08T09:31:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.497+0000 7fcc67fff700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcc580a1480 0x7fcc580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T09:31:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.497+0000 7fcc67fff700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcc580a1480 0x7fcc580a3940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcc68069af0 tx=0x7fcc54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T09:31:41.504 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.501+0000 7fcc65ffb700 1 -- 172.21.15.49:0/3875137238 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcc5c096070 con 0x7fcc680ff6d0 2023-10-08T09:31:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:41 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3802358297' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T09:31:41.632 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:41 smithi049 ceph-mon[99485]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T09:31:41.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:41 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1376012828' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:41.633 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:41 smithi049 ceph-mon[99485]: pgmap v834: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:31:41.711 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:41.711+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcc6810b160 con 0x7fcc680ff6d0 2023-10-08T09:31:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:41 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3802358297' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T09:31:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:41 smithi184 ceph-mon[103675]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T09:31:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:41 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1376012828' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:41.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:41 smithi184 ceph-mon[103675]: pgmap v834: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:31:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:41 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3802358297' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T09:31:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:41 smithi132 ceph-mon[103389]: osdmap e92: 12 total, 12 up, 12 in 2023-10-08T09:31:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:41 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1376012828' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:41.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:41 smithi132 ceph-mon[103389]: pgmap v834: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-08T09:31:42.332 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.332+0000 7fcc65ffb700 1 -- 172.21.15.49:0/3875137238 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v94) v1 ==== 178+0+0 (secure 0 0 0) 0x7fcc5c099690 con 0x7fcc680ff6d0 2023-10-08T09:31:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcc580a1480 msgr2=0x7fcc580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcc580a1480 0x7fcc580a3940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcc68069af0 tx=0x7fcc54009260 comp rx=0 tx=0).stop 2023-10-08T09:31:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc680ff6d0 msgr2=0x7fcc68069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T09:31:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc680ff6d0 0x7fcc68069550 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00d980 tx=0x7fcc5c00dc90 comp rx=0 tx=0).stop 2023-10-08T09:31:42.338 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 shutdown_connections 2023-10-08T09:31:42.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:6800/1314466902,v1:172.21.15.49:6801/1314466902] conn(0x7fcc580a1480 0x7fcc580a3940 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:42.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc68101260 0x7fcc6819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:42.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcc680ff6d0 0x7fcc68069550 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:42.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 --2- 172.21.15.49:0/3875137238 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc681008c0 0x7fcc6819b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T09:31:42.339 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.337+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 >> 172.21.15.49:0/3875137238 conn(0x7fcc680fac40 msgr2=0x7fcc68103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T09:31:42.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.338+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 shutdown_connections 2023-10-08T09:31:42.340 INFO:teuthology.orchestra.run.smithi049.stderr:2023-10-08T09:31:42.339+0000 7fcc6f236700 1 -- 172.21.15.49:0/3875137238 wait complete. 2023-10-08T09:31:42.343 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data_ec' removed 2023-10-08T09:31:42.363 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:31:42.410 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:31:42.470 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:31:42.527 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-08T09:31:42.527 DEBUG:teuthology.orchestra.run.smithi049:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T09:31:42.571 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T09:31:42.594 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-08T09:31:42.630 INFO:tasks.cephadm:Stopping all daemons... 2023-10-08T09:31:42.630 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-08T09:31:42.630 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a 2023-10-08T09:31:42.637 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:42 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/1376012828' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T09:31:42.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:42 smithi049 ceph-mon[99485]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T09:31:42.638 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:42 smithi049 ceph-mon[99485]: from='client.? 172.21.15.49:0/3875137238' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:42.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:42 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/1376012828' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T09:31:42.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:42 smithi184 ceph-mon[103675]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T09:31:42.751 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:42 smithi184 ceph-mon[103675]: from='client.? 172.21.15.49:0/3875137238' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:42.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:42 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/1376012828' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-08T09:31:42.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:42 smithi132 ceph-mon[103389]: osdmap e93: 12 total, 12 up, 12 in 2023-10-08T09:31:42.798 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:42 smithi132 ceph-mon[103389]: from='client.? 172.21.15.49:0/3875137238' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-08T09:31:43.133 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:42 smithi049 systemd[1]: Stopping Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:43.463 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-a[99461]: 2023-10-08T09:31:43.148+0000 7f02455b3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:31:43.464 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-a[99461]: 2023-10-08T09:31:43.148+0000 7f02455b3700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-08T09:31:43.696 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a.service' 2023-10-08T09:31:43.721 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:43 smithi049 bash[184153]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-a 2023-10-08T09:31:43.722 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:43 smithi049 systemd[1]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.a.service: Succeeded. 2023-10-08T09:31:43.722 INFO:journalctl@ceph.mon.a.smithi049.stdout:Oct 08 09:31:43 smithi049 systemd[1]: Stopped Ceph mon.a for b8694e94-65b9-11ee-8db5-212e2dc638e7. 2023-10-08T09:31:43.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:31:43.756 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-08T09:31:43.756 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-08T09:31:43.756 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.b 2023-10-08T09:31:44.047 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:43 smithi132 systemd[1]: Stopping Ceph mon.b for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:44.537 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-b[103365]: 2023-10-08T09:31:44.211+0000 7f563c534700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:31:44.537 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-b[103365]: 2023-10-08T09:31:44.211+0000 7f563c534700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-08T09:31:44.797 INFO:journalctl@ceph.mon.b.smithi132.stdout:Oct 08 09:31:44 smithi132 bash[145053]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-b 2023-10-08T09:31:45.228 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.b.service' 2023-10-08T09:31:45.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:31:45.325 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-08T09:31:45.325 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-08T09:31:45.325 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.c 2023-10-08T09:31:45.747 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:45 smithi184 systemd[1]: Stopping Ceph mon.c for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:46.045 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-c[103651]: 2023-10-08T09:31:45.782+0000 7ff8874d0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:31:46.046 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-c[103651]: 2023-10-08T09:31:45.782+0000 7ff8874d0700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-08T09:31:46.418 INFO:journalctl@ceph.mon.c.smithi184.stdout:Oct 08 09:31:46 smithi184 bash[147869]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mon-c 2023-10-08T09:31:46.425 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mon.c.service' 2023-10-08T09:31:46.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:31:46.496 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-08T09:31:46.496 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-08T09:31:46.496 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.x 2023-10-08T09:31:46.848 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:31:46 smithi049 systemd[1]: Stopping Ceph mgr.x for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:47.130 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:31:46 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x[99732]: 2023-10-08T09:31:46.846+0000 7fe43aef2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:31:47.382 INFO:journalctl@ceph.mgr.x.smithi049.stdout:Oct 08 09:31:47 smithi049 bash[184294]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-x 2023-10-08T09:31:47.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.x.service' 2023-10-08T09:31:47.431 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:31:47.431 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-08T09:31:47.431 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-08T09:31:47.432 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.y 2023-10-08T09:31:47.797 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:31:47 smithi132 systemd[1]: Stopping Ceph mgr.y for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:48.129 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:31:48 smithi132 bash[145200]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-y 2023-10-08T09:31:48.129 INFO:journalctl@ceph.mgr.y.smithi132.stdout:Oct 08 09:31:48 smithi132 systemd[1]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-08T09:31:48.314 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.y.service' 2023-10-08T09:31:48.353 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:31:48.353 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-08T09:31:48.353 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-08T09:31:48.354 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.z 2023-10-08T09:31:48.747 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:31:48 smithi184 systemd[1]: Stopping Ceph mgr.z for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:49.066 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:31:49 smithi184 bash[148019]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mgr-z 2023-10-08T09:31:49.248 INFO:journalctl@ceph.mgr.z.smithi184.stdout:Oct 08 09:31:49 smithi184 systemd[1]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-08T09:31:49.252 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mgr.z.service' 2023-10-08T09:31:49.289 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:31:49.289 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-08T09:31:49.289 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-08T09:31:49.289 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.0 2023-10-08T09:31:49.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:31:49 smithi049 systemd[1]: Stopping Ceph osd.0 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:50.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:31:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-0[121940]: 2023-10-08T09:31:50.287+0000 7fc3bb35c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:31:50.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:31:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-0[121940]: 2023-10-08T09:31:50.287+0000 7fc3bb35c700 -1 osd.0 94 *** Got signal Terminated *** 2023-10-08T09:31:50.632 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:31:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-0[121940]: 2023-10-08T09:31:50.287+0000 7fc3bb35c700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:31:56.382 INFO:journalctl@ceph.osd.0.smithi049.stdout:Oct 08 09:31:56 smithi049 bash[184436]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-0 2023-10-08T09:31:57.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.0.service' 2023-10-08T09:31:57.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:31:57.861 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-08T09:31:57.861 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-08T09:31:57.861 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.1 2023-10-08T09:31:58.297 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:31:57 smithi132 systemd[1]: Stopping Ceph osd.1 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:31:59.047 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:31:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-1[108441]: 2023-10-08T09:31:58.731+0000 7f2663442700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:31:59.047 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:31:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-1[108441]: 2023-10-08T09:31:58.731+0000 7f2663442700 -1 osd.1 94 *** Got signal Terminated *** 2023-10-08T09:31:59.047 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:31:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-1[108441]: 2023-10-08T09:31:58.731+0000 7f2663442700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:04.797 INFO:journalctl@ceph.osd.1.smithi132.stdout:Oct 08 09:32:04 smithi132 bash[145347]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-1 2023-10-08T09:32:06.261 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.1.service' 2023-10-08T09:32:06.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:32:06.305 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-08T09:32:06.305 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-08T09:32:06.305 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.2 2023-10-08T09:32:06.671 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:32:06 smithi184 systemd[1]: Stopping Ceph osd.2 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:32:06.997 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:32:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-2[109416]: 2023-10-08T09:32:06.669+0000 7f117b7b9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:32:06.997 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:32:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-2[109416]: 2023-10-08T09:32:06.669+0000 7f117b7b9700 -1 osd.2 94 *** Got signal Terminated *** 2023-10-08T09:32:06.997 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:32:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-2[109416]: 2023-10-08T09:32:06.669+0000 7f117b7b9700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:12.748 INFO:journalctl@ceph.osd.2.smithi184.stdout:Oct 08 09:32:12 smithi184 bash[148168]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-2 2023-10-08T09:32:13.982 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.2.service' 2023-10-08T09:32:14.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:32:14.023 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-08T09:32:14.023 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-08T09:32:14.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.3 2023-10-08T09:32:14.382 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:14 smithi049 systemd[1]: Stopping Ceph osd.3 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:32:14.882 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:14 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:32:14.387+0000 7fd059b34700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:32:14.882 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:14 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:32:14.387+0000 7fd059b34700 -1 osd.3 94 *** Got signal Terminated *** 2023-10-08T09:32:14.882 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:14 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:32:14.387+0000 7fd059b34700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:16.882 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:16 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:32:16.553+0000 7fd054126700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:52.711695+0000 front 2023-10-08T09:31:52.711722+0000 (oldest deadline 2023-10-08T09:32:16.211814+0000) 2023-10-08T09:32:17.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:17.290+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:17.882 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:17 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:32:17.555+0000 7fd054126700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:52.711695+0000 front 2023-10-08T09:31:52.711722+0000 (oldest deadline 2023-10-08T09:32:16.211814+0000) 2023-10-08T09:32:18.680 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:18.327+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:18.882 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:18 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3[130017]: 2023-10-08T09:32:18.510+0000 7fd054126700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:52.711695+0000 front 2023-10-08T09:31:52.711722+0000 (oldest deadline 2023-10-08T09:32:16.211814+0000) 2023-10-08T09:32:18.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:18.680+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:19.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:18 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:18.583+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:19.682 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:19.315+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:19.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:19 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:19.539+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:19.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:19.681+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:20.269 INFO:journalctl@ceph.osd.3.smithi049.stdout:Oct 08 09:32:20 smithi049 bash[184720]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-3 2023-10-08T09:32:20.270 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:20 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:20.059+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:20.731 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:20.339+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:20.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:20.730+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:21.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:20 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:20.560+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:21.132 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:21 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:21.021+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:21.538 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.3.service' 2023-10-08T09:32:21.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:32:21.580 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-08T09:32:21.580 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-08T09:32:21.580 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.4 2023-10-08T09:32:21.730 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:21 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:21.306+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:21.939 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:21 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:21.581+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:21.939 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:32:21 smithi132 systemd[1]: Stopping Ceph osd.4 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:32:21.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:21 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:21.729+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:22.297 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:32:21 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:32:21.937+0000 7f306026d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:32:22.297 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:32:21 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:32:21.937+0000 7f306026d700 -1 osd.4 94 *** Got signal Terminated *** 2023-10-08T09:32:22.297 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:32:21 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:32:21.937+0000 7f306026d700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:22.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:22 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:22.022+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:22.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:22 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:22.348+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:23.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:22 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:22.629+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:23.247 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:22 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:22.751+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:23.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:23 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:23.017+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:23.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:23 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:23.338+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:24.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:23 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:23.674+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:24.247 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:23 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:23.758+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:24.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:24 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:24.050+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:24.727 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:24 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:24.333+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:24.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:24 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:24.726+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:25.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:24 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:24.657+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:25.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:25 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:25.052+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:25.548 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:32:25 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:32:25.206+0000 7f305a05e700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:00.297938+0000 front 2023-10-08T09:32:00.297920+0000 (oldest deadline 2023-10-08T09:32:24.397789+0000) 2023-10-08T09:32:25.685 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:25 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:25.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:25.983 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:25 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:25.685+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:25.983 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:25 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:25.685+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:26.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:25 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:25.701+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:26.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:25 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:25.629+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:26.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:25 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:25.982+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:26.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:26 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:26.087+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:26.547 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:32:26 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4[116477]: 2023-10-08T09:32:26.161+0000 7f305a05e700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:00.297938+0000 front 2023-10-08T09:32:00.297920+0000 (oldest deadline 2023-10-08T09:32:24.397789+0000) 2023-10-08T09:32:26.711 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:26 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:26.389+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:26.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:26 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:26.519+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:26.994 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:26 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:26.710+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:26.994 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:26 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:26.710+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:27.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:26 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:26.597+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:27.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:26 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:26.708+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:27.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:26 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:26.993+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:27.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:27.119+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:27.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:27.119+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:27.690 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:27 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:27.360+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:27.691 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:27 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:27.360+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:27.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:27.565+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:27.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:27 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:27.690+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:27.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:27 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:27.690+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:28.022 INFO:journalctl@ceph.osd.4.smithi132.stdout:Oct 08 09:32:27 smithi132 bash[145654]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-4 2023-10-08T09:32:28.022 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:27 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:27.619+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:28.023 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:27 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:27.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:28.366 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:28 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:28.000+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:28.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:28.121+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:28.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:28.121+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:28.687 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:28 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:28.365+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:28.687 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:28 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:28.365+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:28.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:28 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:28.528+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:28.968 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:28 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:28.570+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:28.968 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:28 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:28.747+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:28.969 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:28 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:28.747+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:28.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:28 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:28.686+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:28.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:28 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:28.686+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:29.281 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.4.service' 2023-10-08T09:32:29.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:32:29.322 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-08T09:32:29.322 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-08T09:32:29.322 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.5 2023-10-08T09:32:29.337 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:29.050+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:29.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:29.111+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:29.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:29.111+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:29.683 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:29.335+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:29.684 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:29.335+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:29.684 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:29 smithi184 systemd[1]: Stopping Ceph osd.5 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:32:29.684 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:29.682+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:29.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:29 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:29.537+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:29.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:29.682+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:29.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:29.683+0000 7f28a05a5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:32:29.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:29.683+0000 7f28a05a5700 -1 osd.5 94 *** Got signal Terminated *** 2023-10-08T09:32:29.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:29 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:29.683+0000 7f28a05a5700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:30.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:29 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:29.612+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:30.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:29 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:29.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:30.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:29 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:29.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:30.350 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:30 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:30.073+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:30.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:30 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:30.106+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:30.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:30 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:30.106+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:30.674 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:30 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:30.350+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:30.674 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:30 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:30.350+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:30.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:30 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:30.512+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:30.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:30 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:30.673+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:30.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:30 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:30.673+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:31.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:30 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:30.588+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:31.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:30 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:30.728+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:31.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:30 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:30.728+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:31.380 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:31 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:31.052+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:31.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:31.096+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:31.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:31.096+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:31.691 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:31 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:31.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:31.692 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:31 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:31.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:31.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:31.530+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:31.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:31 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:31.691+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:31.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:31 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:31.691+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:32.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:31 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:31.587+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:32.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:31 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:31.743+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:32.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:31 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:31.743+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:32.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:32 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:32.094+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:32.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:32 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:32.094+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:32.430 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:32 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:32.023+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:32.747 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:32 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:32.672+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:32.747 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:32 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:32.672+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:32.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:32 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:32.430+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:32.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:32 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:32.430+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:32.797 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:32 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:32.538+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:32.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:32 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:32.727+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:32.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:32 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:32.727+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:32.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:32 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:32.530+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:33.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:32 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:32.976+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:33.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:33 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:33.066+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:33.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:33 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:33.066+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:33.723 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:33 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:33.450+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:33.723 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:33 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:33.450+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:33.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:33 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:33.486+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:33.997 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:33 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:33.968+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:33.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:33 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:33.722+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.822868+0000 front 2023-10-08T09:31:53.822672+0000 (oldest deadline 2023-10-08T09:32:18.522677+0000) 2023-10-08T09:32:33.997 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:33 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:33.722+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.524641+0000 front 2023-10-08T09:32:02.524756+0000 (oldest deadline 2023-10-08T09:32:24.824699+0000) 2023-10-08T09:32:33.998 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:33 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5[118026]: 2023-10-08T09:32:33.722+0000 7f289ab97700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.725896+0000 front 2023-10-08T09:32:10.725927+0000 (oldest deadline 2023-10-08T09:32:33.626062+0000) 2023-10-08T09:32:34.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:33 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:33.585+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:34.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:33 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:33.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:34.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:33 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:33.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:34.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:33 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:33.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:34.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:34.041+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:34.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:34.041+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:34.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:34 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:34.442+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:34.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:34 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:34.442+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:34.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:34.525+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:34.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:34 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:34.525+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:35.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:34 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:34.631+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:35.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:34 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:34.631+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:35.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:34 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:34.784+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:35.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:34 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:34.784+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:35.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:34 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:34.784+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:35.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:34 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:34.960+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:35.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:35.088+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:35.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:35.088+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:35.563 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:35 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:35.447+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:35.563 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:35 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:35.447+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:35.563 INFO:journalctl@ceph.osd.5.smithi184.stdout:Oct 08 09:32:35 smithi184 bash[148455]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-5 2023-10-08T09:32:35.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:35.529+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:35.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:35.529+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:36.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:35 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:35.585+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:36.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:35 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:35.585+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:36.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:35 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:35.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:36.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:35 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:35.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:36.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:35 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:35.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:36.174 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:35 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:35.921+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:36.174 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:35 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:35.921+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:36.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:36 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:36.119+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:36.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:36 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:36.119+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:36.660 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:36 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:36.403+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:36.660 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:36 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:36.403+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:36.660 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:36 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:36.403+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:36.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:36 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:36.505+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:36.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:36 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:36.505+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:36.922 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:36 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:36.885+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:36.922 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:36 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:36.885+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:36.928 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.5.service' 2023-10-08T09:32:37.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:32:37.008 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-08T09:32:37.008 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-08T09:32:37.008 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.6 2023-10-08T09:32:37.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:36 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:36.602+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:37.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:36 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:36.602+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:37.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:36 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:36.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:37.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:36 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:36.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:37.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:36 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:36.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:37.381 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:37 smithi049 systemd[1]: Stopping Ceph osd.6 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:32:37.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:37.159+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:37.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:37.159+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:37.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:37.379+0000 7fbc2cffd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:32:37.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:37.379+0000 7fbc2cffd700 -1 osd.6 94 *** Got signal Terminated *** 2023-10-08T09:32:37.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:37.379+0000 7fbc2cffd700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:37.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:37.463+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:37.633 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:37 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:37.463+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:37.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:37 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:37.434+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:37.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:37 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:37.434+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:37.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:37 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:37.434+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:38.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:37 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:37.592+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:38.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:37 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:37.592+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:38.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:37 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:37.720+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:38.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:37 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:37.720+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:38.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:37 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:37.720+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:38.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:37 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:37.880+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:38.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:37 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:37.880+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:38.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:38.138+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:38.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:38.138+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:38.494 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:38.138+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:38.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:38.433+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:38.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:38.433+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:38.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:38.433+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:38.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:38.493+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:38.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:38 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:38.493+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:39.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:38.599+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:39.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:38.599+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:39.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:38.694+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:39.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:38.694+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:39.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:38 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:38.694+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:39.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:38.911+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:39.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:38 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:38.911+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:39.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:39 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:39.152+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:39.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:39 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:39.152+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:39.538 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:39 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:39.152+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:39.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:39.468+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:39.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:39.468+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:39.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:39.468+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:39.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:39 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:39.537+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:39.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:39 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:39.537+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:40.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:39 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:39.641+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:40.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:39 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:39.641+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:40.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:39 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:39.733+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:40.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:39 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:39.733+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:40.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:39 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:39.733+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:40.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:39.912+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:40.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:39 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:39.912+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:40.574 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:40 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:40.201+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:40.574 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:40 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:40.201+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:40.575 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:40 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:40.201+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:40.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:40 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:40.430+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:40.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:40 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:40.430+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:40.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:40 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:40.430+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:40.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:40 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:40.573+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:40.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:40 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:40.573+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:41.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:40 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:40.620+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:41.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:40 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:40.620+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:41.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:40 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:40.718+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:41.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:40 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:40.718+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:41.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:40 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:40.718+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:41.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:40 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:40.961+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:41.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:40 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:40.961+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:41.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:40 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:40.961+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:41.587 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:41 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:41.167+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:41.587 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:41 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:41.167+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:41.587 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:41 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:41.167+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:41.587 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:41 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:41.167+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:41.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:41.416+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:41.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:41.416+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:41.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:41.416+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:41.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:41 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:41.586+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.324521+0000 front 2023-10-08T09:32:02.324344+0000 (oldest deadline 2023-10-08T09:32:25.824402+0000) 2023-10-08T09:32:41.882 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:41 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6[138564]: 2023-10-08T09:32:41.586+0000 7fbc265ed700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.527298+0000 front 2023-10-08T09:32:08.527247+0000 (oldest deadline 2023-10-08T09:32:34.427323+0000) 2023-10-08T09:32:42.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:41.613+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:42.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:41.613+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:42.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:41.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:42.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:41.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:42.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:41 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:41.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:42.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:41.973+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:42.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:41.973+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:42.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:41 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:41.973+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:42.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:42 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:42.170+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:42.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:42 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:42.170+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:42.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:42 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:42.170+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:42.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:42 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:42.170+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:42.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:42.427+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:42.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:42.427+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:42.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:42.427+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:43.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:42.575+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:43.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:42.575+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:43.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:42.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:43.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:42.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:43.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:42.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:43.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:42.957+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:43.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:42.957+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:43.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:42 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:42.957+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:43.632 INFO:journalctl@ceph.osd.6.smithi049.stdout:Oct 08 09:32:43 smithi049 bash[184991]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-6 2023-10-08T09:32:43.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:43.220+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:43.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:43.220+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:43.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:43.220+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:43.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:43 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:43.220+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:43.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:43.418+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:43.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:43.418+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:43.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:43.418+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:44.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:43.623+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:44.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:43.623+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:44.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:43.723+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:44.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:43.723+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:44.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:43.723+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:44.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:43 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:43.723+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:44.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:43.936+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:44.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:43.936+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:44.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:43 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:43.936+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:44.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:44.203+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:44.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:44.203+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:44.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:44.203+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:44.383 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:44 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:44.203+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:44.733 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.6.service' 2023-10-08T09:32:44.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:44.413+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:44.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:44.413+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:44.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:44.413+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:44.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:32:44.775 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-08T09:32:44.775 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-08T09:32:44.775 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.7 2023-10-08T09:32:45.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:44.585+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:45.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:44.585+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:45.048 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:44 smithi132 systemd[1]: Stopping Ceph osd.7 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:32:45.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:44.704+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:45.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:44.704+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:45.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:44.704+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:45.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:44 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:44.704+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:45.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:44.936+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:45.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:44.936+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:45.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:44 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:44.936+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:45.537 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:45.126+0000 7ff8b56e9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:32:45.537 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:45.126+0000 7ff8b56e9700 -1 osd.7 94 *** Got signal Terminated *** 2023-10-08T09:32:45.537 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:45.126+0000 7ff8b56e9700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:45.538 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:45.537+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:45.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:45 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:45.245+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:45.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:45 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:45.245+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:45.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:45 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:45.245+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:45.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:45 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:45.245+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:45.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:45.461+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:45.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:45.461+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:45.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:45.461+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:45.797 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:45.537+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:45.797 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:45.537+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:16.116747+0000 front 2023-10-08T09:32:16.116837+0000 (oldest deadline 2023-10-08T09:32:45.517362+0000) 2023-10-08T09:32:45.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:45.720+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:45.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:45.720+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:45.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:45.720+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:45.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:45 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:45.720+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:46.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:45.980+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:46.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:45.980+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:46.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:45 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:45.980+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:46.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:46 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:46.239+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:46.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:46 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:46.239+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:46.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:46 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:46.239+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:46.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:46 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:46.239+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:46.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:46 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:46.488+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:46.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:46 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:46.488+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:46.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:46 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:46.488+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:46.797 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:46.516+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:46.797 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:46.516+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:46.797 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:46.516+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:16.116747+0000 front 2023-10-08T09:32:16.116837+0000 (oldest deadline 2023-10-08T09:32:45.517362+0000) 2023-10-08T09:32:46.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:46.740+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:46.797 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:46.740+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:46.798 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:46.740+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:46.798 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:46 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:46.740+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:47.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:46 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:46.986+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:47.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:46 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:46.986+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:47.247 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:46 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:46.986+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:47.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:47 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:47.277+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:47.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:47 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:47.277+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:47.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:47 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:47.277+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:47.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:47 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:47.277+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:47.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:47 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:47.498+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:47.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:47 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:47.498+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:47.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:47 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:47.498+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:48.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:47.548+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:48.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:47.548+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:48.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:47.548+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:16.116747+0000 front 2023-10-08T09:32:16.116837+0000 (oldest deadline 2023-10-08T09:32:45.517362+0000) 2023-10-08T09:32:48.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:47.783+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:48.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:47.783+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:48.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:47.783+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:48.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:47.783+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:48.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:47.783+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:48.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:48 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:48.024+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:48.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:48 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:48.024+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:48.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:48 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:48.024+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:48.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:48.315+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:48.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:48.315+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:48.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:48.315+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:48.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:48 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:48.315+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:48.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:48 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:48.535+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:48.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:48 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:48.535+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:48.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:48 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:48.535+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:49.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:48.591+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:49.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:48.591+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:49.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:48.591+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:16.116747+0000 front 2023-10-08T09:32:16.116837+0000 (oldest deadline 2023-10-08T09:32:45.517362+0000) 2023-10-08T09:32:49.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:48.772+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:49.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:48.772+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:49.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:48.772+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:49.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:48.772+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:49.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:48 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:48.772+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:49.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:49.064+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:49.498 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:49.064+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:49.498 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:49.064+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:49.498 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:49.064+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:49.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:49 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:49.325+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:49.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:49 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:49.325+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:49.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:49 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:49.325+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:49.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:49 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:49.325+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:49.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:49.498+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:49.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:49.498+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:49.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:49 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:49.498+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:49.546+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.713494+0000 front 2023-10-08T09:32:01.713535+0000 (oldest deadline 2023-10-08T09:32:25.114273+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:49.546+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.915846+0000 front 2023-10-08T09:32:10.915922+0000 (oldest deadline 2023-10-08T09:32:33.815794+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7[124569]: 2023-10-08T09:32:49.546+0000 7ff8afcdb700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:16.116747+0000 front 2023-10-08T09:32:16.116837+0000 (oldest deadline 2023-10-08T09:32:45.517362+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:49.741+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:49.741+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:49.741+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:49.741+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:50.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:49 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:49.741+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:50.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:50 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:50.094+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:50.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:50 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:50.094+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:50.473 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:50 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:50.094+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:50.474 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:50 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:50.094+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:50.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:50.307+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:50.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:50.307+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:50.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:50.307+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:50.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:50.307+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:50.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:50 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:50.307+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:50.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:50 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:50.472+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:50.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:50 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:50.472+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:50.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:50 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:50.472+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:51.007 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:50 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:50.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:51.007 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:50 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:50.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:51.007 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:50 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:50.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:51.007 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:50 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:50.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:51.007 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:50 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:50.729+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:51.297 INFO:journalctl@ceph.osd.7.smithi132.stdout:Oct 08 09:32:51 smithi132 bash[145925]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-7 2023-10-08T09:32:51.471 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:51 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:51.089+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:51.471 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:51 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:51.089+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:51.471 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:51 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:51.089+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:51.471 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:51 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:51.089+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:51.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:51 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:51.334+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:51.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:51 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:51.334+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:51.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:51 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:51.334+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:51.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:51 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:51.334+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:51.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:51 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:51.334+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:51.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:51 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:51.469+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:51.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:51 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:51.469+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:51.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:51 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:51.469+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:51.980 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:51 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:51.700+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:51.981 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:51 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:51.700+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:51.981 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:51 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:51.700+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:51.981 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:51 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:51.700+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:51.981 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:51 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:51.700+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:52.272 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.7.service' 2023-10-08T09:32:52.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:32:52.311 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-08T09:32:52.311 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-08T09:32:52.311 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.8 2023-10-08T09:32:52.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:52.113+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:52.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:52.113+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:52.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:52.113+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:52.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:52.113+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:52.437 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 systemd[1]: Stopping Ceph osd.8 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:32:52.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:52 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:52.341+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:52.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:52 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:52.341+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:52.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:52 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:52.341+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:52.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:52 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:52.341+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:52.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:52 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:52.341+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:52.713 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:52.434+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:52.713 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:52.434+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:52.713 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:52.434+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:52.997 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:52.711+0000 7f3c36d7a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:32:52.997 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:52.711+0000 7f3c36d7a700 -1 osd.8 94 *** Got signal Terminated *** 2023-10-08T09:32:52.997 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:52 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:52.711+0000 7f3c36d7a700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:32:53.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:52 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:52.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:53.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:52 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:52.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:53.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:52 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:52.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:53.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:52 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:52.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:53.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:52 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:52.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:53.469 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:53.133+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:53.470 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:53.133+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:53.470 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:53.133+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:53.470 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:53.133+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:53.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:53 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:53.297+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:53.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:53 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:53.297+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:53.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:53 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:53.297+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:53.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:53 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:53.297+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:53.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:53 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:53.297+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:53.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:53.468+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:53.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:53.468+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:53.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:53.468+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:53.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:53 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:53.468+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:32:54.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:53 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:53.674+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:54.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:53 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:53.674+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:54.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:53 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:53.674+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:54.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:53 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:53.674+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:54.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:53 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:53.674+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:54.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:54.091+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:54.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:54.091+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:54.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:54.091+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:54.497 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:54.091+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:54.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:54 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:54.279+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:54.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:54 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:54.279+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:54.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:54 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:54.279+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:54.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:54 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:54.279+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:54.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:54 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:54.279+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:54.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:54.502+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:54.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:54.502+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:54.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:54.502+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:54.998 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:54 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:54.502+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:32:55.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:54 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:54.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:55.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:54 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:54.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:55.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:54 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:54.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:55.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:54 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:54.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:55.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:54 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:54.696+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:55.492 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:55.079+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:55.492 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:55.079+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:55.492 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:55.079+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:55.493 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:55.079+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:55.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:55 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:55.235+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:55.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:55 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:55.235+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:55.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:55 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:55.235+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:55.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:55 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:55.235+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:55.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:55 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:55.235+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:55.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:55.491+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:55.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:55.491+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:55.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:55.491+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:55.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:55.491+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:32:56.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:55 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:55.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:56.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:55 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:55.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:56.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:55 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:55.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:56.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:55 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:55.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:56.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:55 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:55.738+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:56.481 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:56.050+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:56.481 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:56.050+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:56.481 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:56.050+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:56.481 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:56.050+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:56.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:56 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:56.262+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:56.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:56 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:56.262+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:56.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:56 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:56.262+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:56.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:56 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:56.262+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:56.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:56 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:56.262+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:56.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:56.479+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:56.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:56.479+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:56.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:56.479+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:56.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:56 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:56.479+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:32:57.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:56 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:56.739+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:57.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:56 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:56.739+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:57.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:56 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:56.739+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:57.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:56 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:56.739+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:57.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:56 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:56.739+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:57.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:57.004+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:02.225084+0000 front 2023-10-08T09:32:02.225005+0000 (oldest deadline 2023-10-08T09:32:25.125059+0000) 2023-10-08T09:32:57.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:57.004+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:10.426322+0000 front 2023-10-08T09:32:10.426349+0000 (oldest deadline 2023-10-08T09:32:35.726517+0000) 2023-10-08T09:32:57.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:57.004+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:15.727359+0000 front 2023-10-08T09:32:15.727275+0000 (oldest deadline 2023-10-08T09:32:40.427269+0000) 2023-10-08T09:32:57.436 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:57.004+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:22.728396+0000 front 2023-10-08T09:32:22.728374+0000 (oldest deadline 2023-10-08T09:32:48.028478+0000) 2023-10-08T09:32:57.437 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8[125971]: 2023-10-08T09:32:57.004+0000 7f3c3136c700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:30.829713+0000 front 2023-10-08T09:32:30.829764+0000 (oldest deadline 2023-10-08T09:32:56.729882+0000) 2023-10-08T09:32:57.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:57 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:57.306+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:57.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:57 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:57.306+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:57.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:57 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:57.306+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:57.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:57 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:57.306+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:57.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:57 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:57.306+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:57.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:57.435+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:57.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:57.435+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:57.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:57.435+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:57.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:57.435+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:32:57.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:57 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:57.435+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:32:58.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:57 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:57.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:58.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:57 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:57.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:58.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:57 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:57.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:58.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:57 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:57.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:58.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:57 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:57.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:58.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:57 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:57.760+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:32:58.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:58 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:58.290+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:58.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:58 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:58.290+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:58.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:58 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:58.290+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:58.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:58 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:58.290+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:58.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:58 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:58.290+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:58.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:58 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:58.290+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:32:58.653 INFO:journalctl@ceph.osd.8.smithi184.stdout:Oct 08 09:32:58 smithi184 bash[148717]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-8 2023-10-08T09:32:58.654 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:58 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:58.399+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:58.654 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:58 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:58.399+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:58.654 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:58 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:58.399+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:58.654 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:58 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:58.399+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:32:58.654 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:58 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:58.399+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:32:59.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:58.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:32:59.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:58.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:32:59.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:58.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:32:59.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:58.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:32:59.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:58.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:32:59.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:58 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:58.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:32:59.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:59 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:59.385+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:32:59.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:59 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:59.385+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:32:59.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:59 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:59.385+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:32:59.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:59 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:59.385+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:32:59.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:32:59 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:32:59.385+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:32:59.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:59 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:59.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:32:59.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:59 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:59.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:32:59.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:59 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:59.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:32:59.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:59 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:59.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:32:59.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:59 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:59.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:32:59.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:32:59 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:32:59.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:33:00.015 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.8.service' 2023-10-08T09:33:00.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:59 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:59.767+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:00.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:59 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:59.767+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:00.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:59 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:59.768+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:00.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:59 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:59.768+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:00.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:59 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:59.768+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:00.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:32:59 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:32:59.768+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:00.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:00.054 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-08T09:33:00.054 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-08T09:33:00.055 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.9 2023-10-08T09:33:00.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 systemd[1]: Stopping Ceph osd.9 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:00.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.264+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:33:00.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.264+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:33:00.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.264+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:33:00.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.264+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:33:00.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.264+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:33:00.382 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.264+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:33:00.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:00 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:00.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:00.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:00 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:00.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:00.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:00 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:00.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:00.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:00 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:00.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:00.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:00 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:00.380+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:00.882 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.427+0000 7fa345073700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:00.882 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.427+0000 7fa345073700 -1 osd.9 94 *** Got signal Terminated *** 2023-10-08T09:33:00.882 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:00 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:00.427+0000 7fa345073700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:33:01.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:00 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:00.800+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:01.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:00 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:00.800+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:01.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:00 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:00.800+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:01.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:00 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:00.800+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:01.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:00 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:00.800+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:01.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:00 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:00.800+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:01.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:01.289+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:33:01.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:01.289+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:33:01.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:01.289+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:33:01.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:01.289+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:33:01.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:01.289+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:33:01.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:01 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:01.289+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:33:01.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:01 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:01.341+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:01.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:01 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:01.341+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:01.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:01 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:01.341+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:01.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:01 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:01.341+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:01.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:01 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:01.341+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:02.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:01 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:01.814+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:02.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:01 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:01.814+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:02.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:01 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:01.814+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:02.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:01 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:01.814+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:02.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:01 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:01.814+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:02.297 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:01 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:01.814+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:02.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:02.247+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:33:02.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:02.247+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:33:02.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:02.247+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:33:02.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:02.247+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:33:02.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:02.247+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:33:02.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:02 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:02.247+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:33:02.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:02 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:02.384+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:02.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:02 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:02.384+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:02.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:02 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:02.384+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:02.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:02 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:02.384+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:02.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:02 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:02.384+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:03.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:02 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:02.792+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:03.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:02 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:02.792+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:03.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:02 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:02.792+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:03.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:02 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:02.792+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:03.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:02 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:02.792+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:03.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:02 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:02.792+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:03.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:03 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:03.282+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:33:03.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:03 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:03.282+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:33:03.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:03 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:03.282+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:33:03.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:03 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:03.282+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:33:03.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:03 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:03.282+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:33:03.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:03 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:03.282+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:33:03.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:03 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:03.368+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:03.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:03 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:03.368+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:03.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:03 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:03.368+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:03.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:03 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:03.368+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:03.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:03 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:03.368+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:04.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:03 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:03.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:04.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:03 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:03.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:04.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:03 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:03.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:04.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:03 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:03.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:04.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:03 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:03.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:04.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:03 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:03.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:04.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:03 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:03.744+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:04.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:04.301+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:33:04.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:04.301+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:33:04.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:04.301+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:33:04.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:04.301+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:33:04.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:04.301+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:33:04.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:04 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:04.301+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:33:04.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:04 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:04.323+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:04.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:04 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:04.323+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:04.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:04 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:04.323+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:04.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:04 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:04.323+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:04.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:04 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:04.323+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:05.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:04 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:04.791+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:05.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:04 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:04.791+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:05.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:04 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:04.791+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:05.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:04 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:04.791+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:05.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:04 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:04.791+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:05.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:04 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:04.791+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:05.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:04 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:04.791+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:05.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:05.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:51.122501+0000 front 2023-10-08T09:31:51.122683+0000 (oldest deadline 2023-10-08T09:32:19.323184+0000) 2023-10-08T09:33:05.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:05.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.024617+0000 front 2023-10-08T09:32:01.024547+0000 (oldest deadline 2023-10-08T09:32:26.324341+0000) 2023-10-08T09:33:05.632 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:05.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.525997+0000 front 2023-10-08T09:32:11.526058+0000 (oldest deadline 2023-10-08T09:32:37.425966+0000) 2023-10-08T09:33:05.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:05.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:17.426370+0000 front 2023-10-08T09:32:17.426326+0000 (oldest deadline 2023-10-08T09:32:40.226910+0000) 2023-10-08T09:33:05.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:05.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:24.228620+0000 front 2023-10-08T09:32:24.228630+0000 (oldest deadline 2023-10-08T09:32:49.428787+0000) 2023-10-08T09:33:05.633 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:05 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9[146590]: 2023-10-08T09:33:05.300+0000 7fa33f665700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:32.731045+0000 front 2023-10-08T09:32:32.731120+0000 (oldest deadline 2023-10-08T09:32:57.331482+0000) 2023-10-08T09:33:05.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:05 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:05.276+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:05.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:05 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:05.276+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:05.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:05 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:05.276+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:05.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:05 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:05.276+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:05.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:05 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:05.276+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:06.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:05 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:05.745+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:06.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:05 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:05.745+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:06.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:05 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:05.745+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:06.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:05 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:05.745+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:06.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:05 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:05.745+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:06.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:05 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:05.745+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:06.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:05 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:05.745+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:06.304 INFO:journalctl@ceph.osd.9.smithi049.stdout:Oct 08 09:33:06 smithi049 bash[185269]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-9 2023-10-08T09:33:06.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:06.286+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:06.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:06.286+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:06.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:06.286+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:06.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:06.286+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:06.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:06.286+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:06.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:06 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:06.286+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:07.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:06.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:07.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:06.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:07.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:06.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:07.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:06.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:07.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:06.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:07.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:06.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:07.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:06 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:06.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:07.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:07.246+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:07.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:07.246+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:07.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:07.246+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:07.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:07.246+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:07.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:07.246+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:07.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:07.246+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:07.672 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.9.service' 2023-10-08T09:33:07.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:07.714 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-08T09:33:07.714 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-08T09:33:07.714 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.10 2023-10-08T09:33:08.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:07.702+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:08.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:07.702+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:08.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:07.702+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:08.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:07.702+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:08.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:07.702+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:08.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:07.702+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:08.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:07.702+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:08.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:07 smithi132 systemd[1]: Stopping Ceph osd.10 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:08.547 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.063+0000 7f72fc08d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:08.547 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.063+0000 7f72fc08d700 -1 osd.10 94 *** Got signal Terminated *** 2023-10-08T09:33:08.547 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.063+0000 7f72fc08d700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:33:08.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:08.285+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:08.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:08.285+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:08.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:08.285+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:08.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:08.285+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:08.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:08.285+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:08.749 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:08 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:08.285+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:09.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.681+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:09.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.681+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:09.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.681+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:09.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.681+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:09.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.681+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:09.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.681+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:09.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:08 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:08.681+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:09.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:09 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:09.237+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:09.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:09 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:09.237+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:09.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:09 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:09.237+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:09.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:09 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:09.237+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:09.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:09 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:09.237+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:09.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:09 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:09.237+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:10.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:09 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:09.688+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:10.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:09 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:09.688+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:10.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:09 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:09.688+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:10.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:09 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:09.688+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:10.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:09 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:09.688+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:10.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:09 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:09.688+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:10.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:09 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:09.688+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:10.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:10 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:10.204+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:10.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:10 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:10.204+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:10.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:10 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:10.205+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:10.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:10 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:10.205+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:10.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:10 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:10.205+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:10.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:10 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:10.205+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:11.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:10 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:10.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:11.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:10 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:10.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:11.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:10 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:10.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:11.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:10 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:10.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:11.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:10 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:10.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:11.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:10 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:10.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:11.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:10 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:10.712+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:11.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:11.174+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:11.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:11.174+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:11.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:11.174+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:11.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:11.174+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:11.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:11.174+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:11.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:11 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:11.174+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:12.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:11 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:11.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:12.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:11 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:11.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:12.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:11 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:11.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:12.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:11 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:11.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:12.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:11 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:11.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:12.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:11 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:11.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:12.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:11 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:11.749+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:12.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:12 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:12.187+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:12.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:12 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:12.187+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:12.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:12 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:12.187+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:12.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:12 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:12.187+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:12.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:12 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:12.187+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:12.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:12 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:12.187+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:12.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:12 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:12.187+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:13.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:12 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:12.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:53.646936+0000 front 2023-10-08T09:31:53.646837+0000 (oldest deadline 2023-10-08T09:32:18.346901+0000) 2023-10-08T09:33:13.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:12 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:12.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:03.648209+0000 front 2023-10-08T09:32:03.648184+0000 (oldest deadline 2023-10-08T09:32:28.348404+0000) 2023-10-08T09:33:13.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:12 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:12.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:08.348961+0000 front 2023-10-08T09:32:08.349038+0000 (oldest deadline 2023-10-08T09:32:33.649000+0000) 2023-10-08T09:33:13.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:12 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:12.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6814 osd.3 since back 2023-10-08T09:32:13.649536+0000 front 2023-10-08T09:32:13.649706+0000 (oldest deadline 2023-10-08T09:32:43.650130+0000) 2023-10-08T09:33:13.047 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:12 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:12.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.650523+0000 front 2023-10-08T09:32:23.650542+0000 (oldest deadline 2023-10-08T09:32:47.750622+0000) 2023-10-08T09:33:13.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:12 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:12.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:31.852040+0000 front 2023-10-08T09:32:31.852144+0000 (oldest deadline 2023-10-08T09:32:57.652464+0000) 2023-10-08T09:33:13.048 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:12 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10[132574]: 2023-10-08T09:33:12.715+0000 7f72f567d700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.153367+0000 front 2023-10-08T09:32:41.153454+0000 (oldest deadline 2023-10-08T09:33:03.453232+0000) 2023-10-08T09:33:13.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:13.233+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:13.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:13.233+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:13.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:13.233+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:13.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:13.233+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:13.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:13.233+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:13.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:13.233+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:13.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:13 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:13.233+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:13.954 INFO:journalctl@ceph.osd.10.smithi132.stdout:Oct 08 09:33:13 smithi132 bash[146198]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-10 2023-10-08T09:33:14.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:14.202+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:14.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:14.202+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:14.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:14.202+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:14.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:14.202+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:14.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:14.202+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:14.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:14.202+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:14.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:14 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:14.202+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:15.217 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.10.service' 2023-10-08T09:33:15.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:15.258 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-08T09:33:15.258 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-08T09:33:15.259 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.11 2023-10-08T09:33:15.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.235+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:15.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.235+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:15.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.235+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:15.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.235+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:15.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.235+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:15.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.235+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:15.499 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.235+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:15.499 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 systemd[1]: Stopping Ceph osd.11 for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:15.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.616+0000 7fa95631e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:15.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.616+0000 7fa95631e700 -1 osd.11 94 *** Got signal Terminated *** 2023-10-08T09:33:15.997 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:15 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:15.616+0000 7fa95631e700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-08T09:33:16.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:16 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:16.221+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:16.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:16 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:16.221+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:16.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:16 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:16.221+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:16.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:16 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:16.221+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:16.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:16 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:16.221+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:16.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:16 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:16.221+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:16.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:16 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:16.221+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:17.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:17.193+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:17.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:17.193+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:17.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:17.193+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:17.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:17.193+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:17.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:17.193+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:17.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:17.193+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:17.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:17 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:17.193+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:18.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:18.171+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:18.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:18.171+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:18.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:18.171+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:18.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:18.171+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:18.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:18.171+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:18.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:18.171+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:18.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:18 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:18.171+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:19.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:19.218+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:19.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:19.218+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:19.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:19.218+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:19.497 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:19.218+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:19.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:19.218+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:19.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:19.218+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:19.498 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:19 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:19.218+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:20.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:20.253+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6806 osd.0 since back 2023-10-08T09:31:50.565952+0000 front 2023-10-08T09:31:50.565964+0000 (oldest deadline 2023-10-08T09:32:16.966341+0000) 2023-10-08T09:33:20.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:20.253+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6804 osd.1 since back 2023-10-08T09:32:01.568225+0000 front 2023-10-08T09:32:01.568241+0000 (oldest deadline 2023-10-08T09:32:26.868259+0000) 2023-10-08T09:33:20.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:20.253+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6804 osd.2 since back 2023-10-08T09:32:11.569484+0000 front 2023-10-08T09:32:11.569130+0000 (oldest deadline 2023-10-08T09:32:35.669424+0000) 2023-10-08T09:33:20.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:20.253+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6812 osd.4 since back 2023-10-08T09:32:23.271456+0000 front 2023-10-08T09:32:23.271386+0000 (oldest deadline 2023-10-08T09:32:53.272247+0000) 2023-10-08T09:33:20.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:20.253+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.184:6812 osd.5 since back 2023-10-08T09:32:33.272751+0000 front 2023-10-08T09:32:33.272778+0000 (oldest deadline 2023-10-08T09:32:56.673455+0000) 2023-10-08T09:33:20.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:20.253+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.49:6822 osd.6 since back 2023-10-08T09:32:41.374558+0000 front 2023-10-08T09:32:41.374453+0000 (oldest deadline 2023-10-08T09:33:05.975020+0000) 2023-10-08T09:33:20.748 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:20 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11[134756]: 2023-10-08T09:33:20.253+0000 7fa95010f700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.132:6820 osd.7 since back 2023-10-08T09:32:48.276152+0000 front 2023-10-08T09:32:48.276138+0000 (oldest deadline 2023-10-08T09:33:11.776130+0000) 2023-10-08T09:33:21.747 INFO:journalctl@ceph.osd.11.smithi184.stdout:Oct 08 09:33:21 smithi184 bash[148989]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-osd-11 2023-10-08T09:33:22.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@osd.11.service' 2023-10-08T09:33:22.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:22.939 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-08T09:33:22.939 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-08T09:33:22.939 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.a 2023-10-08T09:33:23.308 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:33:22 smithi049 systemd[1]: Stopping Ceph mds.a for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:23.632 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:33:23 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-a[155168]: 2023-10-08T09:33:23.306+0000 7fb4ea60b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:23.632 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:33:23 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-a[155168]: 2023-10-08T09:33:23.306+0000 7fb4ea60b700 -1 mds.a *** got signal Terminated *** 2023-10-08T09:33:26.570 INFO:journalctl@ceph.mds.a.smithi049.stdout:Oct 08 09:33:26 smithi049 bash[185529]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-a 2023-10-08T09:33:26.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.a.service' 2023-10-08T09:33:26.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:26.802 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-08T09:33:26.802 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-08T09:33:26.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.d 2023-10-08T09:33:27.132 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:33:26 smithi049 systemd[1]: Stopping Ceph mds.d for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:27.632 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:33:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-d[154214]: 2023-10-08T09:33:27.188+0000 7fb009102700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:27.632 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:33:27 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-d[154214]: 2023-10-08T09:33:27.188+0000 7fb009102700 -1 mds.d *** got signal Terminated *** 2023-10-08T09:33:31.055 INFO:journalctl@ceph.mds.d.smithi049.stdout:Oct 08 09:33:30 smithi049 bash[185669]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-d 2023-10-08T09:33:31.252 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.d.service' 2023-10-08T09:33:31.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:31.287 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-08T09:33:31.287 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-08T09:33:31.287 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.g 2023-10-08T09:33:31.632 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:33:31 smithi049 systemd[1]: Stopping Ceph mds.g for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:32.132 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:33:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g[154775]: 2023-10-08T09:33:31.673+0000 7f1ea04e7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:32.132 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:33:31 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g[154775]: 2023-10-08T09:33:31.673+0000 7f1ea04e7700 -1 mds.g *** got signal Terminated *** 2023-10-08T09:33:34.632 INFO:journalctl@ceph.mds.g.smithi049.stdout:Oct 08 09:33:34 smithi049 bash[185806]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-g 2023-10-08T09:33:34.726 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.g.service' 2023-10-08T09:33:34.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:34.761 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-08T09:33:34.761 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-08T09:33:34.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.j 2023-10-08T09:33:35.132 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:33:34 smithi049 systemd[1]: Stopping Ceph mds.j for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:35.632 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:33:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-j[153846]: 2023-10-08T09:33:35.152+0000 7ff9c5909700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:35.633 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:33:35 smithi049 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-j[153846]: 2023-10-08T09:33:35.152+0000 7ff9c5909700 -1 mds.j *** got signal Terminated *** 2023-10-08T09:33:36.882 INFO:journalctl@ceph.mds.j.smithi049.stdout:Oct 08 09:33:36 smithi049 bash[185944]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-j 2023-10-08T09:33:37.063 DEBUG:teuthology.orchestra.run.smithi049:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.j.service' 2023-10-08T09:33:37.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:37.099 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-08T09:33:37.099 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-08T09:33:37.099 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.b 2023-10-08T09:33:37.452 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:33:37 smithi132 systemd[1]: Stopping Ceph mds.b for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:37.797 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:33:37 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: 2023-10-08T09:33:37.450+0000 7fc62cce0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:37.797 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:33:37 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b[139423]: 2023-10-08T09:33:37.450+0000 7fc62cce0700 -1 mds.b *** got signal Terminated *** 2023-10-08T09:33:41.797 INFO:journalctl@ceph.mds.b.smithi132.stdout:Oct 08 09:33:41 smithi132 bash[146456]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-b 2023-10-08T09:33:41.914 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.b.service' 2023-10-08T09:33:41.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:41.949 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-08T09:33:41.949 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-08T09:33:41.949 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.e 2023-10-08T09:33:42.297 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:33:42 smithi132 systemd[1]: Stopping Ceph mds.e for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:42.797 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:33:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-e[138745]: 2023-10-08T09:33:42.333+0000 7f02c9f56700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:42.797 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:33:42 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-e[138745]: 2023-10-08T09:33:42.333+0000 7f02c9f56700 -1 mds.e *** got signal Terminated *** 2023-10-08T09:33:46.777 INFO:journalctl@ceph.mds.e.smithi132.stdout:Oct 08 09:33:46 smithi132 bash[146595]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-e 2023-10-08T09:33:46.783 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.e.service' 2023-10-08T09:33:46.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:46.819 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-08T09:33:46.819 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-08T09:33:46.819 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.h 2023-10-08T09:33:47.047 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:33:46 smithi132 systemd[1]: Stopping Ceph mds.h for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:47.547 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:33:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-h[139084]: 2023-10-08T09:33:47.197+0000 7f23d680b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:47.547 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:33:47 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-h[139084]: 2023-10-08T09:33:47.197+0000 7f23d680b700 -1 mds.h *** got signal Terminated *** 2023-10-08T09:33:50.547 INFO:journalctl@ceph.mds.h.smithi132.stdout:Oct 08 09:33:50 smithi132 bash[146739]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-h 2023-10-08T09:33:50.749 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.h.service' 2023-10-08T09:33:50.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:50.784 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-08T09:33:50.784 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-08T09:33:50.784 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.k 2023-10-08T09:33:51.047 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:33:50 smithi132 systemd[1]: Stopping Ceph mds.k for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:51.547 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:33:51 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-k[138409]: 2023-10-08T09:33:51.163+0000 7faa0dfac700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:51.547 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:33:51 smithi132 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-k[138409]: 2023-10-08T09:33:51.163+0000 7faa0dfac700 -1 mds.k *** got signal Terminated *** 2023-10-08T09:33:54.705 INFO:journalctl@ceph.mds.k.smithi132.stdout:Oct 08 09:33:54 smithi132 bash[146879]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-k 2023-10-08T09:33:54.837 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.k.service' 2023-10-08T09:33:54.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:54.871 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-08T09:33:54.871 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-08T09:33:54.871 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.c 2023-10-08T09:33:55.238 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:33:54 smithi184 systemd[1]: Stopping Ceph mds.c for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:55.497 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:33:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-c[141603]: 2023-10-08T09:33:55.234+0000 7f1cbc69e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:55.497 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:33:55 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-c[141603]: 2023-10-08T09:33:55.234+0000 7f1cbc69e700 -1 mds.c *** got signal Terminated *** 2023-10-08T09:33:58.747 INFO:journalctl@ceph.mds.c.smithi184.stdout:Oct 08 09:33:58 smithi184 bash[149259]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-c 2023-10-08T09:33:58.846 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.c.service' 2023-10-08T09:33:58.884 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:33:58.884 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-08T09:33:58.884 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-08T09:33:58.884 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.f 2023-10-08T09:33:59.247 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:33:58 smithi184 systemd[1]: Stopping Ceph mds.f for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:33:59.747 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:33:59 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-f[141266]: 2023-10-08T09:33:59.257+0000 7fed8c0fb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:33:59.748 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:33:59 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-f[141266]: 2023-10-08T09:33:59.257+0000 7fed8c0fb700 -1 mds.f *** got signal Terminated *** 2023-10-08T09:34:02.747 INFO:journalctl@ceph.mds.f.smithi184.stdout:Oct 08 09:34:02 smithi184 bash[149404]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-f 2023-10-08T09:34:02.913 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.f.service' 2023-10-08T09:34:02.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:34:02.948 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-08T09:34:02.948 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-08T09:34:02.948 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.i 2023-10-08T09:34:03.247 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:34:03 smithi184 systemd[1]: Stopping Ceph mds.i for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:34:03.747 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:34:03 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-i[141944]: 2023-10-08T09:34:03.312+0000 7f62b11b9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:34:03.747 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:34:03 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-i[141944]: 2023-10-08T09:34:03.312+0000 7f62b11b9700 -1 mds.i *** got signal Terminated *** 2023-10-08T09:34:06.747 INFO:journalctl@ceph.mds.i.smithi184.stdout:Oct 08 09:34:06 smithi184 bash[149549]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-i 2023-10-08T09:34:06.765 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.i.service' 2023-10-08T09:34:06.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:34:06.799 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-08T09:34:06.799 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-08T09:34:06.799 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.l 2023-10-08T09:34:07.182 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:34:06 smithi184 systemd[1]: Stopping Ceph mds.l for b8694e94-65b9-11ee-8db5-212e2dc638e7... 2023-10-08T09:34:07.497 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:34:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-l[140926]: 2023-10-08T09:34:07.182+0000 7fac266d7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-08T09:34:07.497 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:34:07 smithi184 ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-l[140926]: 2023-10-08T09:34:07.182+0000 7fac266d7700 -1 mds.l *** got signal Terminated *** 2023-10-08T09:34:10.548 INFO:journalctl@ceph.mds.l.smithi184.stdout:Oct 08 09:34:10 smithi184 bash[149700]: ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7-mds-l 2023-10-08T09:34:10.751 DEBUG:teuthology.orchestra.run.smithi184:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b8694e94-65b9-11ee-8db5-212e2dc638e7@mds.l.service' 2023-10-08T09:34:10.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-08T09:34:10.784 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-08T09:34:10.785 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T09:34:11.099 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:34:12.793 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T09:34:13.099 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:34:14.692 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 --force --keep-logs 2023-10-08T09:34:15.021 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:34:16.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:34:16.639 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:34:16.677 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-08T09:34:16.708 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-08T09:34:16.710 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi049/crash 2023-10-08T09:34:16.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T09:34:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:tar: /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T09:34:16.789 INFO:teuthology.orchestra.run.smithi049.stderr:tar: Error is not recoverable: exiting now 2023-10-08T09:34:16.791 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi132/crash 2023-10-08T09:34:16.792 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T09:34:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:tar: /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T09:34:16.850 INFO:teuthology.orchestra.run.smithi132.stderr:tar: Error is not recoverable: exiting now 2023-10-08T09:34:16.852 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi184/crash 2023-10-08T09:34:16.853 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash -- . 2023-10-08T09:34:16.908 INFO:teuthology.orchestra.run.smithi184.stderr:tar: /var/lib/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-08T09:34:16.909 INFO:teuthology.orchestra.run.smithi184.stderr:tar: Error is not recoverable: exiting now 2023-10-08T09:34:16.909 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-08T09:34:16.910 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-08T09:34:16.954 INFO:teuthology.orchestra.run.smithi049.stderr:grep: /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph.log: No such file or directory 2023-10-08T09:34:16.959 INFO:tasks.cephadm:Compressing logs... 2023-10-08T09:34:16.959 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T09:34:17.003 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T09:34:17.005 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T09:34:17.042 INFO:teuthology.orchestra.run.smithi049.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T09:34:17.043 INFO:teuthology.orchestra.run.smithi049.stderr:: No such file or directory 2023-10-08T09:34:17.044 INFO:teuthology.orchestra.run.smithi132.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T09:34:17.044 INFO:teuthology.orchestra.run.smithi132.stderr:: No such file or directory 2023-10-08T09:34:17.044 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.10.log 2023-10-08T09:34:17.044 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.h.log 2023-10-08T09:34:17.044 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.e.log 2023-10-08T09:34:17.045 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.4.log 2023-10-08T09:34:17.045 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.1.log 2023-10-08T09:34:17.046 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.k.log 2023-10-08T09:34:17.046 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T09:34:17.046 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.7.log 2023-10-08T09:34:17.047 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.b.log 2023-10-08T09:34:17.047 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.b.log 2023-10-08T09:34:17.047 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.y.log 2023-10-08T09:34:17.048 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.0.log 2023-10-08T09:34:17.048 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.g.log 2023-10-08T09:34:17.049 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.a.log 2023-10-08T09:34:17.049 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.0.log: /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.6.log 2023-10-08T09:34:17.049 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.3.log 2023-10-08T09:34:17.051 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.d.log 2023-10-08T09:34:17.051 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.x.log 2023-10-08T09:34:17.051 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.9.log 2023-10-08T09:34:17.051 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T09:34:17.051 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.j.log 2023-10-08T09:34:17.051 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.a.log 2023-10-08T09:34:17.051 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-10-08T09:34:17.052 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.y.log: 93.0% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-08T09:34:17.052 INFO:teuthology.orchestra.run.smithi184.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-08T09:34:17.053 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-08T09:34:17.053 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.j.log: /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-10-08T09:34:17.053 INFO:teuthology.orchestra.run.smithi049.stderr: 94.3% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.d.log.gz 2023-10-08T09:34:17.054 INFO:teuthology.orchestra.run.smithi049.stderr: 94.3% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.j.log.gz 2023-10-08T09:34:17.060 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.5.log 2023-10-08T09:34:17.062 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.2.log 2023-10-08T09:34:17.062 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.11.log 2023-10-08T09:34:17.062 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.z.log 2023-10-08T09:34:17.062 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.8.log 2023-10-08T09:34:17.062 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.f.log 2023-10-08T09:34:17.062 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.c.log 2023-10-08T09:34:17.063 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.i.log 2023-10-08T09:34:17.063 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.c.log 2023-10-08T09:34:17.063 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log 2023-10-08T09:34:17.063 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.i.log: /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.l.log 2023-10-08T09:34:17.066 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log: /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.l.log: 93.1% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-08T09:34:17.124 INFO:teuthology.orchestra.run.smithi132.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T09:34:17.391 INFO:teuthology.orchestra.run.smithi132.stderr: 91.0% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T09:34:17.429 INFO:teuthology.orchestra.run.smithi184.stderr: 94.9% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.f.log.gz 2023-10-08T09:34:17.484 INFO:teuthology.orchestra.run.smithi184.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T09:34:17.542 INFO:teuthology.orchestra.run.smithi049.stderr: 90.1% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-08T09:34:17.567 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T09:34:17.618 INFO:teuthology.orchestra.run.smithi049.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-08T09:34:17.624 INFO:teuthology.orchestra.run.smithi184.stderr: 94.4% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.c.log.gz 2023-10-08T09:34:17.746 INFO:teuthology.orchestra.run.smithi049.stderr: 93.8% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.a.log.gz 2023-10-08T09:34:17.870 INFO:teuthology.orchestra.run.smithi184.stderr: 91.0% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-volume.log.gz 2023-10-08T09:34:18.201 INFO:teuthology.orchestra.run.smithi049.stderr: 93.4% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.g.log.gz 2023-10-08T09:34:18.518 INFO:teuthology.orchestra.run.smithi132.stderr: 93.6% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.e.log.gz 2023-10-08T09:34:18.942 INFO:teuthology.orchestra.run.smithi184.stderr: 91.8% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.c.log.gz 2023-10-08T09:34:19.710 INFO:teuthology.orchestra.run.smithi132.stderr: 91.9% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.b.log.gz 2023-10-08T09:34:19.853 INFO:teuthology.orchestra.run.smithi049.stderr: 90.4% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mon.a.log.gz 2023-10-08T09:34:21.775 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.i.log.gz 2023-10-08T09:35:22.683 INFO:teuthology.orchestra.run.smithi184.stderr: 97.9% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.l.log.gz 2023-10-08T09:35:49.836 INFO:teuthology.orchestra.run.smithi132.stderr: 97.5% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.b.log.gz 2023-10-08T09:36:13.758 INFO:teuthology.orchestra.run.smithi184.stderr: 90.8% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.11.log.gz 2023-10-08T09:36:32.945 INFO:teuthology.orchestra.run.smithi184.stderr: 90.4% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.8.log.gz 2023-10-08T09:36:45.715 INFO:teuthology.orchestra.run.smithi132.stderr: 97.8% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.h.log.gz 2023-10-08T09:37:11.374 INFO:teuthology.orchestra.run.smithi184.stderr: 90.8% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.2.log.gz 2023-10-08T09:37:16.387 INFO:teuthology.orchestra.run.smithi184.stderr: 90.6% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.5.log.gz 2023-10-08T09:37:16.431 INFO:teuthology.orchestra.run.smithi184.stderr: 2023-10-08T09:37:16.431 INFO:teuthology.orchestra.run.smithi184.stderr:real 2m59.409s 2023-10-08T09:37:16.431 INFO:teuthology.orchestra.run.smithi184.stderr:user 5m34.506s 2023-10-08T09:37:16.431 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m12.594s 2023-10-08T09:37:29.382 INFO:teuthology.orchestra.run.smithi132.stderr: 90.5% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.10.log.gz 2023-10-08T09:37:51.671 INFO:teuthology.orchestra.run.smithi049.stderr: 90.6% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.0.log.gz 2023-10-08T09:38:03.841 INFO:teuthology.orchestra.run.smithi049.stderr: 90.6% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.3.log.gz 2023-10-08T09:38:04.695 INFO:teuthology.orchestra.run.smithi049.stderr: 90.7% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.6.log.gz 2023-10-08T09:38:36.703 INFO:teuthology.orchestra.run.smithi049.stderr: 90.6% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.9.log.gz 2023-10-08T09:38:36.704 INFO:teuthology.orchestra.run.smithi049.stderr: 2023-10-08T09:38:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:real 4m19.686s 2023-10-08T09:38:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:user 6m56.483s 2023-10-08T09:38:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m13.534s 2023-10-08T09:38:58.899 INFO:teuthology.orchestra.run.smithi132.stderr: 90.4% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.1.log.gz 2023-10-08T09:39:28.022 INFO:teuthology.orchestra.run.smithi132.stderr: 97.9% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-mds.k.log.gz 2023-10-08T09:39:32.762 INFO:teuthology.orchestra.run.smithi132.stderr: 90.5% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.7.log.gz 2023-10-08T09:40:31.561 INFO:teuthology.orchestra.run.smithi132.stderr: 91.0% -- replaced with /var/log/ceph/b8694e94-65b9-11ee-8db5-212e2dc638e7/ceph-osd.4.log.gz 2023-10-08T09:40:31.583 INFO:teuthology.orchestra.run.smithi132.stderr: 2023-10-08T09:40:31.583 INFO:teuthology.orchestra.run.smithi132.stderr:real 6m14.562s 2023-10-08T09:40:31.583 INFO:teuthology.orchestra.run.smithi132.stderr:user 7m52.634s 2023-10-08T09:40:31.583 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m20.066s 2023-10-08T09:40:31.584 INFO:tasks.cephadm:Archiving logs... 2023-10-08T09:40:31.585 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi049/log 2023-10-08T09:40:31.586 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T09:41:43.126 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi132/log 2023-10-08T09:41:43.127 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T09:42:56.210 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi184/log 2023-10-08T09:42:56.211 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T09:43:54.971 INFO:tasks.cephadm:Removing cluster... 2023-10-08T09:43:54.972 DEBUG:teuthology.orchestra.run.smithi049:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 --force 2023-10-08T09:43:55.399 INFO:teuthology.orchestra.run.smithi049.stdout:Deleting cluster with fsid: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:43:56.376 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 --force 2023-10-08T09:43:57.095 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:43:58.122 DEBUG:teuthology.orchestra.run.smithi184:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b8694e94-65b9-11ee-8db5-212e2dc638e7 --force 2023-10-08T09:43:58.885 INFO:teuthology.orchestra.run.smithi184.stdout:Deleting cluster with fsid: b8694e94-65b9-11ee-8db5-212e2dc638e7 2023-10-08T09:43:59.865 INFO:tasks.cephadm:Removing cephadm ... 2023-10-08T09:43:59.866 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T09:43:59.888 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T09:43:59.912 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-08T09:43:59.944 INFO:tasks.cephadm:Teardown complete 2023-10-08T09:43:59.944 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T09:43:59.961 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T09:43:59.961 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T09:43:59.964 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T09:43:59.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T09:44:00.115 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T09:44:00.125 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T09:44:00.125 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:44:00.128 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:44:00.131 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T09:44:00.146 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2023-10-08T09:44:00.146 INFO:teuthology.orchestra.run.smithi184.stderr:bash: ntpq: command not found 2023-10-08T09:44:00.146 INFO:teuthology.orchestra.run.smithi132.stderr:bash: ntpq: command not found 2023-10-08T09:44:00.168 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:44:00.168 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-10-08T09:44:00.168 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 24 -479us[ -479us] +/- 85ms 2023-10-08T09:44:00.169 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 +95us[ +107us] +/- 59ms 2023-10-08T09:44:00.169 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +325us[ +337us] +/- 71ms 2023-10-08T09:44:00.169 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:44:00.170 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:44:00.170 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2023-10-08T09:44:00.170 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 -497us[ -490us] +/- 85ms 2023-10-08T09:44:00.171 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 7 377 22 +114us[ +121us] +/- 59ms 2023-10-08T09:44:00.171 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +314us[ +321us] +/- 71ms 2023-10-08T09:44:00.171 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:44:00.172 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T09:44:00.172 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2023-10-08T09:44:00.172 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 -473us[ -471us] +/- 85ms 2023-10-08T09:44:00.172 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 +83us[ +84us] +/- 59ms 2023-10-08T09:44:00.172 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +330us[ +330us] +/- 71ms 2023-10-08T09:44:00.172 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T09:44:00.173 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T09:44:00.183 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T09:44:00.184 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T09:44:00.197 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T09:44:00.387 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T09:44:00.575 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T09:44:00.727 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T09:44:00.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:44:00.767 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T09:44:00.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:44:00.802 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T09:44:00.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:44:00.834 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T09:44:00.845 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T09:44:00.854 INFO:teuthology.task.internal:Duration was 3133.580437 seconds 2023-10-08T09:44:00.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T09:44:00.865 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T09:44:00.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T09:44:00.868 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T09:44:00.870 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T09:44:00.934 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T09:44:00.962 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T09:44:01.002 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T09:44:01.282 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T09:44:01.282 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2023-10-08T09:44:01.282 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T09:44:01.318 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2023-10-08T09:44:01.319 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T09:44:01.343 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T09:44:01.344 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T09:44:01.380 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T09:44:01.381 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T09:44:01.383 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T09:44:01.386 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T09:44:01.476 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T09:44:01.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T09:44:01.479 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T09:44:01.481 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T09:44:01.596 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T09:44:01.613 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T09:44:01.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T09:44:01.643 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T09:44:01.671 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T09:44:01.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T09:44:01.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T09:44:01.768 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T09:44:01.772 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T09:44:01.804 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2023-10-08T09:44:01.807 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2023-10-08T09:44:01.809 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2023-10-08T09:44:01.831 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T09:44:01.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:44:01.887 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T09:44:01.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:44:01.904 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T09:44:01.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T09:44:01.920 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T09:44:01.932 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T09:44:01.934 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi049 2023-10-08T09:44:01.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T09:44:01.983 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi132 2023-10-08T09:44:01.983 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T09:44:02.028 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770/remote/smithi184 2023-10-08T09:44:02.029 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T09:44:02.068 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T09:44:02.069 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T09:44:02.071 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T09:44:02.073 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T09:44:02.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T09:44:02.131 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T09:44:02.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T09:44:02.143 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T09:44:02.143 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T09:44:02.146 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T09:44:02.148 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T09:44:02.160 INFO:teuthology.orchestra.run.smithi049.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 09:44 /home/ubuntu/cephtest 2023-10-08T09:44:02.161 INFO:teuthology.orchestra.run.smithi132.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 09:44 /home/ubuntu/cephtest 2023-10-08T09:44:02.174 INFO:teuthology.orchestra.run.smithi184.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 09:44 /home/ubuntu/cephtest 2023-10-08T09:44:02.176 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T09:44:02.187 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T09:44:02.211 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T09:44:02.237 DEBUG:teuthology.nuke:shortname: smithi049 2023-10-08T09:44:02.238 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T09:44:02.260 DEBUG:teuthology.nuke:shortname: smithi132 2023-10-08T09:44:02.261 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T09:44:02.265 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 08:44:51.684378', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T09:44:02.285 DEBUG:teuthology.nuke:shortname: smithi184 2023-10-08T09:44:02.285 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T09:44:02.288 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 08:44:51.683179', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T09:44:02.311 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-07_20:01:29-fs-wip-batrick-testing-20231007.154158-distro-default-smithi/7417770', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 08:44:51.681789', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-08T09:44:02.322 INFO:teuthology.orchestra.console.smithi049:Power off 2023-10-08T09:44:02.322 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T09:44:02.348 DEBUG:teuthology.orchestra.console.smithi049:power off output: Chassis Power Control: Down/Off 2023-10-08T09:44:02.348 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:02.370 INFO:teuthology.orchestra.console.smithi184:Power off 2023-10-08T09:44:02.371 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T09:44:02.383 INFO:teuthology.orchestra.console.smithi132:Power off 2023-10-08T09:44:02.383 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T09:44:02.393 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2023-10-08T09:44:02.393 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:02.403 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2023-10-08T09:44:02.406 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-10-08T09:44:02.406 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:02.416 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-10-08T09:44:02.428 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-08T09:44:06.404 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:07.480 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:07.488 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:07.583 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is on 2023-10-08T09:44:07.589 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-10-08T09:44:07.598 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-10-08T09:44:11.586 DEBUG:teuthology.orchestra.console.smithi049:pexpect command: ipmitool -H smithi049.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:11.594 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:11.601 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T09:44:11.696 DEBUG:teuthology.orchestra.console.smithi049:check power output: Chassis Power is off 2023-10-08T09:44:11.703 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2023-10-08T09:44:11.711 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-10-08T09:44:11.797 INFO:teuthology.orchestra.console.smithi049:Power off completed 2023-10-08T09:44:11.804 INFO:teuthology.orchestra.console.smithi184:Power off completed 2023-10-08T09:44:11.812 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-10-08T09:44:11.917 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} delegate_inos/default mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 3133.58043718338 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-10-08T09:44:11.918 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T09:44:11.996 INFO:teuthology.run:FAIL